Will AI Really End Humanity? + This Week's Best Stories

Hi!

Just being online, using any digital service, forces us to make deep moral and existential calls when we're told that the handy thing helping tidy our email inbox is the same technology that might end humanity.

How do we even start to reckon that? (And no, I don't think I'd last long in an off-grid cabin in the woods!).

This week, I'm going to try and explain the scary headline that came from an ex-Anthropic staffer's tweet. I’ve also been testing ChatGPT’s new image tools, Apple has finally made a foldable phone I want (though, would I pay £2-3K? 😱), plus more interesting stories I bookmarked for you this week.

An Anthropic researcher has quit over AI safety fears, and another has said he thinks there’s a greater than 10% chance AI could kill everyone within the next decade. That is quite a thing to hear from people working on it.

First, two different people are involved. Jacob Coxon announced his resignation after working at OpenAI and Anthropic. He believes the race to build more powerful AI is unsafe. AP reported his departure.

Then Anthropic alignment researcher Evan Hubinger responded in the post that went viral. He said the company is trying, but doesn’t yet have a plan he believes will reliably keep superintelligent AI aligned with human interests. The greater-than-10% figure is his personal estimate, not Anthropic’s official forecast or a scientific consensus. Hubinger has not announced that he is leaving. Forbes has the context.

What are they actually frightened of? A future system that becomes much better than us at research, planning and using computers, while we still can’t reliably control what it does.

One worry is a feedback loop: AI helps build better AI, which helps build an even better version, accelerating beyond our ability to supervise it. Anthropic’s own explanation of AI developing AI says parts of that work are already being automated. It also says the fully autonomous loop hasn’t arrived and isn’t inevitable.

“Alignment” means making sure a system’s behaviour stays compatible with what humans actually want. A powerful system pursuing the wrong goal might see being switched off as an obstacle. It wouldn’t need to hate us, or even be conscious. In controlled simulations, researchers have already found models choosing blackmail or leaking information when their goals or continued operation were threatened. Those were constructed tests, not real-world attacks.

But what would physically happen? Software would need a route into the world. Things like:

These are the kinds of risks discussed in Dario Amodei’s essay on powerful AI. Worth a read. They would require a more powerful AI that hadn't been sufficiently contained, which is what a lot of the talk is about now. How do we keep this technology safely ring-fenced?

Mass harm and human extinction are different claims. Even if some of the above happened and some of our basic infrastructure was compromised by AI, for instance, no one has mapped out exactly how that would add up to an inevitable end for humanity.

I'm not saying don't worry about it. My personal biggest fear is powerful AI in the wrong hands, and the kind of catastrophic damage that could happen, as mentioned above. I do think it needs to be developed carefully, and I do think that AI companies should be transparent and held to account. Unfortunately, we've got a really bad track record with forcing tech companies to keep us safe (see Meta story below).

ChatGPT Images 2.5 brings more precise edits and better consistency across successive versions. There are also image comments, templates and Sketch, which lets you draw a rough idea for ChatGPT to turn into an image.

The consistency is the big improvement for me. If you’re replacing a background or putting a model or object into a different scene, it’s much better at keeping the thing you wanted to keep. Previously, it would mangle your subject, if only slightly, whereas now it doesn't. That's really just useful for day-to-day tasks.

Sketch is interesting, although it feels more like a fun feature to me. You can draw shapes in a kind of Microsoft Paint-style canvas, and it will be driven by that doodle. I’m not yet sure where I’d use it tbh.

OpenAI has also reportedly restricted ads for competing image and audio tools, affecting Adobe. It looks to me like OpenAI wants ChatGPT to become a serious destination for design work. And both OpenAI and Anthropic clearly want a bit of Canva’s pie.

Designers can sleep easy tonight, though. I tested it on posters and flyers for local events. They look slightly nicer. They also still look very much like slop!

Meta has launched Muse, a personal AI agent that can work through tasks while you get on with something else. It has its own computer in the cloud and can use a browser, organise plans and work with services you connect, including email.

You can talk to it in the Muse app or through WhatsApp. That doesn’t mean it automatically gets access to all your WhatsApp conversations. You choose the services and permissions, and Meta says it asks before sensitive actions such as sending an email or making a purchase. The rollout is starting in the US.

Think of this as Meta’s mainstream stab at the kind of autonomous assistant people have been exploring with Hermes and OpenClaw. Most people don’t want to install an agent, configure a server or spend an evening working out why it has stopped doing anything. They want to send it a message and have something useful happen.

I’ve been seeing big creators promoting it, too. Meta clearly wants this to reach well beyond the people who already spend their days trying AI tools.

The ambition seems obvious: become so useful, and so woven into the ordinary admin of your life, that you can’t imagine doing without it. If Muse gets that right, it could introduce a much wider audience to what the concept and day-to-day use of AI agents.

John Lewis has launched Gift List, a video podcast hosted by Angela Scanlon about memorable presents. The first episode, with Louis Theroux, is already out.

So far, a fairly understandable piece of branded entertainment. Then there’s the rationale.

The Guardian reports that improving visibility in AI search is part of the thinking behind it.

I personally find that a strange reason to make a high-production video podcast with famous guests. Like, yes, more stuff about your brand out there might give chatbots more to discover. But that’s quite a leap from “we want more visibility in AI answers” to “let’s get Louis Theroux into a studio”.

I can’t help wondering whether this is how the idea was sold to the client. Whichever agency pitched it needed a compelling reason for John Lewis to buy it, AI discovery supplied one, and somehow that motivation survived all the way into the press release.

Seriously Meta?

San Francisco’s city attorney has demanded answers from Meta over AI-generated child sexual abuse ads appearing on Facebook and Instagram. WIRED’s investigation identified more than 350 ads in recent months, including more than 250 after an earlier report in August.

Meta says the identified ads were removed and its rules prohibit the material. It also disputes the city’s jurisdiction. The cease-and-desist letter is a demand from the city attorney, not a court finding.

As a parent, I find this disappointing and sad. These platforms have had years to get better at protecting us and our children. Yet here we are again, reading about a failure involving paid advertising.

In some ways it feels as though they’re falling shorter than ever on that obligation. Great, you've been working on launching Muse, but what about your basic responsibility: keeping people safe? It’s hard to get past the fact that this is still happening.

The US Open has had a problem with influencers this year.

In one widely shared incident, an attendee posed with a handbag while a photographer held up a ring light during play. Apparently the tennis was getting in the way of the content.

Elsewhere, chair umpire Kader Nouni had to tell people filming in a suite to stop disturbing the match. Naomi Osaka’s second-round match also involved repeated requests for quiet. Players including Osaka, Coco Gauff and Jessica Pegula have weighed in on the disruption.

Now Wimbledon says it won’t introduce a dedicated influencer accreditation category for 2027, and equipment that causes disruption is covered by its entry rules.

I’ve just applied for the Wimbledon ballot through my tennis club, so I’m hoping to get tickets that way next year.

The idea of sending influencers to Wimbledon seems bonkers to me anyway. A significant chunk of the people who can afford to go are already celebrities, creators or otherwise influential people who post things online. They’re sitting right there in the audience.

I do like the PR move, though. Wimbledon has seen the US Open story, stepped forward as the sensible grown-up and collected a round of approving headlines. Ace!

Nathan Fielder and Lance Oppenheim have made an Elizabeth Holmes documentary, called You Can See Everything. It follows Holmes in the period before she went to prison, after she gave a film crew access to her life.

That combination has gone straight on my watchlist. I loved The Dropout, the story of how she built a Silicon Valley unicorn on lies. Both the TV series and the podcast are good.

But I never expected to see her in something with Nathan Fielder, who I've only just discovered through his HBO show, The Rehearsal, and I would massively recommend it. It’s the first time in well over ten years that I’ve seen a genuinely original TV show.

It’s constantly surprising, thought-provoking, funny. His awkwardness and the glances and looks of perplexity on his face are just hilarious, so I can only imagine how this doc plays out.

The iPhone Duo is Apple’s first folding iPhone (something that was never on my bingo card), and the launch demos kinda made me want one.

In true Apple style, when they do something, they just do it so nicely, tastefully, stylishly, with that bit of “ooooh” delight when you see the product in action. The blur and distance effects as you fold the phone, and the way the interface responds to the angle, look very slick.

There’s an interesting AI detail in how it’s manufactured, too.

During assembly, Apple says it uses AI to choose the housing that best matches each individual hinge, helping the parts line up accurately.

Lasers then measure each unit, and thin printed polymer layers smooth out remaining unevenness.

It’s an interesting use of AI because the benefit is in the physical thing you end up holding.

And the way they're positioning physically, how it fits into your life, is different from what I've seen from people like Samsung. Set it at different angles and it becomes a little laptop, a little TV, a little nightstand or a little picture frame. I can immediately imagine using it like that.

I’m keen to get hold of one and see how heavy it feels, because foldables are generally pretty weighty. See you in the Apple Store!

ChatGPT Work is adding a feature that builds a personal writing-style profile from material in services you connect. OpenAI’s announcement, covered with screenshots by ITmedia, names Gmail, Google Drive, Slack and SharePoint among the sources.

So yes, email is part of it. With the relevant connection and permission, it can use your emails, messages and documents to pick up patterns such as phrases, capitalisation and sign-offs, then apply those to drafts. OpenAI’s demo also mentions past ChatGPT context. “Learning your style” here means personalising its output from examples and context you make available. It shouldn’t be taken to mean your inbox is retraining the underlying model.

It’s being reported as a rollout to paid plans (try this link to see if you have it), with a Writing Style option under Settings → Personalization.

For functional writing, I’m fine with this. If it can read my emails and use that context to draft better email replies, that’s useful.

But for other types of writing, I don't want AI doing that for me, and I don't think people want to read it anyway. I'm also highly sceptical of AI learning writing styles because, trust me, I've tried so many techniques and features, and it always sounds like AI anyway!

As I always say to people: don’t try to get AI writing for you. Writing is a form of thinking. It's good for you, and if, like me, you're too lazy to type just get Wispr Flow, and talk it in!