The AI experts I trust most are the ones who make it slightly inconvenient to trust them.
They show the prompt, the output, the cost, what broke and what they changed. They don't simply announce that a model is astonishing, terrifying or about to reorganise civilisation before lunch.
That's my filter now. Not confidence. Receipts.
A following isn't a method
AI commentary has a nasty incentive problem. Calm, qualified work takes time. Absolute certainty gets a clip, a course and a speaking slot. The person telling you to stop and test the thing benefits less than the person shouting that your company will be obsolete by Thursday.
Call me cynical, but if every tool is a breakthrough and every week changes everything, the content business may be working better than the AI.
This doesn't mean useful voices must be neutral. Neutrality is often just timidity with good stationery. I want a point of view. I also want to see where the fact ends and the performance begins.
Follow the work, not the wardrobe
Simon Willison is worth reading because he publishes the workings. He shows prompts, outputs, timings, costs and failures, then tells you what he thinks. In one recent review, he ran a 17 GB model on his own laptop and noted that one image took 21 minutes. That's more useful than a leaderboard crown because it tells a normal buyer what the experience actually involves.
He also uses a deliberately silly test involving a pelican riding a bicycle. A pelican on a bicycle isn't a business process, unless your strategy off-site went particularly badly. The point is that a repeated odd task can reveal differences in price, speed and behaviour, provided nobody mistakes the bird for science.
Ethan Mollick is useful for a different reason. He puts AI into recognisable work and describes the setup. A class of executive MBA students used several tools to build working prototypes in four days, despite few of them having coded before. Crucially, he also says the prototypes weren't working startups.
That distinction could do with its own fire alarm. AI can make the demo cheap. It hasn't volunteered to find customers, collect the cash, answer complaints and remember the tax return. Apparently the machine intelligence prefers product launches too.
The correction is the product
The people worth following don't merely publish a result. They publish the reason the next result may be better.
METR's earlier experiment found that experienced open-source developers took 19% longer with AI tools. Its later work ran into serious selection problems: some developers didn't want to work without AI, some avoided submitting tasks they expected AI to handle well, and people running several agents made time measurement messy. METR's conclusion was refreshingly awkward. The newer data was an unreliable signal, so the study design needed to change.
Nothing makes an online prophet sweat like the sentence, "our data is a bad proxy". Yet that's exactly the sentence that earns trust. A correction isn't a failure of expertise. It's the evidence that expertise is present.
Epoch AI makes the same point from the benchmark side. On a widely used coding test, changing the software wrapped around the model altered results by as much as 11% for one model and 15% for another. That wrapper supplies the prompts, tools and operating environment. In plain English, the same runner can post a different time when you change the shoes, track and support crew.
So when a lab waves a benchmark table, remember who benefits. The lab gets a headline. The buyer gets a number stripped of the setup that produced it. A benchmark isn't a lie, but it can be an invoice wearing a lab coat.
My test for AI experts
I don't need every useful AI voice to agree with me. I need them to leave a trail. What did they test? What did it cost? What failed? What part is measurement, and what part is their judgement? Have they corrected anything in public?
A person who never changes their mind in AI either isn't paying attention or has found a very durable business model in certainty. Neither helps a founder deciding where to spend time and money.
The best AI expert isn't the loudest person predicting the future. It's the one who leaves enough evidence for you to disagree before you spend.
Who do you still follow because they show their workings, and who have you unfollowed for selling the weather?