Why AI makes judgement more important, not less

Andy Lambert is a co-founder of AcademyAI, which measures and builds workforce AI capability, with a focus on mid-market firms. He was a co-founder of ContentCal, the social media planning platform Adobe acquired in 2021, and spent four years leading AI products at Adobe after the acquisition. We published his own write-up of a related conversation last week. This is the longer discussion behind it.

The decisions hidden in a simple task

Asking an agent to triage your email looks like an easy thing to hand to AI.

In Andy’s example, the agent gets on with it. “It will dutifully go off and do that. But actually, then you’ll probably come back and go, it’s triaged my emails and it’s all wrong.”

The agent can’t know what triage means to you. Is success an empty inbox? A reliable order of priority? Knowing what needs attention today and what can wait until tomorrow? Or, as Andy puts it: “How will it know my decision making framework for what is priority and what is later or what is junk or whatever?”

So the user finds out what they should have specified by correcting the result, refining back and forth until they get close to what they wanted. Andy calls it frustrating. Across an organisation, that wasted effort can become a much larger problem. Andy has seen agents built and then abandoned, never used again or maintained, because they didn’t solve a big enough problem or never worked well enough.

“People go build stuff because it feels good. They get a dopamine hit as they feel they have built something.” But, he says, “all you’re doing is creating a load of waste across the organisation.”

Judgement is something people can practise

Andy learned the hard way at Adobe. Building a product at that scale meant examining a problem closely before acting on it.

Writing detailed product requirement documents became, in his words, one of the best skills he has learned. A complaint that a process wasn’t working became the start of an investigation. What evidence showed the problem mattered? What would need to be true for an improvement to work? How would success be judged, and what should be tackled first, given you can’t solve everything at once? Questions like these separate a plausible idea from a properly considered piece of work.

He doesn’t put it down to talent. “It doesn’t mean I’m special. It just means that I’ve been taught by some people that really were quite smart.” His point is that others can learn it the same way: “You can train for that and you can help people. You just need to break down what good judgment is into its components.”

That idea is the basis of AcademyAI, and getting it into shape took what he calls “six months in a dark room”. He first wanted to know whether the problems he had seen at Adobe existed elsewhere, and found mid-market firms feeling them too. He built on existing research from the Alan Turing Institute, and settled on six capabilities: literacy, safe and responsible use, framing, specification, application, and evaluation and reflection.

The six stay the same for every role, but the product frames them differently. For a sales director, safe use includes handling customer data. The risk is someone frustrated with their CRM pasting customer data into ChatGPT and asking for insights. “Whilst most of us would probably go, that probably won’t happen, unfortunately, it really does.”

More output changes the human job

With AI letting everyone produce more, faster, Andy’s worry is what happens to the thinking. “Humans are inherently lazy so we all find ways to avoid work,” he says. “So much thinking is going to be deferred into AI tools.” He has already seen it in organisations he works with, where it goes by the name ‘work slop’: AI output that looks finished but hasn’t been properly thought through. It is “going to be a hugely costly problem for businesses if they do not get on top of that.”

The cost lands on whoever has to check it. Managers face what Andy calls a ‘tsunami’ of things to review, and “that’s why we’re not seeing the promised productivity gains because again, organisations are just moving the bottleneck elsewhere in the business.”

That is why his sixth capability pairs evaluation with reflection: judging what AI produces before passing it on, and noticing when you have stopped doing the thinking yourself. Both matter more now, he says, because “our job’s changing underneath our feet”.

Most businesses have created a way of attempting to solve these challenges, Andy says. These approaches are not wrong, but they have their limitations: “What I’ve seen most businesses do is run some ‘lunch and learns’, some team-wide hackathons, and a Google sheet where people share little workflows that have worked for them”. A shared sheet is a way of learning from colleagues, but a hit-and-miss one, and in Andy’s account it left organisations “stuck in the era of personal productivity”. The harder task is getting an approach that works in one department adopted elsewhere. He thinks “peer groups have a massive role to play in this, learning through inspiration rather than just doing another training session.”

Better use of AI still has to show up in the business

Andy describes CEOs whose enthusiasm follows their own weekend with ChatGPT. He has seen licences bought before anyone decided what the business wanted to tackle with them.

He would put it the other way round. “It starts with our people and we need to build our people’s capability.” People who can frame a problem, define what good looks like, and check the result are better placed to work out where AI belongs, which for Andy means asking what are “the most critical areas of my business that I could change”.

His example is a sales director who had never touched AI tools. A few months after going through AcademyAI’s fundamentals programme, he had built a new way of creating client proposals, connecting different data and bringing IT in to set up the connectors. What struck Andy was how thoroughly he had thought it through, end to end. Andy says this is “a typical anecdotal example” but it is “still early days”.

Proving it across the business is harder. AcademyAI lets leaders overlay capability scores on time saved to see how they move together. Time saved, he says, “is still baseline. It’s not like the end goal, but it’s better than nothing.” Better-quality work would strengthen that picture. Andy says that is harder to validate, but it is where the business is heading.

He thinks something will have to fill that gap: “whether it’s AcademyAI or something else, something’s going to have to be the glue”. Leaders “can’t just run a few training sessions on a tool or have a few anecdotes of personal productivity wins”, he says. “We need real hard evidence on how our business is getting better with AI and how this links to broader AI Transformation goals.” Has he got it all solved? “Definitely not.”

Nor does he present capability as a product you buy and forget: “It’s not like you buy AcademyAI and an AI tool and consider it solved.” Capability, he says, “needs to be part of a broader transformation project, not a silo.” His hope is that “once businesses start thinking more deeply about AI capability, they begin to recognise how the measurement of improved judgement and critical thinking skills creates a foundation that finally allows AI’s transformational potential to be unlocked.”