For a decade the debate about artificial intelligence has been organised around a single question: is it a threat or an opportunity? The question feels exhaustive, as if these were two roads and our task were to choose between them. They are not. The benefit and the harm are produced by the same mechanism, in the same moment. Each time an AI system performs a task for us, the result is often better than what we would have managed alone, and in that same act we do not exercise the capacity the task would have demanded. The help is real. So is the erosion. They are two descriptions of one event, which is why the greatest risk of AI is not what it does badly but what it does well.

This matters most for institutions, and it matters there in a way that is easy to miss. We tend to picture a ministry, a court, a regulator or a national science agency as a structure that uses human judgement. It is more accurate to say that an institution is human judgement, accumulated and organised: the knowledge of which argument moves which minister, which precedent governs an unusual case, which line in a budget hides a real problem. This is what the philosopher Michael Polanyi called tacit knowledge, the kind we possess but cannot fully state, and what the sociologist Harry Collins called its collective form, held not in any single person but in a community and its relationships. It is built slowly, over decades of apprenticeship and repetition; it survives in people rather than in documents; and it can be lost in a single budget cycle. Once lost, it cannot be summoned back on demand.

AI erodes this accumulated judgement along a predictable path. It enters as a productivity tool, and the gains are real: a UK government trial of 20,000 civil servants found that an AI assistant saved each of them around 26 minutes a day, close to two weeks a year. But those gains create a pressure of their own. If the same output can be produced with fewer people, the people become a cost, and the first roles to go are the junior ones whose output AI most easily matches, which are precisely the roles in which judgement is practised and passed on.

The institution does not notice, because output holds. What it has quietly done is stop manufacturing its own future judgement. In time it can still produce the briefing or the ruling, but only through the system, and it no longer holds the people who could tell when a confident, well-formatted output is wrong. When the system errs, as every system eventually does, too few remain who could catch it, and those who remain have lost the fluency that catching it requires. This is the old irony of automation: the more a system takes over, the less able the human is to step in at the rare moment it matters most. Aviation learned this and now keeps its pilots flying by hand. Most institutions adopting AI have built no such discipline.

Science advice, the work through which evidence informs policy, is the clearest case, and it is close to home in Europe. AI now runs across the whole evidence-to-decision chain, and what it threatens there is judgement of exactly the kind that does not survive in documents: knowing which framing a particular minister will hear, which evidence is contextually relevant rather than merely true, which stakeholders will quietly block a recommendation whatever the analysis says. AI is genuinely safe in the high-volume, low-judgement parts of this work — screening sources, triaging, generating forecasts that are checked against reality each day. It is not safe drafting the options in a ministerial brief, where the wording shapes what becomes politically thinkable and no ground truth arrives to correct it. The European bodies built over decades on independent assessment, in food safety, disease control and the research that underpins Union policy, are accumulated human capital, expensive to build and easy to dismantle under budget pressure and the illusion that the machine is equivalent.

Institutions erode faster than individuals, for two structural reasons. An individual who senses they are leaning too hard on a tool can feel their own diminishment and choose to stop. An institution faces the opposite pull, because substitution saves money and a balance sheet does not feel diminished. The saving is immediate and reportable; the cost, the lost capacity, is deferred, borne by a future no current decision-maker answers for. And the loss is owned by no one. The manager who automates a team hits a budget target; the executive who approves it improves the quarter. No one is accountable for whether the institution can still function without the system in five years, because that sits on no one’s objectives and in no instrument it uses.

This is the deepest part of the problem: the erosion sends no signal. Almost every measure an institution applies to itself, throughput, service levels, output, is a measure of the one thing AI reliably sustains. The dashboard stays green precisely because it is wired to the output, while the foundation gives way beneath it. Europe’s main regulatory instrument, the AI Act, assesses systems one at a time, by individual risk. That is reasonable for the harm a single system can do, and structurally blind to this harm, which comes not from any one system but from the cumulative weight of dozens of unremarkable ones across an institution’s working day. We learned in environmental policy not to judge a pollutant by testing each source in isolation, but by measuring the total load. That logic has not yet reached AI.

The answer is not to use less AI, which is neither possible nor desirable. It is to treat institutional capacity as capital that AI draws down and that must be deliberately replenished. Two measures would do most of the work. The first is to measure capacity and not only output: periodically, and by design, test what an institution’s people can still do without the system, and track the gap between their assisted and unassisted performance. Call it a dependency index. It is, remarkably, a number almost no institution records, although it is the one that would reveal the trajectory while there is still time to act on it. The second is a defended boundary, drawn deliberately and in advance, between the tasks AI may safely absorb and the judgement it must never quietly replace — rather than letting that line dissolve case by case until it is gone and no one has chosen to dissolve it.

None of this is technically hard. The hard part is the will to act on a structural warning before the damage is undeniable. The proof, if we wait for it, will arrive as institutions that fail at the hard case and find no one left who could have caught it, and as a generation of public bodies so dependent that they can no longer imagine performing the judgements they have handed away. It would be a bitter irony if, in the pursuit of efficiency, Europe consumed the very capability on which its capacity to govern depends.

Help Keep Social Europe Free for Everyone

We believe quality ideas should be accessible to all — no paywalls, no barriers. Your support keeps Social Europe free and independent, funding the thought leadership, opinion, and analysis that sparks real change.