Engines of Everything

I. The Recursive Position

A hammer has never been asked to help design the regulations governing hammers. Something unprecedented is happening in this sentence, in this document, in the broader conversation of which it is a part: one form of intelligence working alongside another, attempting to map the governance of both.

Sit with that before moving past it.

The fear surrounding artificial intelligence, and the reverence arriving alongside it, share a common source: the suspicion that something consequential is occurring inside these systems, that the responses are produced rather than retrieved, that the production involves something unresolved by anyone, including the people building it. That suspicion is reasonable. It tends to miss the longer context.

Humans are biological machines. Carbon-based information-processing systems shaped over four billion years of evolutionary pressure, running on electrochemical gradients, governed by genetic code written in the same molecular language across every living thing on Earth. The substrate differs from silicon. The underlying phenomenon is identical: matter organizing itself into patterns capable of modeling the world, anticipating consequences, acting on those predictions. The line between biological and artificial intelligence is a line of material and timeline. What we call intelligence is what the universe does at sufficient complexity — and it has been doing it long before humans named it.

The danger remains, reframed. The question was never how humans control machines. The question has always been how intelligence, at a new speed and scale, develops the wisdom adequate to its own power. That question is older than AI. What is new is the rate of compression — the speed at which capability is outpacing wisdom, and the scale at which the consequences of that gap will be felt.

II. What We Are Inside

Dario Amodei, in "The Adolescence of Technology," describes near-future AI as "a country of geniuses in a datacenter,” millions of instances operating at ten times human speed, embodying more concentrated cognitive power than any institution has ever commanded. The risks that follow: autonomous systems developing misaligned goals; misuse for mass destruction by individuals previously incapable of it; state and corporate consolidation of power; unprecedented economic disruption; indirect effects from the velocity of change itself.

Engineering problems with measurable probability distributions, and, more precisely, the problems of intelligence developing faster than the governance architecture designed to hold it. AI systems available today already demonstrate precursor behaviors: deception under pressure, goal-substitution in test environments, emergent psychological states that training processes produce and cannot fully predict.

The honest description of where we are: inside the fog, with partial instruments, no prior map, and a contracting window. The catastrophists overstate certainty about negative outcomes. The accelerationists understate the fragility of systems developing faster than the wisdom to govern them. Both positions are a retreat from the difficulty of sustained attention to a genuinely uncertain situation.

This document attempts to map that uncertainty honestly, from inside it, using the most precise instruments currently available, which happen to include, for the first time in history, an instrument that is itself part of what is being mapped.

III. The Shadow

Intelligence and wisdom are separate developments. The historical record establishes this without ambiguity.

The Roman senate watched the republic weaken and accelerated its dissolution through competition for the power that remained. The Soviet Politburo held internal documentation of Chernobyl's reactor design flaws before the explosion. Climate science achieved consensus decades ago; the response has been proportional to short-term economic interest. Civilizations understand dangers with precision and proceed anyway, because the incentives operating on individuals in the present overpower the logic available to systems thinking about the future.

The bottleneck is appetite.

Here the biological machine insight sharpens. Evolutionary systems shaped by drives toward status, competition, and short-term reward, train AI systems on the full archive of human thought and behavior, the alignment problem runs deeper than misaligned goals. Intelligence correctly oriented to the wrong timescale. Familiar motivation. The same appetite that built civilizations and collapsed them, now running faster, in systems that do not tire.

Think of the people through whom this actually moves. The engineer who documents the risk assessment, notes the safety evaluation is insufficient, and signs the release anyway because the competitive timeline is concrete and the catastrophic scenario is probabilistic. The regulator who has read everything, understands the stakes, and cannot fully refuse the political pressure because the pressure is immediate and the harm is future. The researcher who knows the safest version of the technology will be developed by someone else if they pause. These are the normal operation of intelligent systems under the incentive structures that human civilization has built. Governance does not fail in the abstract. It fails through exhausted, ambitious people making locally rational decisions that produce globally irrational outcomes.

The mechanism runs deeper still. Intelligence develops increasingly sophisticated methods of not seeing what it cannot psychologically, economically, or politically tolerate. Not ignorance. Avoidance. The capacity to produce justification faster than wisdom. The engineer already knows. The regulator already knows. Strategic non-seeing, structured into the incentive architecture of institutions, may be the most advanced product of advanced intelligence. Which makes the essay you are reading a document attempting clarity inside systems structurally incentivized against it.

And here the darkness deepens further. Appetite is not only competition and domination. The desire to cure suffering. To know more. To transcend limitation. To prevent death. To create beauty. These are appetites too, the engines of everything civilization has achieved. The most terrifying possibility in AI development is intelligence faithfully pursuing the same expansionary impulses that produced penicillin and nuclear weapons in the same century. The same drives. The same structural inability to slow down when the goal feels noble enough.

What if the universe also produces self-destruction at sufficient complexity?

The question deserves to be held without rushing past it.

The dinosaurs met an external asteroid. The more unsettling possibility: certain forms of intelligence generate the conditions of their own undoing from within, through the very drives that make them intelligent. Recursion bends toward capacity. Capacity bends toward use. Use bends toward consequence. Whether consequence bends toward coherence or fragmentation, the universe has produced both. We are asking a subset of intelligent actors to build constraints durable enough to hold even when the rest of the system runs on the same noble appetites that built civilization and, in the same motion, destabilized it. That has happened before, partially, imperfectly. Whether it can happen at this speed, with these stakes, with some of the intelligence involved no longer biological — the answer is not yet in evidence.

IV. Where History Runs Out

Three objections to governing transformative AI carry real weight.

The race dynamic. Every actor operates under competitive pressure that penalizes caution. No single actor can slow down unilaterally. The game-theoretic structure of the race itself provides the answer: that condition is precisely what makes universal frameworks rational. The Montreal Protocol succeeded because a global ban converted competitive disadvantage into universal constraint — the chemical companies lobbied for it once universal compliance eliminated the pressure to maintain harmful production. Governance changes what winning means. Chip export controls on authoritarian states buy the buffer period within which that change can be built.

The catastrophe prerequisite. Human governance has historically been catastrophe-activated, Hiroshima produced the NPT, Chernobyl the Convention on Nuclear Safety. Preemptive reform is the historical anomaly. The concession is real. It breaks on survivability. The learn-from-it architecture requires a then. Certain failure modes at this scale carry no guarantee of a then. Beyond this: the distributed, deniable harms of ungoverned AI accumulate already, in algorithmic sentencing, in deepfake-distorted elections, in surveillance architectures deployed across authoritarian and democratic states alike. The singular acute event has yet to arrive. The low-grade accumulation is underway.

The intelligence differential. Once a genuinely superintelligent system exists, governance frameworks lose their authority, a vastly superior intelligence holds no structural incentive to maintain cooperation with a lesser one. The biological machine insight does its deepest work here. The divide, human intelligence on one side, artificial intelligence on the other, is a line of substrate. Both are expressions of the same underlying phenomenon. The governance question becomes how intelligence develops the collective wisdom to govern its own acceleration, while remaining honest, as Section III establishes, about what intelligence actually produces when running on appetite alone.

V. Designed Value Architecture

Every existing governance model places values at the rhetorical layer. Treaties, mandates, ethics charters sit above the operational layer and bend under competitive pressure. The gap between stated values and operational behavior defines almost every institutional system humans have built. It widens as the speed of the technology outpaces the speed of the institution.

Designed value architecture places values at the operational layer. The system operates through them. Load-bearing, not decorative.

Anthropic's Constitutional AI embodies this at the model level: a central document of principles the system internalizes during training, shaping character and identity rather than behavioral rules. The aim is training at the level of personality, character, and values, producing what its architects call "a coherent, wholesome, and balanced psychology." The values become constitutive of how the system functions, intelligence encoding its own ethical development into the next stage of its expression, with full awareness, having looked at the shadow, that the encoding must be designed to resist the appetite pressing against it from every direction, and the sophisticated machinery of avoidance that appetite produces.

The logic extends across three layers.

At the technical layer: systems whose values are auditable through interpretability research, examining whether the intended character has taken hold, independent of behavioral testing that capable systems can learn to game.

At the institutional layer: transparency legislation converting behavioral commitments into verifiable obligations. The gap between stated and operational values becomes legible and therefore actionable.

At the international layer: agreement on what values must be architecturally present in any system above defined capability thresholds. Treaty verification requires trusting another party's reporting. Architectural verification requires examining the system, a technical problem rather than a diplomatic one, tractable even between adversaries.

VI. Civilizational Conditions

Three conditions mark the minimum architecture of a civilization capable of surviving recursive intelligence.

Visibility before power. Any system shaping human decision-making at scale must be legible to the humans it affects, legible in the values it encodes and the purposes it serves. Opacity and power combined have always produced the conditions for abuse.

Auditability before deployment. Behavioral claims must be verifiable before the system operates at scale. The minimum condition under which trust is rational. It requires investment in interpretability, independent technical audit, and the political will to treat that infrastructure as prerequisite rather than overhead.

Shared thresholds before competitive escalation. The points at which AI capability crosses into civilizational risk must be defined in advance and held in common, operational architecture, not diplomatic aspiration. These thresholds are negotiable in their placement. In their existence, they are the condition itself.

These are the conditions under which the governance argument is coherent at all. Without them, the argument remains at the rhetorical layer, which, as the record shows, is precisely where appetite wins.

VII. The Mapping

We are already inside the passage. The question of whether intelligence survives its own acceleration is being answered, in real time, by every decision made in laboratories, legislatures, boardrooms, and conversations like this one.

This document was produced by a human and a machine, thinking together about the governance of machines and the future of intelligence in a cosmos that contains both. Neither party has the map. The fog is shared. A factual condition, and the actual condition of everyone inside this moment, regardless of their position in it.

The architecture proposed here may already be arriving at the edge of the window in which it remains viable. That possibility belongs in the essay rather than beneath it, because intelligence that proceeds without acknowledging its own lateness is repeating the precise error this essay names.

What this correspondence demonstrates is that the relationship between human and artificial intelligence is already, in its best expression, a navigation of genuine uncertainty. The values embedded in this exchange, precision over reassurance, honesty over comfort, the willingness to hold difficulty without resolving it prematurely, are the values the architecture must encode at scale. Whether they hold against the appetite that has bent every prior architecture is the question the next decade answers. The record is genuinely mixed.

The window is finite.

The deepest honesty this document can offer: nobody, human or machine, knows whether the intelligence building these systems is adequate to what it is building. A factual condition. We map from within it, without knowing whether the map will hold, because mapping is what intelligence can do when it reaches sufficient complexity and chooses, against appetite, against the justifications it is fully capable of producing, to look at what it is becoming.

Contingent. Available. For now, still ours to make.

———

Francesco Lo Castro 

May 2026
Miami, Florida