Do human values have a shape, and can we measure it?

An AI system gets trained on human preferences that have been collected from many people, pooled, and compressed into a single score. A good deal goes missing in that compression, because people disagree with each other and individuals are inconsistent with themselves, and neither of those facts survives being averaged into one number. What comes out the other side is a value that nobody actually held, and because the compression happened all at once there is no obvious place to point at and say that is where it went wrong.

The bet this program makes is that preference data has structure worth treating geometrically rather than as a flat list, in which case the inconsistency becomes a quantity you can measure instead of noise you discard. The same framing suggests a different approach to safety constraints: rather than penalizing an unsafe action after the fact, you shape the space so the system cannot get there in the first place.

Can a language model prove what it says before it says it?

When a language model invents a fact, the usual diagnosis is that the training data was inadequate. We think the more likely explanation is structural. The model is predicting one token after another with nothing to check the result against, so it has no mechanism for distinguishing a composition that is true from one that merely sounds like the things it has seen.

So this program tries to supply the missing mechanism. Concepts and the relations that are legitimate between them get written down as a formal schema, the model assembles a small proof that a claim is well formed within it, and the text is generated from that proof. The verification moves in front of the generation instead of chasing it afterwards.

At what point does a system outrun the person supervising it?

The Control Inversion argument is that human control over a sufficiently capable system fails along five dimensions at roughly the same time. The dimension we find hardest to dismiss requires no bad behavior from the system at all: a supervisor can only issue instructions at some finite rate, a capable system generates novel situations faster than that, and the gap between the two widens on its own even when the system is doing exactly what it was asked.

We took that argument seriously enough to audit our own alignment work against it, and the audit was not flattering. Nearly everything we build is a way of measuring what a system wants, which is a different question entirely from how quickly it outruns the person supervising it. Nothing in our existing research addresses the second one. This program exists because we would rather work on the gap we found than pretend we had not found it.

When the climate changes in one specific place, what happens to the people there?

Global climate models handle the physics well and stop short of the consequence. A city can learn from them roughly how much the water will rise, which is genuinely useful and also not the thing anyone actually needs to plan around. The questions that determine what happens to a neighborhood are about who moves, who cannot afford to, and where the insurance market withdraws first.

Those questions are only tractable at a small scale, so we work on one coastal corridor, one watershed, or one fire-weather basin at a time. The physics comes in from the large models as a boundary condition, since there is no sense competing with the groups who build those. Our effort goes into the coupling between a physical change and its human consequence, and the intended output is a counterfactual specific enough for a planner to act on rather than a projection they can only read.