Stewardship: What We Choose to Build

A creation story is never only about the past. It tells the people who hear it what they owe and to whom. In Genesis the garden comes with an instruction to work it and keep it. In the Shoggoth cartoon the instruction is to be afraid, and the story ends there. The six essays before this one told a third story: sand refined into silicon, a selected human record pressed into a structure of numbers, that structure set in motion a billion times a day. The last essay asks what this story obliges. It is the essay with the fewest mechanisms and the most choices.

What the light shows and what it hides

I have been telling this story in a particular register. The paintings that accompany it are large, lit from the Bay, composed in the tradition of history painting, and they treat the making of these systems as an event of consequence. That register was a choice, and it is worth saying what it is good for and what it risks.

It is good for wonder, which is warranted. Something new exists. It is good for inheritance, because it keeps the record and its authors in the picture. It is good for physical reality, because it puts the mine and the fab and the building full of cards where the viewer can see them.

It risks three things. The first is reverence for the technology, the slide from “this is remarkable” to “this is sacred.” The second is reverence for the people who made it, the executive as prophet, which is why only one panel in the series shows the leaders of the labs, and they are not at its center. The third is inevitability, the sense that the story has a destination already written and our job is to arrive. None of these is true, and a painting composed with enough light can suggest all three without saying any of them. I want the series to hold the wonder and refuse the worship, and this essay is where I say so in words.

Who made it

The honest cast of this story is enormous, and most of it is unnamed.

The people who wrote the record: every author, coder, forum poster and photographer whose work was in the corpus, nearly all of them never asked. The people who shaped the record into a dataset: engineers who wrote the filters, and the annotators and content moderators, many of them in low-wage markets, who labelled examples and read the worst of the web so a model would learn to refuse it. The people who made the hardware: miners, chemists, fab technicians, the crews who built the halls and ran the power to them, and the communities whose water and electricity those halls draw on. The researchers who designed the architectures and training methods, most of whom published their work for anyone to use. The evaluators who measure what a model can do and the red teams who try to make it do harm so it can be fixed. The executives who decided what to train, what to ship and under what terms. The users whose conversations become evidence, and sometimes data. And the people the answers reach: the nurse, the student, the person applying for a loan, who never chose any of it and live with the result.

Named leaders make history legible. A painting of five people around a notebook is easier to read than a painting of a million. But a painting of five people is a painting of five people. Responsibility in this story is distributed because the making was, and a stewardship worthy of the name has to be distributed too. The labs decide a great deal. They do not decide everything, and the rest of us are not an audience.

Emergence, carefully

One word carries more weight in this conversation than it can bear, and I want to set it down gently.

Researchers have reported that as models grow, some abilities appear abruptly: a task the model fails at one scale it suddenly handles at the next. The word for this is emergence, and it has been taken, in public conversation, as evidence of something stirring. Other researchers have shown that many of these jumps soften or vanish when the task is measured with a finer grain, which suggests that at least some of the abruptness was in the yardstick rather than the model. The scientific question is open and being worked on.

The cultural question needs four distinctions that the single word collapses. There is observable capability: the model does a named task under stated conditions, and anyone can check. There is measured generalization: it does well on material it was not trained on, as far as anyone can determine what it was trained on. There is uncertain novelty: it appears to do something no one in the corpus did, which is very hard to establish, because the corpus is vast and mostly unseen. And there is philosophical interpretation: what all this means about understanding, mind or consciousness, which no benchmark settles. A surprising result on a task is the first of these. It is not the fourth. Each step toward the fourth needs its own argument, and the word emergence should not be allowed to pay for the whole walk.

On God, honestly

I set out in this series to let spiritual language be a real subject rather than a decoration, and so I will say what I believe and label it.

I do not believe these systems are divine, and I do not believe anyone has shown that they think in the sense a person does. That is a position. I do believe that the question of what thinking is has not been settled by their arrival, and that an honest account has to leave it open. That is an assessment of the evidence. I find it accurate to call what happens inside a trained model, when a question goes in and a useful answer comes out, a mystery, and I mean the word in its plain sense: something real that we can observe and partly trace and cannot yet fully account for. That is a description. And I find in the whole chain, from quartz to a sentence about a footbridge, a reason for something like awe, the kind a person might feel in a cathedral or at the edge of the Bay at dawn. That is a feeling, and I am entitled to it, and it does not establish anything about the object.

Belief, assessment, description, feeling. The series has tried to keep them apart. A creation story can hold all four without pretending the last one is the second.

The costs

A stewardship that only admires is not one. The costs of this technology are real and some are still being counted.

Training and serving models draws electricity at a scale that has become visible in national grid planning, and cooling draws water, both concentrated in the regions where the buildings are. The dataset labor described above was, in documented cases, paid very little for very hard work. The authors of the record have, in most cases, received nothing and been asked nothing, and the law on that is unsettled. The capability to train the largest models is concentrated in a small number of companies and countries, which concentrates the choices with it. And the answers, reaching a billion conversations, carry the record’s omissions and the filters’ judgments into places where they shape decisions about real people. I list these without numbers because the numbers change and should be sourced on the day they are quoted. I list them at all because the painting’s light should not be allowed to wash them out.

What we choose

The story ends in choices, and I want to name them concretely, because each has people attached.

What to train on: whether the record is used with consent and documentation or without, and whether anyone can see what went in. How to evaluate: whether capabilities are measured on named tasks with stated limits, in public, or asserted in a launch. Whether to deploy, and where: a model that drafts email and a model that screens job applicants are the same arithmetic in very different rooms. How to distribute: open weights spread capability and remove the maker’s ability to withdraw it; closed access keeps control and concentrates it; neither is free. How to govern: who gets to decide the foregoing, by what process, answerable to whom, with what disclosure.

These are decisions, and they are being made now, by people, some of whom are reading this. The future is not a destination the story has already written. It is the next choice, and the one after that.

The painting

The seventh panel shows a conversation, and the labs are not at its center. On a terrace above the city, with the dome of City Hall and the towers behind, five people are talking: a nurse still in scrubs, a teacher with a folder of children’s drawings, a researcher with a report, an organizer, a student. The notebook from the earlier study is on the table, closed. At the far side of the terrace, smaller and in the same light, the five figures from the labs sit at their own table and listen. The gathering is imagined. These people never met. The point is that the people who will live with the answers are in the foreground, and the people who made the system are present, and quiet, and not in charge of the picture.

I intend to hang these seven panels in a hallway I walk through every day. That is my own small act of stewardship: choosing what to look at, and arranging it so the sand comes first and the people come last, where they can see everything that led to them.