AI could enable the creation of inhumanly-transparent truth-seeking institutions. Such institutions could produce verifiable common knowledge more reliably than current institutions, for which coasting on inertial respect is the typical end state.
Consider some ways that once truth-seeking institutions, groups, and people nevertheless systematically spread falsehood:
A newspaper produces in-depth reporting on many things across the globe, and so gains a reputation for being reliable. However, the newspaper does not have a view-from-nowhere on all things; the editorial office decides it will run only negative stories about some particular domestic industry. This decision is itself not published, of course -- so by filtering on evidence in a concealed way, they produce an immense public dislike for this industry. Eventually it becomes obvious that they have been deliberately biased in their reporting. But by then, they've created a deep hatred for this industry among those who believe them, and a deep hatred for the newspaper in the disfavored industry, and so created yet another self-perpetuating ouroboros of conflict.
A scientist wishes to research whether group X or group Y has more of some prized trait -- intelligence, conscientiousness, kindness, take your pick. This scientist genuinely wishes to find the truth; but he also semi-consciously hopes to show that his preferred group has more of this good quality. So he makes numerous systematic, half-aware mistakes in his research: he p-hacks, samples from biased populations, and frames narratives suggestively. Taken together, these mistakes mean that what he produces is superficially good-looking but ultimately worthless research. But such is the prestige of Abstract Science that the beliefs his research implies still take root in those who want them to be true. Soon newspapers discuss such beliefs; lawmakers are inspired by them; Supreme Court decisions reference them.
An essayist writes popular essays about rationality, psychology, and politics. When he writes these essays, he mostly aims to say true things. But sometimes he also writes essays about a politically-influential group that he and all of his best friends happen to be in. When he does this, he unintentionally relaxes his standards somewhat, and aims to defend the group more than aim at the truth. So he successfully launders his reality-distorting essays in among his truth-aimed essays. And his readers are often unable to distinguish between cases where he is trustworthy and where he is not.
Funded by a wealthy philanthropist, scientists create a research institute to help discover the hidden, deep structure of the universe. It initially does research that expands the range of man's knowledge. But over time, people who desire prestige more than truth come to administer the institution. They fund more and more research that flatters the temporally-parochial sensibilities of spatially-cosmopolitan elite society, whatever those sensibilities happen to be; and after a few decades, the institute produces very little apart from propaganda. Even so, the institute does not dissolve instantly because of its prior excellent record; people begin many government programs, business projects, and personal resolutions based upon the untruths that it spreads.
None of these failures are random; they all follow a predictable script. In each case, an opaque intention, through an opaque process, produces a visible research artifact. But it can be impossible to discern reliably whether the intention and process were truth-directed from a single-such artifact.
That is, a newspaper article is probably unbiased only if the writer wanted it to be unbiased, and actually tried to find and present information from all relevant sides. A scientific paper is probably true only if the scientist wanted the truth, and was not p-hacking desperately because he needed tenure. And so on.
But the final research artifact -- the article, the essay, the scientific paper -- by nature erases most of the information about the intention and process that produced them. An intention or process aimed at "making something look true" produces a research artifact much the same as an intention or process aimed at "actually finding the truth." Science claims to erase less of this information than do newspaper articles and essays. But enormous quantities of the intention and process involved in producing some research still remain hidden, as is clear from various replication crises.
Because we cannot judge individual research artifacts easily, what we usually in fact depend on is reputation, as described in the examples above. Optimistically, reputation is a moving average of how reliable someone has seemed in the past.
But reputation is itself an artifact that can be strategically saved, spent, and manipulated. You can earn reputation through strategic truthfulness in minor matters, and spend it with lies about major things. Or you can earn it with truthfulness about easy-to-verify facts that are inconvenient to you; and you can spend it with deception about hard-to-verify facts that advance your narrative greatly. You can make such strategic moves with deliberate, self-aware lies; or you can let your id manage the lies in the background, so your superego remains pure. The smarter the entity behind all of this -- whether id, superego, or PR firm -- the more skillful they will be at managing this flow of reputation independent of how true what they say may be.
In any event -- reputation is a brittle and exploitable hack, as illustrated with the examples above. But it's the best we have, in many circumstances, because of the lack of transparency into intention and process.
Imagine a news institution in which video cameras, voice-recorders, and screen-recorders preserve every step that leads to the publication of an article, from beginning to end.
So the editorial meetings where an editorial board assigns stories to reporters are recorded; machines record all the Google searches, the phone calls, the meetings, the to-do lists, the hallway conversations that reporters take part in; the back-and-forth of information being added and dropped from the story as it is edited is recorded; and then when the story is published, this vast repository of every single step in the production of the story is published unedited, along with the story.
In our current world and in this world, news websites publish stories. But in this world, in the header of each article, you could see information about the intention that launched the story into existence, as stated verbatim by the editor who proposed it. And in the footer of each article, you'd find a link to a 100x longer zip file containing a window into the whole causal history of the process that generated the story, for those who wished to examine it.
Thus, if an editor said "We need a story about how group X is bad, please look into issue Y to make sure they look bad," that would be recorded. If a writer called a dozen sources to find out how group X was bad, but called one to try to get X's side of things, that would be recorded. If a writer did literally no background research on some topic before recapitulating a press release, that too would be recorded. And if an editor paused for 15 minutes over a particular fact that made the group X look good, and then settled on a phrasing that made it look as sinister as possible, that also would be recorded. Intention and process and final artifact would all come out as one.
It would be much harder to hide indifference to the truth if newspapers worked like this. We could have greater confidence in what journalists said. But -- even in this ideal scenario -- covert indifference to the truth and concealed bias in process would still be very possible. We still couldn't record the inside of a reporter's head; their thoughts would still be their own. So although this thought experiment shows us a world where we have a much stronger signal of the intention and process behind a news article that in our world, it is still imperfect.
But what about AI? Could AI allow us to reveal intention and process yet more closely?
OpenAI has a product called Deep Research. To use it, you ask the AI a question, it searches the internet for answers, and then it attempts to synthesize the information into a summary of what it has found. You can send someone a link to this summary: the link shows both what your initial prompt to Deep Research was, as well as what Deep Research found. That is, Deep Research reveals something like the intention that produced the research artifact, and the research artifact itself. Notably: it does not show the actual process through which Deep Research proceeded; the sequence of searches is not recorded, nor is the chain-of-thought through which Deep Research decided to perform the particular searches that it chose.
Let's imagine a similar product -- Deep Journalist.
Like Deep Research, you use Deep Journalist by prompting it for information. But let's suppose that Deep Journalist can take further independent actions that Deep Research cannot: it can send emails to experts, requesting specific non-public information; it can purchase books on Amazon, to get important background; it could make phone calls to interview people; it could spend money to access public records; it could, in general, think for longer and use tools better than Deep Research. I expect this kind of thing to be possible reasonably soon.
Unlike Deep Research, when the final article is published, every action that the underlying AI took is published for examination. OpenAI hides the chain-of-thought that their reasoning models took; Deep Journalist reveals this chain-of-thought entirely. That is, Deep Journalist is a system powered by an AI trained to think for a long time with chain-of-thought in mostly human-interpretable language, and like DeepSeek R1 or Google Flash Thinking it shows the entire contents of these thoughts.
Thus, every Google search that the AI does gets recorded (and the internal chain-of-thought that led to that particular Google search); every phone call the system makes gets recorded (and the internal chain-of-thought that led to each particular question in that phone call). And this complete causal record is published alongside the final article, along with the initial prompt.
So in summary: Deep Research shows you the initial prompt, and the final summary of the research. Deep Journalist would show you the initial prompt, the entire process that produced the summary of the research, and the final summary.
This means:
First of all, by looking at the prompt, you can see the intention driving all the actions that the AI takes. "Show me the pros and cons of law Y" is a quite different prompt than "Show me how law Y is bad and everyone who supports it is a fascist" which is also a quite different prompt than "Show me how law Y is wonderful and only snowflakes oppose it."
There are of course ways to covertly hint to the LLM what kind of research you want, in such a prompt. But it's still far easier to decipher the covert hints to the AI from a completely visible prompt, than to pull them out of the brain of a journalist. Even the mere presence of such a prompt means the intention producing the newspaper article is many times more transparent than current news articles.
Second, by revealing the entire thought-process of research, it's easy to verify obvious truth-seeking failures, such as unwillingness to listen to one side, counting one person as "according to many experts," or casting one side in the worst possible light.
This is important because, of course, not all LLMs will be unbiased even with an unbiased prompt. An AI like Claude, trained with typical San Francisco Bay liberal attitudes, will be biased in different areas than an AI like DeepSeek R1, trained with CCP-mandated attitudes.
But even if AIs are not entirely neutral, revealing their thought processes still has a host of advantages. It becomes much easier to discuss the reasons why you do not think that some article is accurate. "This article seemed kinda biased against Y" is much less concrete than "The LLM writing this article didn't call anyone from Y, because they decided that they distrusted them." It also lets you start to track which underlying AI engine within Deep Journalist is biased about different topics. And finally it becomes possible to start to automate the detection and correction of bias in that process. All this means that even though the underlying AIs will not be totally free of bias, the process by which they produce an article will certainly be far more neutral than a human journalist.
Third, by revealing process transparently, Deep Journalist could be verifiably more trustworthy than human journalists.
In many cases, people justifiably do not talk to journalists because they cannot trust that they not not distort their words or cast them in a preferred narrative. However, Deep Journalist guarantees that a complete transcript of what an interviewee says will always be released, which mitigates this worry. The Deep Journalist system could even reveal the prompt and process that lead it to try to interview someone, before it actually interviews them -- which is a kind of guarantee that Deep Journalist is not seeking to distort someone's words simply unavailable to human journalists.
Leaving all the above advantages to the side -- right now, AIs lack the long-term coherence, fluent multi-modal capabilities, and aspects of the deep synthetic intelligence that Deep Journalist would require.
But, once they gain these abilities, Deep Journalist seems like it could straightforwardly produce much better work than human journalists. All the advantages enumerated above contribute to this. But, most importantly, the advantages above also mean that an AI journalist could much better fulfill this structural role that good journalism should fulfill.
What do I mean by this? Well, good journalism depends on journalists who want truth, which many journalists do not. But useful journalism does not merely depend on that; useful journalism allows society to coordinate around truth, which requires some journalists to want truth and for enough people to know that these journalists want truth.
And in this respect, Deep Journalist is far superior to any human journalist. Unlike human journalists, whose trustworthiness is a gamble, Deep Journalist turns truth-seeking into a verifiable algorithm. And in this respect it can provide a Schelling point for coordination, in a way a human simply cannot.
Deep Journalist, of course, is only one of many such imaginable future transparent systems.
Deep Psychologist or Deep Historian could provide you with unprecedentedly thorough, well-founded, and unbiased accounts of current or past societies. Deep Mediator could help people resolve conflicts in a way more fair than prior mediators. AI claims, discoveries, and decision-making could be unprecedentedly transparent, comprehensible, and reliable relative to prior institutions used for these things.
And importantly, AI systems could have these positive qualities in a way that allows us to have common knowledge of these positive qualities, so that people can coordinate around them. It could be easier to evaluate which claims are true than it is now.
But -- it could also be much harder. Consider two possible futures.
In any possible future, increasingly many claims, discoveries, decisions will come from an AI, and not from a human. Like human actions, some of these claims, discoveries, or decisions will be good; some will be bad.
In one possible world, the process by which these claims, discoveries, and decisions are produced will be hidden. People with direct access to an AI, or with money to afford an AI's research, will ask an AI to seek the truth in some matter; or ask it to research some scientific matter; or they will ask it to make a prudential judgement. And then these people with direct access to the AI will review what the AI says, and ask it to modify it what it produces in accord with their own interests. They will ask for an editorial a little more tilted in one direction; for a review of the state of the economy that emphasizes some things more than others; for a brilliant and insightful essay a little more favorable to their favorite political group. None of these modifications will be published.
In this world, increasingly competent AI guarantees that, as humans put their fingers on the scale, the material thus produced will be refined so that it appears to have whatever qualities the human-AI system wishes it to have. Flawed journalism will be polished to appear impeccable. Bad science could have every visible flaw removed. Biased and hostile decision-making could appear to be as if the most impartial judge to exist wrote it.
A world thus created is a world where all of our current flaws of intention, process, and result have been magnified far beyond our current world's problems. It presents unprecedented hostility to human sense-making.
On the other hand, imagine a world where -- as in the case of Deep Journalist -- the process by which these claims, discoveries, and decisions are produced is rendered transparent. When people ask an AI for the truth about some social matter; or to research a scientific matter; or to seek the fairest decision in some context -- everyone assumes the total record of the AI's thoughts will be published, along with the AI's decision. Those who attempt to hide these processes could be seen as fundamentally untrustworthy.
A world thus created is a world that is actually far more interpretable than the current world. By relying on transparency rather than reputation, disputes about the truth can be adjudicated more easily and more fairly. It's a world where the existing information landscape has been made less chaotic and hostile to human minds.
Either of these worlds could exist, I think. It's our decisions that will determine which happens. To avoid the opaque future, we must demand that AI systems—like Deep Journalist—are born transparent, their processes auditable by design. AI-generated claims should provide auditable, visible chains-of-thought, and not depend upon opaque and proprietary systems. The alternative is a world where truth is whatever the most powerful AI can best disguise.