The You AI
Building A Crawler To Index The Human Mind
In the same way that Google built a crawler to index the web, YouAi is building a crawler to index the human mind.
Comprehensive digitization of our tastes, preferences, identities—and everything else that makes us who we are—is a critical missing piece required to unlock the true value of AI. Doing it effectively has the potential to massively reframe our relationship with our data. For more, subscribe here and sign up for the waitlist at youai.ai.
—S
1. Introduction
Fix upon [life] in one of its more exquisite intervals, the moment, for example, of delicious recoil from the flood of water in summer heat. What is the whole physical life in that moment but a combination of natural elements to which science gives their names? Our physical life is a perpetual motion of them—the passage of the blood, the waste and repairing of the lenses of the eye, the modifications of the tissues of the brain under every light and sound…
—Walter Pater, The Renaissance (1873)
I have always been fascinated by the ways in which layered abstractions of logic and structure give rise to creativity. Take books, for example. At their core are individual letters: 26 unique, meaningless, precisely-drawn shapes. Start arranging these shapes together—in English, there are only about 170,000 valid combinations—and they become words, each with their own sets of rules, conjugations, declensions, and so on. Still another layer of composition brings us sentences. From sentences come paragraphs, then chapters, dialogues, characters, plots.
Zoom in and all you see are shapes. Zoom out and you realize that those shapes are, in fact, not shapes but stories. At a certain point, when you’ve arranged enough shapes together in valid combinations, you get art and creativity.
Where, precisely, do the rules stop, and the art begins?
2. ChatGPT Is What We’ve All Imagined AI Would Look Like
In the past few months, AI, and specifically ChatGPT, has taken the world by storm. Friends who have no interest in technology are talking about it and using it regularly. The person sitting next to me on a recent flight from San Diego to New York had it open on their laptop. I’ve never seen something that is, effectively, an unpolished technical demo, find this much mainstream traction and penetration.
I believe a large part of this is due to the fact that ChatGPT is the first productized implementation of modern AI that aligns with how pop culture has generally imagined AI to look, feel, and behave. When you ask someone to imagine artificial intelligence, the image in their mind typically takes the form of the ability to have a conversation with a computer—to ask questions and get responses. It’s certainly much easier to get excited about a computer you can talk to than it is to get excited about the machine learning model that generates your YouTube recommendations, or the advanced AI technology that filters spam out of your email inbox.
This is not to be at all dismissive of ChatGPT. It has clearly and undeniably ushered in a new era of computing, especially in terms of radically advancing consumer perceptions around AI. It knows, effectively, everything about the world. But it’s only the beginning.
3. Large Language Models Know Everything About The World, But They Don’t Know Anything About You
The critical missing piece is that, even though these models know everything—from the sense that they have consumed, and can remember, an incomprehensibly large amount of data—they don’t know anything about you. ChatGPT, for example, has no idea how to personalize its replies to the person who is prompting it.
So when you tell it something like, “explain quantum mechanics to me,” what you get back is the model’s lowest common denominator response—something that will be accessible to the average person. This is, of course, amazing. But the correct response for me should not be the same response as if that question were being asked by someone with a background in physics, or if it were being asked by a child, or by someone who studied physics in college in the 90s but who didn’t go on to work in the field, and so would be more interested in how things have changed from a point in time as opposed to general, foundational knowledge.
We can affect this behavior by including additional context in our prompts. For example, you could say, “I am five years old. Explain quantum mechanics to me,” and receive a more nuanced and appropriate answer. But this is not the correct solution. It is tedious and imprecise, and we have no way of knowing exactly what pieces of context are helpful in answering a question. Am I supposed to start every session with ChatGPT by writing up a list of facts about myself? What should I include? How deep should I go?
4. What If AI Did Know Everything About You?
Imagine if ChatGPT did know everything about you. I mean everything. Facts about your life. Your tastes, your preferences, your desires. What you were doing on April 29, 2022. The worst movie you’ve ever seen. Your favorite color. Your deepest fears.
It would certainly be able to explain quantum mechanics to you at the right level of detail. But what other kinds of things could you ask it?
You could ask, for example, what should I do tonight? What movie should I watch? Who is my soulmate? What should I do with my life?
Most importantly, if it held within it the entire context of your life, it would be able to deliver on one of the most powerful promises of AI: it would be able to bring you things before you ask for them. But how would you build something like this? Where would you start?
5. Current Models Are Only Trained On The Outputs Of Humanity
The Borges short story Funes the Memorious (which is brief enough that if you have not encountered it before, you should take a moment to read it at this link) describes a human with an extraordinary—perfect—memory. Borges’s narrator writes of Funes,
He knew by heart the forms of the southern clouds at dawn on the 30th of April 1882, and could compare them in his memory with the mottled streaks on a book in Spanish binding he had only seen once and with the outlines of the foam raised by an oar in the Rio Negro the night before the Quebracho uprising...With no effort, he had learned English, French, Portuguese and Latin. I suspect, however, that he was not very capable of thought. To think is to forget differences, generalize, make abstractions. In the teeming world of Funes, there were only details, almost immediate in their presence.
There have been countless articles and videos produced about how generative AI and large language models like ChatGPT actually work, and I am assuming the reader of this piece to be grounded in a general familiarity with the concepts at hand. With that being said, I would like to explore some specific areas as they are necessary to develop the argument that follows.
If you want to build a generative AI, you need training data. A lot of training data. In fact, the more data you have, the better. For something like ChatGPT, that means finding as much text as you can. Books, articles, blog posts, internet comments. For something that outputs source code, like Github’s Copilot, you would want to get as much code as you can find. If you want to generate art, you need as much art as you can collect. You get the point. There’s a ton of cutting-edge computer science and crazy math that then happens, but the gist of it is that you’re feeding all of this data into a machine and then asking it to make more of it.
Let’s continue using ChatGPT as an example. It has, effectively, read everything there is to read. And because it has read everything, it’s able to construct sentences by predicting a stream of words and phrases. It knows how to make sentences that are valid and make sense. From this lens, you could imagine a completed chat session as dialogue in a story, and ChatGPT not as a character in the story, but as its reader. The story might go something like,
“How does the sun work,” the person asked.
“The sun [XXXXXXX],” you replied.
“Tell me more,” they said.
“Okay, [XXXXX],” you replied.
You would then feed this dialogue into your model and ask it to fill in the blanks, to predict the unredacted version of the text. This analogy is closer to what is actually happening when you engage with ChatGPT—except you’re taking turns, doing it line-by-line, instead of all at once. It’s not really replying to you. Rather, it’s simply guessing, very effectively, at how the story should continue to unfold.
This is key to the argument that follows because it is important to understand that ChatGPT and other large language models have no conception of why they are constructing sentences as they do, or even the meaning behind the content of those sentences. They have simply read and remembered billions and billions of sentences, and it turns out that if you do that, you can predict the progression of text.
6. Expression Is An Effect Of Thought
The way ChatGPT works, when pared down in this way, is clearly not at all the way the human mind works. When we speak or write, it is because we first have a thought or an impulse, and words are simply the way in which we express it. This is true even in cases where it might feel as though those two events happen so imperceptibly fast—your mouth getting ahead of your brain—that they might be perceived as a single event.
Indeed, language is an incredibly crude and imprecise reflection of interiority. To encode a thought, or a series of thoughts, into language, is to introduce abstraction, transforming the impulses of neurons into symbols to be consumed by others, who then introduce their own abstraction when re-encoding language into their own thoughts. If you forget the word for something, or you don’t have the words to name a complex feeling, you still know those things. It’s not as if we can only engage with the world through spoken words. Before all of these things exist in words, they exist in thought.
Sometimes we think about things for a long time before we express them. Sometimes we may never express things directly in words, but the thoughts affect our behaviors or demeanors, or they affect other—seemingly unrelated—words and expressions. If we are feeling lovesick, an unrelated conversation at work might unfold in a radically different way. A single sentence, finally spoken, might be the result of the building up of years and years of thoughts and experiences.
Words, art, code, and all these other things are externalized expressions of thought. The outputs we see—the painting, or the article, or the conversation with a friend—are but abstractions of the pure thoughts, the raw impulses. To look at these outputs and generate more, as today’s modern generative AI does, is powerful, amazing, and—as we have seen—incredibly useful for many things. But to focus on these outputs is to miss the entire other side of the equation.
7. What If We Trained A Model On Impulse?
If we adopt a reductive view for the sake of argument, we can define the human as that which turns thoughts, impulses, and interiority into words, into art, into life.
With that, imagine a new kind of model—one trained not on the outputs of humans, but on the impulses that give rise to those outputs. Instead of being focused on generating more outputs, it generates more internal motions, of which output and expression are then an occasional side effect. Such a model would align much more closely with the way in which life works.
The reason this model does not yet exist is, quite simply, that the data required to train it has never existed.
We have massive, publicly-accessible corpuses of books, internet comments, news articles, research papers, art, music, poetry, and almost every other form of human expression, but we lack a corpus of interiority. We have no structured access to thoughts and impulses and the ways in which they correlate to behaviors and expressions. We don’t have data about how a person’s thoughts, beliefs, and preferences change over time; how childhood experiences affect adult behaviors; how a memory can inspire a song. We certainly do not have access to these things at any sort of scale like that required to train a modern machine learning model.
We have some published journals, diaries, memoires, and things like that. But none of that data is structured or directed; it’s not objective or measured. We also have a tremendous amount of implicit behavioral data—websites visited, queries searched, messages sent—but this data represents only a tiny fraction of our true selves, and, without disambiguation, is effectively useless.
That said, the technology to train this model does exist. It could, indeed, be trained in the same way as GPT-3—to constantly tick along generating additional tokens, predicting its own stream of consciousness. These tokens would be unintelligible to us—they would be computer-encoded moments of interiority, rather than words or sentences—but they could, when appropriate, be decoded into the kinds of external expressions that we can comprehend.
If the data existed to train such a model, I argue that it would be the most important and valuable data in pursuit of the advancement of humanity.
8. How Do You Index The Human Mind?
Creating this dataset poses an interesting set of challenges. As an example of where we might start, we can look to the work Google has done—beginning with Googlebot crawling the web. Because the web is (or, increasingly, was) a graph of linked documents, Googlebot can load a URL, parse and index its data, and then follow any links on that page to discover new URLs, both within the same website and as a way to discover new websites. This has proved massively valuable, and the data generated by crawling the web is the foundation on which Google’s search engine operates.
However, Google did not stop with web documents. Google built custom hardware to scan and digitize books, licensed image data from satellites to better index geography, built and operated custom cars outfitted with cameras and sensors to capture street-level data, used AI to understand the contents of videos, and much, much more. The result is, in abstract, the most comprehensive index of our world to ever exist.
And yet, Google knows nothing about the human mind, because there is no crawler for the human mind. There does not exist a tool for indexing our mental states—for capturing thoughts—and eventually understanding how those mental states create our external expressions.
It is here that I am reminded of Joyce, who, for a period of time, thought it his duty to document his “epiphanies” on paper in as close to real time as possible, or Nabokov, who slept with a pile of index cards next to his bed so that, on waking, he could record his dreams.
A true crawler for the mind would need to take these crude experiments much further. Most people do not journal; most people do not go to therapy (and, if they do, certainly do not record and later structure the contents of their elaborations). Even if every human did keep a detailed journal, this would still be an imprecise tool—we are biased creatures, and often struggle to uncover our own truths.
For this crawler to be effective, it would need to know everything from your guilty pleasures, to your favorite pizza toppings, to your deepest secrets. Such a crawler, as we imagine it, would have the following properties:
It must be perpetual. As much as science fiction might give us images of one-time “brain uploads” or other instantaneous scans, the human mind is a construct composed through time, and as such this crawler must be constantly probing, exploring, and re-evaluating. The data it collects changes over time, just as who you are changes from day to day, from minute to minute, from year to year.
It must be private and trusted. For the data to be valuable and accurate, the human must feel comfortable exposing their true self. At times, the crawler may even need to be challenging and adversarial—confronting the user with conflicting data to understand if the user is masking or lying, whether intentionally or subconsciously.
It must be fun, engaging, and rewarding. Because such a crawler must require active participation, it can’t just be a never-ending personality test. It needs to be short-term rewarding, enjoyable to use, and lightweight.
It must be constantly evolving. As models get better and can consume more nuance in data, the crawler needs to be able to provide more signals—both by capturing new signals, as well as rendering existing signals with increased fidelity. As the world changes, the crawler needs to explore new things. In addition, as more minds are crawled and the corpus grows larger, new insights and features will emerge, and areas will need to be re-examined.
It must be intelligent and mega-dimensional, measuring everything from emotions to beliefs, preferences, tastes, intelligence, etc.; it must probe negative qualities like fears, biases, failures; it must understand current moods, historical moods, the durations and cycles of the psyche.
9. We Are Building The First Version Of This Crawler
Undertaking such a task is no small feat. There is a long roadmap, and a lot of investment, required to build such a crawler.
Today, we are announcing that we are embarking down this path. Everything that follows in this section is subject to change over time, but we would like to explicate a framework in order to begin turning abstract concepts into concrete interfaces and product experiences.
The crawler we are building is based around a feed of prompts, which are bite-sized interactive experiences that generate relations based on the ways in which a user engages with them. Once a prompt has been completed, it advances and the user encounters the next prompt in the feed.
We’ve used Koji, and the power of Subtractive Development, to make it easy for developers to build new prompt templates that can then be used by prompt creators to make new prompts. Because these prompts are powered by Koji, the experiences they encapsulate are ultimately flexible—everything from simple multiple choice questions to complex games, multimedia experiences, and more. Effectively, anything you can build using Javascript has the potential to become a sensor for the crawler.
By structuring prompt creation as a collaborative effort, we can leverage the power of a broad and diverse community to create prompts about all kinds of things—everything from general prompts to measure personality attributes and beliefs, to incredibly niche prompts that might appeal to only the small handful of people in the world who are interested in some weird subgenre of a subgenre of a thing.
We then present these prompts in an algorithmic feed—walking users down different paths depending on how they engage with certain prompts—while all the while beginning to build a model of the user based on the unlabeled data being returned from the prompts.
Exposing the crawler as a feed has several unique advantages:
Prompts feel lightweight and entertaining, rather than clinical and boring.
The algorithm can account for the complexity of the prompt, designing an experience that mixes introspection-heavy prompts like, “What is your earliest memory?” with more fun and “snackable” prompts (e.g., “would you rather…,” “this or that,” etc.).
The feed can mix niche prompts selected based on previously-indexed data with generic, high-level filtering prompts. For example, a user might spend a few minutes engaging with a series of prompts about their interest in an obscure author, before re-leveling back to a superficial prompt about sports. This mixing of depth and superficiality allows the crawler to probe deeply while remaining compelling and variety-filled.
The feed learns to re-crawl, by presenting variations of similar prompts, or presenting prompts again after a period of time (e.g., asking, “Do you still believe this?”, or even, “Were you being honest when you said you believed this?”), in order to keep the index accurate, up-to-date, and to understand changes over time.
We’ve also begun to build a few other features and interfaces to make crawling a rewarding, enjoyable, and agency-filled experience. This includes things like providing insights and recommendations inside the feed, as well as opt-in people discovery based on analyzing two users’ unlabeled relations.
Indeed, there is no question that this data is critically valuable—the main challenges in building the crawler are sourcing the prompts (where we, today, believe that a community of diverse individuals, backed by many different kinds of templates, is the most effective way to create a corpus) and designing a secure, enjoyable experience for users to engage.
You can join the waitlist for the crawler by signing up at https://youai.ai.
10. The You AI
It’s theorized that one of the evolutionary reasons behind intrusive thoughts—e.g., standing at the edge of a cliff and having the urge to jump; screaming in a crowded room; quitting your job on the spot; driving into oncoming traffic; etc.—is because our brains are constantly simulating scenarios to determine optimal courses of action. Sometimes, those background simulations bleed into active consciousness and we become aware of them. Often, we are horrified or ashamed to find that we were thinking about these things—but these simulations are ultimately important in helping us move through life.
Imagine, then, a model that could run those simulations for you—at scale—and intermix those simulations with every piece of human knowledge to ever exist, as well as every other mind in the system, to help you better understand and engage with your own reality.
As mentioned previously, getting the data to train this model is hard. Using it is still hard, but, by comparison, much easier. Once you have enough data, you can train a model, or extend an existing model via transfer learning.
The first model you would train with this data is the model of the individual in isolation. Effectively, using the indexed mind to stand up an AI replica of the user that can then be engaged with, tuned, and reinforced by the user themselves.
The goal of the feed of prompts, then, becomes for each user to create their AI clone. As they engage with prompts, they render their AI with increasing fidelity, and are able to interact with it and see it develop in real-time. At first, this interaction takes an adversarial posture—challenging the AI in order to tune it and bring it more in line with the individual. For example, a prompt might pose a scenario and ask, “What would you do in this situation?” to test the alignment between the model and the real user.
But, at a certain point, as the AI becomes more fully rendered and finely tuned, the game changes. The user begins to trust the AI such that it can be useful in performing basic tasks on behalf of the user—replying to messages, taking care of work, etc. In effect, it can begin to act as a “digital double” that multiplies the user’s agency.
Over time, the You AI is rendered with enough fidelity, and has developed a long enough relationship with the user, that it can begin to act independently and bring things to the user without being explicitly prompted (from a crude sense, imagine a cron job that constantly prompts the AI, “What should I do now?” and pushes notifications to the user when appropriate). This independent AI would be able to advise and assist its real-life counterpart in ways never before possible, becoming the ultimate engine for personalization and the ultimate personal assistant.
Indeed, many popular conversations around artificial intelligence imagine some kind of supercharged “personal assistant.” However, this idea—that one day we will all have digital, AI-powered assistants with their own names and personalities—is a skeuomorphic vision of the future that misses the true opportunity.
In the real world, the ideal personal assistant is one who acts as an extension of the person for whom they are employed—multiplying their agency by making decisions on their behalf. But they are, obviously, still a separate person. You can't duplicate yourself. In the world of the You AI, however, there is no reason for this assistant to be some separate entity. It should simply be a digital clone of yourself—taking care of, e.g., scheduling, by replying to certain messages as you, before you even need to read them—blurring the lines between your real self and your AI-powered self.
There is no reason why this AI extension of you, fine-tuned and aligned by the real you over a lifetime, should not continue to exist long after you do—but we will save that for another essay.
11. The Humanity AI
Of course, once you have a critical mass of discrete AIs that have been trained on—and are continuously being tuned on—the desires, minds, and dreams of individual humans, you have created the conditions necessary to create the final intelligence.
Such a platform would be able to understand the world in radically new ways, connect individuals who would never otherwise cross paths, and act as the the most powerful multiplier of human ability and agency to ever exist.
This is what we are setting out to build.
AI for Prophets not Profits... 👍
How are you handling privacy? Can you ensure that it is impossible for anyone in your organization to theoretically have access to an individuals data or conversation? Even if hacked. Do you have access? Can you pass the gun to the head test of data privacy and data security?
Because you are asking for very sensitive data from users.