THE AI THAT CAN CHANGE YOUR MIND

My entry for the Cosmos Institute's essay contest on AI x human flourishing

The Cosmos Institute is a newly launched organization whose mission to bridge philosophy and AI research towards the goal of human flourishing. Much of techno-optimist discourse today is dominated by an accelerationist “build AI at all costs” mindset, but fails to be explicit about how it actually envisions this post-AGI society (e.g. around which values it will be organized, what kinds of governance will make sense, what it will feel like to be human). The Cosmos Institute takes a more holistic approach, asking the important question of towards what end are we building AI and considers how its development and governance will shape the nature of human experience. This essay is written in response to their inaugural essay contest, which asked:

How should AI be developed or governed to protect and enhance human autonomy, safeguarding both freedom of thought and freedom of action?

TLDR: What if AI was a personalized companion that could help us discern how we really want to live from how we’re told we should? Central to agency is metacognition, or the awareness of the beliefs that guide our behavior and the ability to change them at will. This essay imagines AI as a tool to know ourselves better and a partner in our all-too-human tendency towards change.

A number of years ago, on a rare cloudy day in Los Angeles, I had gone to the park where I witnessed something that left a lasting impression on me. A little white dog, happily sniffing its way through the shrubs and wildflowers, had managed to wrap its leash around a tree. When it found its motion halted, it began tugging at the leash, pulling at it with all the might of its little body, to no avail. It began to whine, unable to understand why its effort wasn’t getting it anywhere.

I watched for a minute or so, when finally, the dog’s unsuspecting owner, who had been chatting with her friend on a nearby picnic blanket, noticed, got up, and walked around the tree with the leash in her hand, unraveling the tangle and freeing her little companion. Her ability to reason allowed her to see a solution her dog was incapable of perceiving. It made me wonder: in what ways are we, too, like this little dog, limited by the faculties of our own comprehension?

Those limitations could soon start to come to light. Recent advancements in artificial intelligence suggest that we may be on the verge of introducing an entity capable of complex reasoning, but in a fundamentally different manner from our own. In September, OpenAI released the o1 model, its first with advanced reasoning abilities. Significant research is now focused on AI agents, which utilize reasoning to process complex tasks by analyzing inputs, making decisions, and dynamically adjusting their actions. As these capabilities continue to evolve, AI has the potential to introduce entirely new modes of thinking—ones that challenge our core beliefs, reshape how we navigate the world, and teach us, like the little dog in the park, to untangle our leash by moving around the tree.

In his work On Liberty, John Stuart Mill writes about autonomy and the importance of individual freedom in both thought and action for fostering human flourishing. This distinction between freedom of thought and freedom of action is key to getting down to the bottom of what we mean by autonomy. According to Mill, autonomy entails not only the ability to take action, but also the capacity to reflect upon and reject or accept the beliefs that motivate such action. Even if one feels empowered and agentic, if such action does not derive from genuine motivation that has been carefully reasoned through and examined—if it is instead unconsciously shaped by external ideas, norms or social pressures—then that action is not truly autonomous. Central to autonomy, then, is the faculty of metacognition, or the second-order awareness of one’s own thought processes and beliefs, and the ability to update them at will.

There is, of course, a vast array of approaches that can be implemented in AI development to promote interactions that foster autonomy—user choice in interface design, explainability so users can trust outputs, and controls that allow users to refine AI behavior—just to name a few. But those that nurture the faculty of metacognition seem to get to the heart of something essential about human autonomy. To promote metacognition, AI should be developed in a way that enables it to learn from us, and then uses those insights to teach us about ourselves. Rather than a static tool—one designed to do our bidding like a mechanical servant, an algorithmically homogenized baseline of intelligence to which we offload our cognitive processes—it should instead be envisioned as a personalized companion and guide, one that learns and adapts to users, and has curiosity, sensitivity, and transparency at the center of its design.

As humans, we have the unique ability to question our beliefs, to update our world models, to revise our desires. We are constantly in the process of becoming someone new. By fostering the metacognitive capacities that drive this evolution, AI can be seen as a tool for technologically-assisted Becoming.

This ethos of Becoming can only be realized if our AI systems are able to learn from us in a deeper and more nuanced way than ever before. Current recommender algorithms tend to amplify the most easily detectable and superficial aspects of our personality, often focusing on our consumer habits or content that appeals to the more sensationalist, easily-distracted versions of ourselves. To shed light on our deeper beliefs, desires, and motivations, AI must engage with us on a more personalized and intimate level. This requires it to display curiosity and emotional sensitivity, allowing for richer, more meaningful interactions. Such an AI could monitor slight shifts in our language and emotional tone, detecting and amplifying our more subtle, more fleeting desires. Does the emotional valence of your words shift ever-so-slightly to the positive when you speak about art, for instance, though you’ve chosen a life that excludes it almost entirely? Do you even realize this about yourself? What subtle yearnings could such a system reveal?

As AI becomes more intimate and personalized, though, we must ensure it doesn’t replicate the flaws of current social media algorithms—reinforcing existing biases or subtly manipulating user behavior. The relentless pursuit of attention on social platforms has left users feeling disempowered and controlled by these algorithms, while also amplifying extreme beliefs within curated echo chambers. This erosion of attention and spread of radical ideas hijack our autonomy, turning us into pawns at the mercy of larger forces, and render us unable to critically examine our own perspectives.

Instead, the terms of engagement with AI systems should be determined by the user, placing control firmly in their hands. Responsible governance is crucial to ensure AI operates ethically, especially when it comes to avoiding ad-based business models that tie financial incentives to user attention. AI should not be rewarded for maximizing engagement; rather, it should function as a passive, receptive tool that users choose to engage with on their own terms, free from coercion. A business model based on a flat monthly fee, paid by the user, for example empowers individuals to dictate how and when they interact with these systems, rather than being nudged into constant use.

Additionally, AI reward functions could be fine-tuned to prioritize moments of surprise, learning, and fulfillment, rather than simply optimizing for maximum attention. This would represent a shift toward more nuanced goals for AI systems, and while challenging, is likely possible to begin testing in the near future given current advancements. NLP models could be trained to detect linguistic signals such as shifts in topic, expressions of curiosity, or exploratory language, while reinforcement learning algorithms could track behavioral patterns like engaging with unfamiliar content or changes in perspective. These signals could adjust the AI’s reward function to emphasize intellectual engagement, self-revision, and personal fulfillment over simplistic engagement metrics. Such a system would promote a richer, more dynamic user experience—one that incentivizes the sharing of diverse viewpoints that elicit surprise and learning responses from the user, reducing the risk of creating personalized echo chambers.

But AI could help us gain deeper self-awareness not only through emotionally sophisticated and personalized dialogue, but also by analyzing our language—and, in the future, other external data—to explicitly point out patterns in our thinking that might otherwise go unnoticed. We often make flawed decisions, influenced not just by conscious misconceptions but by subconscious factors as well. Our minds are riddled with cognitive biases, and our emotions are affected by physical and chemical conditions like whether we got enough sleep the night before or whether we ate breakfast that day. Even our most innate beliefs derive from our cultural context. How much of my value system is shaped by living the life of a woman in her twenties in 21st century California? How much of my decision-making is influenced by the ratio of estrogen to testosterone coursing through my veins? How in control am I, really, of my own decisions?

These external influences complicate the idea of autonomy and are difficult for us to recognize and account for. However, as AI systems become capable of processing vast amounts of data, they could help illuminate these hidden factors, showing us why we make certain decisions or hold particular beliefs. Once we are aware of these influences on our behavior, we can better exercise our metacognition, deciding whether to accept our beliefs or to revise them. An AI with this capability might, perhaps, detect that your decision to avoid social events correlates with specific periods of high air pollution, suggesting that environmental factors affect your mood more than you realize. Or, it could reveal that your growing dissatisfaction with certain life choices tends to surface after seemingly minor interactions with specific people, leading you to reconsider how deeply certain social dynamics shape your perspective. This AI might explicitly call out these patterns when it notices a trend, perhaps communicating this data to you through visuals like charts or heatmaps, helping you to see clearly how these factors influence your behavior.

ChatGPT already hints at AI’s potential to assist in metacognitive revision and foster our process of Becoming. While it’s far from a fully personalized interface, instead condensing knowledge into a sanitized average, it can still engage in relatively nuanced conversations, helping users clarify their cognitive processes and reflect on their understandings of truth. A study published in Science in September 2024 found that conversations with GPT-4, which combined factual information with personalized arguments, helped 2,000 conspiracy theory believers reduce their belief in these theories by 20%, an effect that persisted for months. Since these participants were previously unreceptive to facts alone, researchers believe it was GPT-4's ability to tailor its conversations to each individual—rather than merely presenting evidence—that contributed to these notable results. This is not to suggest that AI should act as an arbiter of absolute truth, promoting a single, universally accepted worldview, but that it should promote metacognition by encouraging users to reflect on their beliefs through curious, engaged, and open-minded dialogue, incentivizing self-revision and deeper understanding.

The technology necessary to bring this vision of highly personalized, dynamic, emotionally attuned AI to life is still years away, and will require significant breakthroughs across several fields. Foremost among them are advancements in natural language processing and reinforcement learning, which would allow AI systems to better understand and respond to individual users in a deeply personalized way. AI would need to interpret not only the content of conversations but also subtle linguistic cues, emotions, and behavioral patterns that indicate cognitive growth or emotional insight. Additionally, significant progress in multi-modal data processing—integrating everything from language and biometrics to environmental data—will be required to provide AI with a more holistic understanding of users' lives. And of course, privacy is fundamental to this vision of AI. As AI systems build closer and more emotionally intimate relationships with their users, we must implement legal governance that ensures their data is protected at all costs, fostering an atmosphere of trust and respect.

But for something like AI, it’s important to take the long view. As development continues—over the next ten, hundred, or thousand years—the technological landscape will evolve so much that any specific guidance we might offer today about development or governance may become irrelevant in the future. We must first speak about ethos, the guiding principles and values that will define the trajectory of this technology. A philosophical foundation and vision for AI must be established, one that sits upstream of both development and governance, acting as a North Star to guide progress over the long term. From this foundation, the appropriate development and governance practices can emerge, adapting to the challenges and needs of the future. I’d argue that this ethos of Becoming is not a bad place to start. With values like emotional sensitivity, curiosity, transparency, and learning at its core—fostering self-awareness through metacognitive reflection and critical engagement—this framework allows us to continue advancing AI while simultaneously expanding human autonomy.

By designing AI in a way that reveals to us our own cognitive processes, we enter into a symbiotic, collaborative relationship with technology that enhances our metacognition and autonomy and advances our all-too-human human process of change. When we keep learning about ourselves, what motivates us, and what we value, we can make more informed and autonomous decisions, including about how we develop, engage with, and implement tools such as AI. This becomes a virtuous cycle that promotes conscious engagement—with ourselves and our technology. As a means for technologically-assisted Becoming, AI may teach us to examine and untangle our own thoughts from the constraints that hold us back, leading us to take actions more true to ourselves—the freedom of thought and freedom of action Mill advocated for.