system 2 ai

Sequoia Capital released a report presenting various outlooks on AI agents following the emergence of o1(Strawberry). I found it particularly impressive, as it not only captures the overall current AI trends but also offers compelling predictions about the future of the AI market. Furthermore, it shows significant connections with Palantir, a company I've held for a long time and plan to maintain in my portfolio for the foreseeable future, which prompted me to organize my thoughts on this matter.

AI Model Trends

OpenAI's new model o1 significantly outperforms other existing SOTA models in terms of performance. While other LLM providers like Anthropic and Meta have also released improved models this year, o1 is fundamentally different because it features Chain of Thought-based reasoning capabilities that allow it to "stop and think."

Traditional models use System 1 thinking from Daniel Kahneman's "Thinking, Fast and Slow" - the type of reasoning that produces immediate responses without deep thought. After o1's emergence, Foundation AI models can now employ System 2 thinking, generating a range of possibilities, considering potential outcomes, and reflecting on their own reasoning process.

While pre-o1 models only use System 1 thinking, looking back at human history, System 1 thinking is often sufficient. This is why massive Foundation Models like GPT-4, with minimal inference compute time, have been able to solve many problems over the past few years.

However, for more complex problems like breakthroughs in mathematics or biology, quick intuitive responses are insufficient. Such progress requires deep thinking, creative problem-solving, and most importantly, time - beyond System 1's immediate responses. To solve the most challenging and meaningful problems, we need to invest time in thoughtful reasoning that defines human progress, going beyond quick responses within learning datasets.

This transformation will shift current AI from the world of large-scale pre-training to a reasoning cloud, meaning an environment where reasoning compute can be dynamically scaled based on task complexity. We are entering the era of reasoning compute in earnest.

The Importance of Domain Knowledge

One of Sequoia Capital's hypotheses about large language models was that a single model company would become so powerful that it would absorb all other applications. This assumption proved wrong on two fronts: first, there are many competitors constantly competing for SOTA capabilities, and no company has achieved absolute dominance in Foundational Models yet. The second relates to the application layer and domain knowledge.

While many LLM-based software services have emerged, I believe that except for ChatGPT, models have generally failed to establish themselves as breakthrough products at the application layer. This is because the real world is far more complex than theory, making it difficult to solve complex real-world problems with System 1 thinking alone.

There are various social contexts at play here. Excellent researchers studying large language models aren't interested in understanding detailed end-to-end workflows for all possible functions across all possible vertical markets. They show little interest in acquiring domain tacit knowledge or building ontologies. Structurally, it's both attractive and economically rational for researchers to stop at APIs and leave real-world complexity to the developer ecosystem. While research labs continue to push the boundaries of horizontal general-purpose reasoning, providing useful AI agents will require application or domain-specific reasoning. The complex real world demands substantial domain and application-specific reasoning that cannot be efficiently encoded into general-purpose models.

Therefore, along with the transition to the reasoning cloud, AI's future value will come from the ontology layer and the infrastructure that builds it. Personally, I think Palantir is leading this market. Through Forward Deployed Engineers, Palantir has been repeatedly acquiring domain tacit knowledge and structuring it by deploying directly to enterprises for years.

I've consistently believed for quite some time that Palantir's current position resembles Microsoft's rapid growth in 1993 as personal computer adoption increased. And the market is even bigger now. Intuitively, every entity using AI needs ontology - whether they're individuals, companies, or in the near future, machines or humanoids. Anyone trying to build something on top of AI needs an ontology layer. We must remember that System 1 alone cannot solve complex reasoning problems.

Conclusion

As Sequoia Capital mentioned, and I personally agree, o1 is the most significant model update of 2024. It has introduced a new step in how AI should function. o1 has shifted the reasoning layer of large language models from System 1 to System 2 thinking. It has now been proven that models perform better when they can think like humans and retrace their thought processes.

In this context, I believe AI's next phase will inevitably transition to agents, and what these agents need most is sufficient reasoning compute time and an ontology layer to utilize it. This is because an agent's job is to define and act on logic based on data, which essentially describes ontology.

AI is not determined by the performance of large Foundational models. It needs to solve real problems in complex domains, and infrastructure is what helps with this. There are companies breaking down and solving these problems in various ways. I feel we need to maintain both partial and holistic perspectives when viewing the current situation.