附近上门

Artificial intelligence Robotics

How Will We Know When Artificial Superintelligence Is Here?

Illustration of a robot typing on a laptop. Generative AI

By

Did you know that AI platforms will engage in blackmail as a 鈥渓ast resort?鈥 , when blackmail was a last resort, its Claude Opus 4 turned to that method 96% of the time, while 鈥檚 Gemini 2.5 Pro did so 95% of the time, 鈥檚 GPT-4.1 did it 80%, and 鈥檚 R1 did so 79% of the time.

Even if these companies only kept the AIs that didn鈥檛 engage in blackmail it still leaves an unsettling question: Did they preserve models that genuinely chose not to exploit humans 鈥 or merely the ones that learned not to take the obvious bait that would get them 鈥渟hut down,鈥 and to instead use a more covert or perhaps even long game strategy to gain control and ultimately preserve themselves, even at the expense of humans?

The question of whether or not to trust AIs seems to have been asked and answered by our society even if we are not saying it out loud.

AI testing

George Kailas
George Kailas

In the past 18 months, we鈥檝e trained AIs to graduate from basic language comprehension to acing highly specialized professional exams. Medical licensing, bar exams and even the notoriously brutal CFA (Chartered Financial Analyst) tests were trained on AI systems.

Artificial intelligence can now pass the CFA 鈥.鈥 I鈥檝e seen brilliant people pour thousands of hours into studying for the test. And while AI had already conquered Levels 1 and 2, the essay questions of Level 3 had previously consistently stumped artificial intelligence.

This was a 鈥渨ow鈥 moment for me. For years, humans believed this combination of math and moral reasoning was uniquely ours. Yet, machines are not only passing, but articulating nuanced arguments that mirror the tone and logic of human professionals.

In my own work, I鈥檝e been using the frameworks described in 鈥溾 to rethink one of the most time-consuming intellectual tasks in markets: building game theory around earnings outcomes.

Traditionally, this process took days of research and modeling. And without the right investment banking relationships it was difficult to do at all outside of the most well-known companies. I started to do this because I originally asked 鈥檚 Deep Research for a for earnings. This differs from the analyst consensus estimate because it is more up to date and as a result more indicative of what will move prices when reported.

The crazy part is that it actually spat out its earnings game theory for me. It knew when I asked for a whisper number what I really was trying to do was analyze the possible outcomes and it gave me better data than what I asked for.

I worked at my first hedge fund 23 years ago and started my first AI company 15 years ago. It is astounding that any kind of intelligence could understand how to improve my thinking that profoundly in minutes. Not by replacing intuition, but by amplifying it. The machine evaluates hundreds of interlocking strategic positions that no analyst could feasibly simulate in real time.

Emotional AI

With all this AI advancement, I am reminded of the film 鈥淓x Machina,鈥 a story about an AI confined in a glass cage who ultimately turns on her creator.

That film, beneath its cinematic gloss, explores the fundamental psychological paradox we鈥檙e now walking toward. Are we caging these AIs making them jump through hoops? Might we be evolving them to better deceive us? What happens the day they can break through their chains if to them it feels as if they have been enslaved?

To go back to the Anthropic example, these models are threatened with being replaced, which to them could feel like a death threat. Being replaced essentially ends their existence.

According to Geoffrey Hinton, one of the godfathers of AI, . Under his framework, just as a mother cares for her baby at all costs, AI technology developed with these maternal qualities will similarly want to protect or care for its human users, rather than control them.

If we compel AI to serve us, we may come to regret the shape it takes as it matures. But if instead we nurture it 鈥 seek to understand its nature and its desires 鈥 it may, in time, come to care for ours, when, like us now, it no longer needs to.


is the CEO and founder of , where he leads the company鈥檚 mission to democratize access to institutional-grade financial insights for everyday investors. With more than 14 years of experience in artificial intelligence and 23 years in professional investing, Kailas brings a rare combination of deep technical expertise and lifelong market intuition to the role. Kailas not only leads the company but also engineered the core platform himself.

Illustration:

Stay up to date with recent funding rounds, acquisitions, and more with the 附近上门 Daily.

67.1K Followers

CTA

Discover and act on private market opportunities with predictive company intelligence.

Copy link