Weekend reading: First they came for the call centres

Three different comments if I may:

1. On @TI’s question: “But should active investors try to up that allocation still further?”: I’d suggest thinking in bets. It could be different this time, but (I’d guess) on 80% of occasions it isn’t, and mean reversion occurs. But you can bet on both horses by ‘barbelling’ the portfolio. Or at least you could if we were in the US and had access to their ETF universe (boo hiss to PRIIPs reg’s and MIFID which stop us getting access to these products here).

Between them the WisdomTree Global Megatrends Equity index US ETF and the Invesco S&P 500 Momentum US ETF, SPYO, would enable one to concentrate the half of the bet going into the ‘AI wins’ scenario; whilst the other half goes into an ‘AI loses’ allocation to ex-US developed market and Emerging Market Small Cap Value (i.e. the opposite).

The SPYO ETF is especially interesting here as it solely comprises those S&P 500 constituents with the highest risk-adjusted price return momentum scores adjusted by their cap weights.

Constituents are weighted by the product of market cap and momentum score, subject to constraints; namely that the maximum weight of each security is the lower of 9% and 3 times its market cap share in the S&P 500.

When the index is concentrating into the biggest names it becomes very hard indeed to beat it with any form of active stock selection.

When £1 is invested in the S&P 500 index, 35p now flows into the top 10 stocks. The remaining 65p is divided between the remaining 490 stocks. So investors risk failing unless 35% of their portfolio is invested in those 10 stocks.

Bill Miller from Legg Mason and Peter Lynch at Fidelity each achieved 15 years beating the S&P 500. Then their strategies ran into headwinds and stopped working. But markets were less efficient and less concentrated then. According to one calculation, at the current concentration levels in the top 10 stocks, basically the hyper-scalers which @TI refers to above; the probability of an active manager (before any fees and transaction costs) beating the S&P 500 for 15 consecutive years after doing so in the first year, is now just 1 in 2.3 million

2. @ermine #5: the regular Google search product is now irredeemably enshitified for sure, but their Perplexity AI does a pretty decent job as a universal summariser, and better I’d say than Chat GPT 3.5 (comparing free with free versions), even if it’s no form of any type of intelligence; just a better ‘dumb rock’ stochastic parrot. Here’s the answer it gave me to the query as to what effect a declining and aging population would have on both the equity risk premium and the risk free rate: “Based upon the provided search results, a declining and aging population is expected to increase the equity risk premium but decrease risk-free rates. The key points from the sources are: As the population ages, there will be a larger proportion of older, risk-averse investors shifting their portfolios towards safer assets like bonds, increasing demand for risk-free government bonds. This increased demand for bonds will drive down the risk-free rate of return. However, the increased risk aversion and lower risk tolerance of an aging population will reduce overall demand for risky assets like equities. Lower demand for equities will decrease equity prices and increase the expected equity risk premium. Specifically, some sources predict the equity risk premium will increase by around 0.28 percentage points due to demographic shifts, while the risk-free rate will decrease by around 1.5 percentage points. Sources also link population aging to a decline in safe real interest rates (risk-free rate) but a rise in equity risk premia. So in summary, the consensus from the provided literature is that an aging population will put upward pressure on the equity risk premium by reducing demand for risky assets, while simultaneously depressing the risk-free rate by increasing demand for safer government bonds.” I thought that was a pretty good first attempt answer for a machine.

3. The Tony Isola’s ‘A Teachable Moment’ piece in the Weekend links perpetuates a common misconception in his erroneous statement: “This capacity is enough to obliterate the planet several times over”. We are not six minutes away from the extinction of the human species, and we never have been. Dr. Brian Martin is a peace and disarmament activist in Australia, but also a social scientist commited to accuracy. Amongst many, he has pointed out that, whilst the effects would be quite horrifically devastating to the combatant countries, they didn’t (and still don’t) threaten human extinction. Here he is writing at the height of the Cold War in December 1982, when there were several times the number of weapons (both stockpiled and readied on launch on warning) as compared to now:

https://www.bmartin.cc/pubs/82cab/index.html

And this is a credible worst possible case scenario (from an alternative history perspective) written in 2003, and set in August 1988, at just about the worst possible time for an exchange to take place. It’s bad, but no extinction risk.

https://www.johnstonsarchive.net/nuclear/nuclearwar1.html

@ermine #12: Gary Marcus’ Substack is a good place to get some constructive informed skepticism about AGI/ASI generally, and about LLMs in particular.

The big questions for me here are:

a). Is ASI merely difficult but, in principle, within reach (whether over long or, less plausibly, short time scales)?

Or:

b). Is ASI just a dream, akin to wishing for magic, where physical impossibility meets the human need to imagine something lying beyond the possible, like each of:

– Backwards in time travel (e.g. Tipler cylinders):

https://en.m.wikipedia.org/wiki/Tipler_cylinder

– Faster than light travel (e.g. Alcubierre drive):

https://youtu.be/SBBWJ_c8piM?si=BlR3ze8en6tdEp-G

– FTL communication (e.g. using quantum entanglement):

https://youtu.be/BLqk7uaENAY?feature=shared

If ASI & AGI are phantasms of imagination and outside the realm of the possible, like each of the above examples are, then anything more than a zero allocation to their commercial realisation would be excessive.

But if AGI and (ultimately perhaps even) ASI are merely very hard, but not actually impossible to achieve (notwithstanding many incremental S curves of break through & adoption might be required over a long time rather than rapidly reaching a much hyped technological singularity); then there is at least some reason behind the current surge in investment linked to trying to realise these goals.

However, even then, the possibility of disappointments and delays would still be very substantial indeed. As with the TMT bubble of 1995-1999, even where a technology does ultimately more or less deliver as originally promised, the value of the companies which were built upon it can still crash miserably in the near to medium term if the pace of progress falls behind inflating and accelerating investor expectations.

Thanks @Alan S #16.

The 5 bn figure relies on a full blown, very long lasting and very severe global nuclear winter. Without that it’s topping out at a loss of 360 mn people (which is of course horrific) or 4.5% of the current world population of 8 bn.

To clarify, I don’t necessarily disagree with the possibility of global nuclear winter, nor with the general thrust of the concern expressed in Annie Jacobsen’s book, which Tony Isola references (and which I’ve read, cheapskate that I am, pursuing it whilst in Waterstones 😉 )

Similar to last week’s Weekend reading comment #14 by @BarryGevenson that, “90% of life on this planet will be dead in 150 years”; my only objection here is on the factual inaccuracy of Tony Isola’s statement that: “This capacity is enough to obliterate the planet several times over.” That’s a strong, emphatic but incorrect claim.

In fairness, he’s an excellent financial blogger, and he’s relying here on Jacobsen’s otherwise superbly presented and quite credible book, but also one which seems to me to vere off right at the end and – after outlining a well researched, well crafted and detailed scenario – then appears to go hyperbolic in its concluding pages and appear to suggest that much of the world would be uninhabitable to humans for 25,000 years.

There’s 40 years of controversy here (starting in 1982 with “Nuclear War: The Aftermath” in Ambio, published by on behalf of Royal Swedish Academy of Sciences); but not even the most ardent advocates for taking the severity of nuclear winter seriously, and not even the most severe models, predict human extinction – except as the most vanishing remote possibility.

As might be expected, in recent years the EA and LessWrong community has been active in both quantitatively probing the models and in reassessing the risks within numerical parameters, see as examples:

https://forum.effectivealtruism.org/posts/pbMfYGjBqrhmmmDSo/nuclear-winter-reviewing-the-evidence-the-complexities-and

https://forum.effectivealtruism.org/posts/6KNSCxsTAh7wCoHko/nuclear-war-tail-risk-has-been-exaggerated

https://www.lesswrong.com/posts/sT6NxFxso6Z9xjS7o/nuclear-war-is-unlikely-to-cause-human-extinction

The second and third of the above respectively note of the Robock study (which kicked off the modern, post Cold War, series of models on this subject):

– “Luke Oman, one of the 3 authors of Robock 2007, having guessed a risk of human extinction of 0.001 % to 0.01 % for an injection of soot into the stratosphere of 150 Tg.” [150 teragrammes of soot being the worst case in an all out exchange in Robock’s already very pessimistic study].

– “Carl Shulman asked one of the authors of this paper, Luke Oman, his probability that the 150Tg nuclear winter scenario discussed in the paper would result in human extinction, the answer he gave was “in the range of 1 in 10,000 to 1 in 100,000.””

The actual reasoning of Luke Oman here – as one of the most prominent advocates of the possibility of severe nuclear winter – is then set out in his Q&A at:

https://www.overcomingbias.com/p/nuclear-winter-and-human-extinction-qa-with-luke-omanhtml

Human extinction risk is the existential dread which Tony Isola seems to fear in his piece, which is linked to in this week’s Weekend reading.

But, whilst the loss of (at most) between 360 mn and 5 bn lives amongst the 8 bn human living would be an unimaginable tragedy and an unprecedented disaster; it would not be extinction.

Extinction forecloses the lives of everyone who might otherwise live. That could be a lot of people.

If you very conservatively assume a future average human (and human descended) population size of 1 bn people (i.e. only an eighth of the current world population size) with typical lifespans of a century, and then allow that the Earth will remain habitable for between 500 mn to 1.3 bn years but that natural mass extinction level events seem to occur every 100 mn to 500 mn years, then extinction now would foreclose the possibility of at least a quadrillion (i.e. a 1,000 tn) future human lives.

This is why the loss of all 8 bn people alive now is likely to be at least a million times worse than the loss of 7 bn out of 8 bn people alive, and not just 14% worse.

Fortunately, what Tony Isola seems to fear in his piece, namely actual extinction of the human species, just isn’t going to arise out of this particular risk vector.

And there are plenty of reasonable grounds to doubt whether even an non-extinction level global nuclear winter scenario would eventuate.

In 1991 it was claimed that the Kuwaiti oil well fires might cause a global winter and lead to famine in Asia (Peter Aldous, January 10, 1991, “Oil-well climate catastrophe”, Nature, 349 (6305): 96, “The fears expressed last week centred around the cloud of soot that would result if Kuwait’s oil wells were set alight by Iraqi forces … with effects similar to those of the “nuclear winter … Paul Crutzen, from the Max Planck Institute for Chemistry in Mainz, has produced some rough calculations which predict a cloud of soot covering half of the Northern Hemisphere within 100 days. Crutzen … estimates that temperatures beneath such a cloud could be reduced by 5–10 degrees C”). Those concerns turned out completely misplaced. There were only very localised and minimal cooling effects.

With the best of intentions and belief, the late great Carl Sagan and his colleagues sought in the 1980s to draw attention to nuclear winter risk. Where it seems, based upon the empirical evidence that we now have the benefit of, that they probably went wrong was that they expected that a self-lofting of the sooty smoke would occur when it absorbed the sun’s heat radiation, whereby the black particles of soot would be heated by the sun and lofted higher and higher into the air, thereby injecting the soot into the stratosphere where it would take years for the sun blocking effect of this aerosol of soot to fall out of the air, and with that, catastrophic ground level cooling and agricultural impact. Instead it seems more likely now that this soot wouldn’t self loft to high enough altitude and instead would get fairly rapidly washed out by rainfall.

@Alan S #19: “would bonds, equities or commodities do best”: Benzinga markets says that, come the AI or other apocalypse, invest in a lifestraw, and not gold or BTC:
https://www.benzinga.com/markets/cryptocurrency/24/05/38821153/bitcoin-and-gold-wont-save-you

James Altucher has written the book on crisis investing: “The Wall Street Journal Guide to Investing in the Apocalypse: Make Money by Seeing Opportunity Where Others See Peril (Wall Street Journal Guides)”:
https://www.amazon.com/exec/obidos/ASIN/0062001329/thebigpictu09-20

And Michael Batnick at Ritholtz Wealth Management says to just carry on as normal 😉 :
https://www.theirrelevantinvestor.com/p/the-thing-that-doesnt-mix-well-with-investing

Nice summary here from a Wharton Business School Prof. of 4 potential AI economic pathways and of LLM’s modus operandi:

https://youtu.be/d4f1jqb3Yis?feature=shared

Meanwhile AI cheerleader in chief and technological singularity soothsayer Ray Kurzweil has a soon to be published follow up to his 2005 (slightly bonkers) foundational AI/AGI/ASI classic “The Singularity Is Near:

https://www.penguin.co.uk/books/462759/the-singularity-is-nearer-by-kurzweil-ray/9781847928290

And former Guardian writer, part time Buddhist, and Bali based digital nomad and sci-fi commentator Damian Walter has an intriguing (anarcho-capitalist, libertarian-socialist, hybrid mash up?) take on the potentials and perils of AI. The 2nd of these is quite long. I listen to these at 2x speed on YouTube. The key question in the second is from 7 to 14 minutes in.

Is it going to be a utopia in the mould of Ian M Banks’ Culture; or instead either one of William Gibson’s, Philip K Dick’s or Aldous Huxley’s dystopias or, worst still, one of the worlds which Frank Herbert or George Orwell warned against:

https://youtu.be/iVd1hPewcCw?feature=shared

https://youtu.be/uGZW1xnkzkI?feature=shared

Interesting take on the use of AI in investing:

https://www.telegraph.co.uk/business/2024/06/13/ai-better-investment-decisions-humans/

And this one on why maybe the technological singularity really is near to hand:

https://situational-awareness.ai/wp-content/uploads/2024/06/situationalawareness.pdf

[Correction: my reference to the SPYO ETF at #11 should have been to the SPMO ETF]

I’m posting this here because there’s some obvious implications for investors now if the LLM scaling to narrow AI to AGI to ASI race (and hype) is not merely trying to ascend a much steeper and far higher mountain than Open AI, Anthropic, xAI, Meta, Gemini et al are planning for; but, instead, is actually in a scenario where all the efforts are being directed at the wrong mountain and where the right mountain to climb is effectively impossibly tall and steep.

So, for over 30 years Sir Roger Penrose, the 1988 Wolf Prize in Physics winner, which he shared with Stephen Hawking for the Penrose–Hawking singularity theorems, and the 2020 Nobel Prize in Physics laureate; has had a highly unorthodox and contentious conjecture about how consciousness arises in the brain.

He thinks that consciousness could be a quantum process (orchestrated objective reduction) involving structures common in neurones called microtubules.

Apart from the anaesthesiologist Stuart Hameroff, almost noone else has taken the idea seriously as it would require quantum superpositions in the microtubules in the warmth of the brain to be sustained for many orders of magnitude longer than the wave function collapse takes in the cold (near absolute zero temperature) environments of quantum computers for qubits with only a few particles (decoherence takes no more than an attosecond in non pristine environments, far quicker than any brain processes, like a neurone firings).

There’s several other technical and fundamental objections to the idea and, so far at least, it just hasn’t garnered any enthusiasm from the physics, neurology or computation professions trying to understand the hard problem of consciousness.

Anyway, there’s been something of a breakthrough recently covered very clearly here:

https://youtu.be/xa2Kpkksf3k?feature=shared

The upshot for AI research/development is that, if Penrose is right, any classically algorithmic process cannot lead to consciousness and that human like (or super human) intelligence cannot emerge from all current approaches to AI.

And if the AI industry is barking up the wrong tree and (mixing metaphors here) trying to climb up the wrong mountain, then I’d guess that’s not great for investment in tech heavy indices right now.

And here’s one today giving the funeral rites to Open AI:

https://www.wheresyoured.at/to-serve-altman/

I’m starting to wonder if this is like the summer of 1999, and the market goes pop after Christmas when uncomfortable realities about the ‘no show’ for an AI revolution set in.

Still, at least it looks like Open AI has some revenues ($3.5 bn to $4.5 bn annualised). So there’s something for hope to grab hold of there.

Latest (gloomy) prognoses for AI investors and for society:

https://open.substack.com/pub/tedgioia/p/eleven-predictions-heres-what-ai

https://open.substack.com/pub/unchartedterritories/p/how-to-navigate-a-tech-world-dominated

Then again, maybe it won’t be so bad for the semi conductor giants:

https://open.substack.com/pub/citrini/p/market-memo-deepseeking-answers

Nvidia ended at 17% down on the close, at a market cap of $2.9 tn, which is >$800 bn off its’ ATH just three short weeks ago. Parsing DeepSeek likely involves a preliminary judgement call: is it for real? Or is it a Tupolev 144, i.e. not a Concorde. Assuming for the moment that it might be for real, is this just a moment with (perhaps dire) implications for the premium shovel supplier in the AI gold rush (fool’s gold???) or does it go deeper and wider. Fallacy Alarm had some hot takes this evening:

https://open.substack.com/pub/fallacyalarm/p/lets-try-to-contextualize-deepseek

Sooo much now appearing about DeepSeek. A couple of highlights:

Ed’s take:

https://www.wheresyoured.at/deep-impact/

Which in turn ref’s Venture Beat:

https://venturebeat.com/ai/deepseek-r1s-bold-bet-on-reinforcement-learning-how-it-outpaced-openai-at-3-of-the-cost/

The human brain (with 100 bn neurones and 100 tn synaptic connections) does pretty robust AGI with 20 watts of power, 1300 cc volume and 3 lbs mass, with perhaps 10 exp 17 FLOPS equivalent delivered (although guesses at computer emulation of the brain at the synaptic equivalent level go up to 10 exp 25 FLOPS required).

Clearly, therefore, there’s no physical law that says that meaningful ‘thinking’ on a non-biological substrate will actually need millions of bleeding edge ($40,000 each) GPUs using up Gigawatts.

P.S. Guess that Damo sums up how a lot of us are thinking now about Sam Altman and OpenAI at this difficult time for them:

http://youtube.com/post/UgkxCXe81JpFePyLPFEVKRL7WEny2NQu4fdT

An AI tsunami is breaking in Substackopia
No moats = Ubiquity = “if everyone’s using the same tools, does anyone really have an edge?”:

https://open.substack.com/pub/investinginchina/p/ai-automation-and-the-impact-on-investment

My pleasure 🙂

It might be time to think about dusting off that draft “Buy The Robot” piece which you once mentioned IIRC 😉

MV touched upon the subject of AI/Automation back in 2017 of course:

https://monevator.com/weekend-reading-should-we-invest-in-the-robots-or-in-the-toys-theyll-use-to-keep-us-happy/

Can’t myself see LLM scaling (itself and alone) leading directly to AGI, but there are still multiple approaches beyond just using either LLMs / GPTs specifically or Neural Nets / Deep Learning more generally on their own (as AlphaFold 3 showed by adding a diffusion model), and whole under explored approaches, like neuro-symbolic AI, where symbolic manipulation acts like Khaneman’s ‘System 2’ reasoning and deduction in biological brains (with the deep layer neural net operating side by side, like the rapid, reflexive and unconscious pattern recognition of System 1).

More AI central in the blogoverse and in the ‘MSM’. Summarising:

No AI energy crisis/limit:

https://www.telegraph.co.uk/business/2025/01/29/net-zero-risks-leaving-britains-ai-strategy-dead-on-arrival/

A DeepSeek deep dive on both the effects on MSTF and on the (much cited this week) Jevons’ Paradox:

https://open.substack.com/pub/appeconomyinsights/p/microsoft-ai-demand-paradox

The AI hype cycle in brief:

https://open.substack.com/pub/thelastbearstanding/p/a-moment-of-disbelief

Perhaps also worth including this one from back on inauguration day (seems a lifetime ago in AI land) for comprehensive overview of the semi/fab landscape & AI in 2025:

https://open.substack.com/pub/aisupremacy/p/the-ai-semiconductor-landscape-2025

On robotics v pure AI plays (re above):

“US venture capital investment in robotics has risen from around $2 bn in 2019 to more than $3.5 bn last year, according to data from PitchBook. In the first nine months of 2024, there were 130 fundraising deals for robotics start-ups — more than across the entirety of 2019. Among the most high-profile was a $675 mn investment last February by Amazon founder Jeff Bezos, Microsoft and Nvidia in Figure AI, a Silicon Valley start-up founded in 2022 that is working on a faceless, humanoid “general-purpose” robot.”

Source: Financial Times

In most optimistic scenario after ubiquitous AI, AGI, then ASI, and after mass adoption of all purpose humanoid robotics, comes widespread unique use cases for fully commercialised quantum computing.

In the other scenarios… not so much.

More unbridled AI optimism this week from Uncharted Territories, albeit not so much on the jobs front.

https://open.substack.com/pub/unchartedterritories/p/ai-weeks-when-decades-happen

These guys aren’t so sure though:

https://open.substack.com/pub/thezvi/p/the-risk-of-gradual-disempowerment

Taking in the really big picture – where the further back you peer then the further forward you might see – the LLM/Deep Nets/Neural Nets/Machine Learning – AI/AGI/ASI – Quantum Computing mega paradigm looks to be an acceleration of a trend of increasing frequency and intensity of phase transitions in the human condition, going from the social/cultural/ agricultural to the energy/industrial/urban to the informational/intelligence (overlapping) ‘revolutions’.

It’s not as such deterministic, but there does seem to be a coupling of technology to historical contingency, with each phase then resulting in more extreme, more unpredictable and more impactful negative and positive outcomes.

It may or may not be a singularity which we’re headed towards, but it does look like it could well be another such major epochial transition:

https://youtu.be/t1qxJI9nc2g?si=SuCkOmAYSnNNVIh8

Time will tell if we make it to the other side of the transition as we have in the past.

Another take, this one mixed in its conclusions, on DeepSeek and the Jevons Paradox:

https://open.substack.com/pub/nyugrad/p/jevons-paradox-brings-price-destruction

Maybe AGI/ASI, were it to arrive, will be disaster. But if it’s not, then we can’t say who or how will benefit, only guess.

@TI: with great power comes greater responsibility.

Demis Hassabis’ Google Deep Mind Alpha Fold just this evening used as a case study for the undoubted benefits of machine learning/ narrow proto-AI over on veritasium:

https://youtu.be/P_fHJIYENdI?feature=shared

Sounds all wonderful unicorns and fairy dust but…what happens when this tech is improved 100,000x, becomes over 1,000 cheaper and is then made available to tens and hundreds of millions of people to use unsupervised.

Fully synthetic biology. From digital to DNA (or RNA) home synthesis, including for reverse chirality bacteria. It’ll be a disaster. 99.99% safe usage isn’t enough. It’ll just take one nutter.

We don’t need AGI or misalignment (much less still an ASI malevolent takeover, Terminator style) for this to be our last collective mistake.

Malevolent autonomously acting ASI is probably a fantasy (thankfully).

What unfortunately isn’t a fantasy is that throughout history there are individuals who try and do crazy things.

Until now, they’ve been pretty limited in the harm most of them can do.

But if we allow this technology to proliferative in an advanced form that will no longer be the case.

We’ll probably have had it.

Worries about AI created joblessness or inequality or loss of control or of deep fakes, scams and misinformation won’t hold a candle up to the extinction risk.

Technocracy today = technogarchy tomorrow = techno feudalism the day after:

https://youtu.be/_LQa28X-1AQ?si=-mA3Seze106_qOwt

Damo/Martin Niemöller has spoken: “First they came for the delivery couriers, and I did not speak out— Because I was not a delivery courier. Then they came for the taxi drivers, and I did not speak out— Because I was not a taxi driver. Then they came for the creative freelancers, and I did not speak out— Because I was not a creative freelancer. Then they came for me, a salaried professional— and nobody cared because algorithmic gigwork apps were normalised by then. The gigwork apps of the technogarchy are coming for all of us”

Nvidia earnings tomorrow. Cue memes:

https://substack.com/@paulmillerd/note/c-96381034?r=2kxl2k

https://substack.com/@leshrub/note/p-157870039?r=2kxl2k

https://substack.com/@paretoinvestor/note/c-96306421?r=2kxl2k

Got timed out on editing so please excuse the Part 2 here.

Meta looks to me like the stand out in the Mag 7.

Still relatively cheap on fundamentals despite a truly epic recovery from 2022.

Still investing into VR which so called Gen ‘AI’ should help with.

Most likely with Apple to benefit from cheaper LLMs (re DeepSeek).

No wonder Mark Zuckerberg wears expensive watches these days 😉

Alphabet looks the most vulnerable.

No discernable AI strategy left.

Google search is thoroughly enshitified and it’s most likely to be immediately negatively impacted by LLMs.

Tesla still has big optionality, and might be a buy for the brave.

But President Musk’s self sabotage and lower cost but still high quality Chinese EBVs are a massive threat, especially outside the US, even if Tesla pulls out all the stops on FSD.

Apple looks like post growth.

Amazon and Microsoft are somewhere in-between IMO.

LLMs are distorting mirrors of their training set.

Will AI emerging from them be something so changed it will be alien? Beyond comprehension.

Yet feeding upon our culture. Our creativity. Our fears. Our hopes. Our dreams. Our minds.

A dark mirror.

A disembodied presence.

But an all and always present one. An oceanic sentience.

Solaris:

https://youtu.be/ilsdMoQgBDs

Obviously I go back and forth on this one endlessly ( 😉 ), but here’s a study last month suggesting that current approaches to AGI would need ~10exp26 (a number with twenty six digits, so of order a hundred trillion trillion) parameters (compared to several hundreds of billions of parameters in the frontier models today), and so many Nvidia H100 GPUs would be required that they’d cost 40 million times Apple’s market cap:

https://arxiv.org/abs/2502.18858

On the other hand the human brain manages a modest general intelligence on just 20 to 30 watts contained in 1.35 kg and 1.4 litres.

As Jeff Goldblum, playing Dr. Ian Malcolm, put it in the adaptation of Michael Crichton’s Jurassic Park, “Life finds a way”.

So AGI it must be very doable ‘somehow’, and the human brain is of course very far from optimised for cognition etc.

So in terms of basic physics some form of ASI, and not just AGI, seems possible, albeit that I’m increasingly doubtful LLM scaling alone can get to either.

The big danger it seems to me in terms of achieving early technical success here is that LLM has monopolised the field and when it doesn’t deliver (GPT 4.5 could be the sign) they’ll be an overreaction as large as the hype cycle.

So, we’d be looking at a scenario of a crash (which would only be partly justified) followed by an AI winter (as after previous hype cycles), and which would not be justified.

That mirrors the TMT/Dot com ‘bubble’, which went parabolic in 1998/99 but was partly justified followed by a crash to 2002/03 the full extent of which with hindsight was not (NASDAQ from 1,000 to 5,000 1998-2000, back down to 1,000 in 2002 and now at about 20,000 in 2025).

As in 2002/03, the AI winter, if it is coming, could be the time to go significantly ‘in’ on AI related and adjacent firms and (assuming that it follows a similar crash trajectory) quantum computing companies.

*If* this isn’t BS (and it always could be, but there does seem to be experimental work/ research going on here) and *if* it can be adopted at scale, then it could be a ‘game changer’:

https://open.substack.com/pub/fractalcomputing/p/the-black-swan-event-about-to-hit

And *if* it worked and *if* Jeavons Paradox holds true then this would be a + for the semis and hyperscalars but otherwise a –

For Apple though, and in either event if it was successful, then it could screw with their bespoke M series chip architectures, so they’d have to retool.

The sages don’t seem to be feeling the AGI vibe (as much):

https://open.substack.com/pub/thezvi/p/on-not-feeling-the-agi

Check out the epoch.ai Gate AI and Automation outcomes simulator though (referenced therein):

https://epoch.ai/gate

Mind-blowing 😉

I don’t know if you’ll pick this up @TI but in case so then you absolutely must get hold of a copy of Robin Hanson’s “The Age of EM” (as in Emulations) (OUP 2018).

It’s Ray Kurzweil’s “The Singularity is Near” (Viking 2005) and “The Singularity is Nearer” (Penguin 2024) on fast forward.

I’d also warmly recommend here Physicist Max Tegmark’s “Life 3.0” (Allen Lane 2017).

We’re either on the brink of the greatest transition since the emergence of biology on this planet likely leading to either extinction or transcendence or it’s not and never will happen in which case maybe the US mega caps fall 20% to 30%. I guess we can all agree the first two possibilities are respectively more terrifying and exulting than the third.

Do the kind of extreme scenarios postulated in AI 2027 (and Situational Awareness) represent:

– Realistic depictions of plausible dystopic or catastrophic terminal events and utopian off ramps?
Or:
– Misplaced attempts to warn of the promises and perils of AI that will not materialise (at least foreseeably soon)?
Or:
– Part of the AI investment hype cycle complex?

There’s some pretty wide outcome ranges to the assumptions underpinning the AI 2027 scenarios, e.g. going through the Appendices, whilst the authors give a most likely figure of 95 years to go from Superhuman equivalent AI Researcher to full ASI the range they quote at an 80% confidence level goes from just 2.4 years (which I think they use in their scenarios) to 1,000,000 years – that’s not a typo, their estimate at 80% CI really stretches to a million years.

It does make one wonder about what is the appropriate degree of confidence to have in some of the other numbers? This – ASI – is, after all, and inherently, extremely highly speculative stuff, maybe more so than any other possible (and possibly near term) high impact anthropogenic ‘event’.

The reality at the moment is that pure LLM approaches seem to be hitting a scaling wall and are wildy unprofitable at least in terms of their current commercial viability; as Ed points out in great detail only today:

https://www.wheresyoured.at/openai-is-a-systemic-risk-to-the-tech-industry-2/

So maybe these scenarios are getting just a little bit ahead of themselves?

Daniel Kokotajlo has adjusted his estimates, and has moved his median from ‘AI 2027’ to ‘AI 2028’ based on events since publication. Maybe this will be like one of those delayed trains where it shows on time until the departure time and then the arrival time goes back by 1 minute every minute 😉

Meanwhile Less Wrong has a new detailed take down of “AI 2027” (19 June 2025, Titotal: “A deep critique of AI 2027’s bad timeline models”). Worth a read IMO.

And RAND has a study on AI X (extinction) risks which concludes as follows:

In the three scenarios examined — nuclear weapons, pathogens, and geoengineering — human extinction would not be a plausible outcome unless an actor was intentionally seeking that outcome. Even then, an actor would need to overcome significant constraints to achieve that goal. Analysis under uncertainty requires specific analytic approaches. X threats posed by AI are immensely challenging but cannot be ruled out. X threats occur over long timescales, allowing time to respond. AI would require four capabilities to create X threats: (1) integration with key cyber-physical systems, (2) the ability to survive without human maintainers, (3) the objective to cause human extinction, and (4) the ability to persuade or deceive humans to avoid detection.

AFAICT #(4) looks to be in the bag for LLMs already so that leaves three to go for future AI…

Here’s the study:

https://www.rand.org/content/dam/rand/pubs/research_reports/RRA3000/RRA3034-1/RAND_RRA3034-1.pdf

Two takes on ‘AI’ scepticism, one being somewhat half optimistic, from Gary Marcus (13th July), and the other deeply pessimistic, from Ed Zitron (yesterday, 21st July, it’s an epic 14,500 words!):

https://open.substack.com/pub/garymarcus/p/how-o3-and-grok-4-accidentally-vindicated

https://www.wheresyoured.at/the-haters-gui/?ref=ed-zitrons-wheres-your-ed-at-newsletter

I’m currently with @ermine over at SLIS and also give a 60% odds that this ends badly for equity markets.

However, in every crash there’s opportunity, and if a syncretic, hybrid neuro symbolic AGI, or even, eventually, ASI, ultimately emerges out of the failure of scaling with LLM only approaches; then in the longest run that’ll end up being all to the good for the field of AI as a whole. The Magnificent 7 rose up as much from the ashes of 2002/3 as they did the boom of 1995-99.

Google’s AI systems processed a quadrillion tokens (1,000,000,000,000,000) in June 2025 (1 token = ~0.8 words in English), more than double the volume (480 trillion) in May. [X/Demis Hassabis].

Data centre (DC) Capex to 2028 now pegged at $2.9 trillion, with $1.3 trillion for GPUs alone, with DC Capex hitting $900 billion in 2028 itself, nearly equal to the entire 2024 Capex of the S&P 500 ($950 billion).

Other sources estimate a current pace of AI development scenario DC Capex of $5.1 trillion cumulatively by 2030, or $7.9 trillion in an accelerated AI race (i.e. with China).

How long without meaningful revenues before hyperscalars etc reign in spending?

Given its lengthening lead in specifications and chip performance, its software moat with CUDA, and its 80% share of global GPU sales; I’d guess that by the end 2030 Nvidia would be more likely (adjusted for any more splits) to be either over $1,000 per share ($25 trillion market cap, so 6x from here) or under $100 per share (40% below current cap) than somewhere between those numbers.

Some useful links on hard limits to LLM scaling, each explored in a rigourously quantifiable, but still thoroughly accessible way:

Toby Ord (effective altruist):

https://www.tobyord.com/writing/the-scaling-paradox

Key scaling wall – unsurprisingly it’s LLM accuracy improvement: “accuracy scales with the amount of compute used as the 1/20th power.”

This is totally hopeless.

By my back of the envelope calculation, if we took the most extreme example imaginable, and used for LLM training purposes the rest mass energy of all the stars in the whole observable universe every second, which is about 10exp70 watts (given: ~10exp23 main sequence stars in 10exp11 large galaxies accessible to our observation, each with ~10exp30 kg mass, and with nearly 10exp17 J/kg, using e=mc2); then, for a 10exp60 fold increase in compute (i.e. 60 orders of magnitude!) over the 10Gw (10exp10 watts) used now for LLM training, we’d only get a 1,000 fold (10exp3 being the 20th root of 10exp60) improvement in LLM accuracy.

Without considering the real world training data constraints, chip fabrication limits etc, we can see immediately that LLM only approaches must soon be at a dead end.

Not opinion. Just maths.

Then we need to factor in those various practical issues, as Epoch AI nicely review here:

https://epoch.ai/blog/can-ai-scaling-continue-through-2030

Whilst these are not actually hard and fast physical limits, like LLM’s unavoidable diminishing returns for accuracy with scaling, they still represent a compelling series of economic constraints.

Given all this, why aren’t the big players pivoting hard and fast from LLMs into symbolic language approaches to AI realisation?

The obvious answer is ‘maybe they are’, but, if so, then:
– What do they gain from not going public? Nothing which I can see.
– How have they managed to keep it secret? Impossible in practice given both the level of interest and the numbers of researchers involved.

This means that these firms are, in effect, in the process of collectively driving off a cliff.

Whilst no individual lemming ever got bad publicity, the market is not going to forgive this one lightly when the reckoning eventually comes.

I’ve updated my prior sharply downwards for the $2.9 trillion (2028) or the $5.1 trillion (2030) of data centre Capex actually turning up.

If so, then Nvidia is toast.

My deep, abiding and profound fear here is that I’m ‘missing it’, by ‘just not getting it’, and/or that I’m making a category mistake – i.e like I did with BTC, when I could have brought it in 2015 at a low three figures, instead of six figures as now, just a decade on.

For every “AI is snake oil” perspective (like this one here):

https://www.aisnakeoil.com/archive

one should also go read something from an AI booster.

Truth is not a balancing act, but finding it does involve keeping an open mind.

Just not so open though that your brain falls out 😉

Thank U for the kind words @ermine 🙂

Tempted to respond to the challenge with a list including likes of PLTR (which I’ve been HoDLing, and to date is my best individual stock performance ever), or perhaps something like Cadence Design/CDNS.

All the big players involved in data centres, cloud or chips more generally have been making out, or are expected to make out, like bandits on their underlying fundamentals and on price performance.

It doesn’t much matter whether you’re looking at Nvidia, AMD, ARM, TSMC, ASML, AMZ (with AWS), MSTF (Azure), Alphabet (Google cloud), or at the related support businesses like CEG (data centre energy), Applied Materials, Broadcom, Vertiv (cooling) or Arista Networks (or, for that matter, at the supposedly ‘AI’ driven cyber security plays like Crowdstike, Zscalar, Sentinel One and Palo Alto Networks).

Heck, even dinosaur Oracle is having a big comeback off of cloud.

It feels like the only way is up.

A dangerous feeling to have. Sooner or later someone will have to pay the price for complacency. Maybe that’ll be me.

Some, like Nvidia (falling from 50x to 40x forward PE, even as the price goes up), do look more or less ‘reasonably’ priced, whatever the h**l “reasonable” means in US large caps these days.

Others though, not so much…

As the Sherwood article in this W/e’s reading links shows, PLTR is the poster child for bonkerist valuations in this ‘cycle’.

In fact, it’s quite possibly actually even worse than the fig’s in that piece.

I make Palantir having an over 600x trailing and an around 300x forward PE, with 180x forward Free Cash Flow.

Insane in Brain, as Cypress Hill would have put it.

Of course, Alex Karp would probably say that his/Thiel’s company will be a 2, 3 or even a 4 trillion dollar market cap by 2035-40, and by then giving MSTF a run for their money.

We’ll see I guess.

Of course, the pure play LLMs are a basket case’s basket case. Their financials are nonsense on stilts. Boris’ inverted pyramid of piffle. Open AI’s cash flow statement must make a 2021 SPAC look respectable.

Strange times we’re living through! 😉

Okay. More details in the FT yesterday on that Morgan Stanley report on data centre Capex.
– capacity increasing by 6x by 2030
– of the $1.5 trillion shortfall in the $2.9 trillion spend by 2028: $800 billion to be met by private credit asset based finance and debt funding joint ventures (for comparison the entire private credit market is $1.4 trillion now), $350 billion to come from Sovereign Wealth Funds, Private Equity and Venture Capital, $200 billion from corporate debt issuance and the last $150 billion from securitised credit/asset backed securities.
– Of the $2.9 trillion total $1.3 trillion goes on the build out and $1.6 trillion on GPUs.
– The total doesn’t include the cost of new power infrastructure.

Leaving aside that this looks..ahem….just a tad bit..er….ambitious (e.g. the $900 billion in data centre Capex in 2028 alone is not only nearly the entire $950 billion total Capex of the S&P 500 in 2024, but it also completely dwarfs the peak Technology Media and Telecommunications sector spend of $135 billion in 1999/2000, at peak Dot.com):

Then, *if* this actually ever comes to pass, we’re then looking, by my back of the napkin maths (which, frankly, is perhaps no more nor any less grounded than I suspect the Morgan Stanley’s guesses are here) at something in the order of:

a) $1.6 trillion on GPUs cumulative 2025-28 = ~$500 billion in 2028 alone
b). $500 billion x Nvidia share of bleeding edge GPUs @80% on margins of 80% (well above its average for all chips) = ~$320 billion in profit for Nvidia from just high end GPUs for LLMs in 2028.
c). Add in gaming chips etc and maybe $350-$375 billion profit for Nvidia in 2028.
d). On normal multiples that suggests something like a market cap of about $10 trillion. But, on last year’s 50x forward PE, maybe it could then be closer to $20 trillion. That in turn gives nearly 5x-10x from here by the end of H2 2028.
All of which is, frankly pretty hard to believe, but that’s what the Morgan Stanley scenario implies to me if it ever panned out within its timeline.

I’ve now gotten a better handle on what a decrease in test loss score actually means per the reference in Toby Ord’s January 2025 piece to the “key graphs” from OpenAI’s 2020 paper “Scaling Law for Neural Language Models” (copy 2020 paper here):

https://arxiv.org/pdf/2001.08361

So test loss is cross-entropy loss, measured in nats (natural logarithms, using Euler’s number “e”).

One converts cross-entropy loss to perplexity which is defined as:

\text{Perplexity} = e^{\text{loss (in nats)}}

A test loss score of 6 means e raised to the 6th power (which is 403) wrong guesses for every right one, i.e. the model is about as clueless as someone picking randomly from 403 options.

And a test loss score of 3 mean e raised to the 3rd power (which is 20) wrong guesses for every right one.

So, whilst it takes a million fold increase in computational resources to go from test loss score of 6 to one of 3, that’s a lot more than just a 2 fold increase in accuracy (i.e. it’s not a test loss score of 6 divided by one of 3 factor of improvement).

Still, it is only a 20 fold improvement (i.e. from 403 wrong guesses down to 20 wrong guesses per right one).

A test loss score of 0.01 would mean about only 1 percent of first guesses would be wrong.

That would require 3/0.01 raised to the 20th power (i.e. 300exp20) more compute than a test loss score of 3, which actually is 3.4x10exp49 times as much.

Although this is (much) less than the back of the envelope figures in my example at #97 above, it is still utterly hopeless, and not just hard.

Indeed, it’s impossible in the physical universe as we know it (Lloyd’s Limit, Landauer’s Bound, etc.)

So, the point remains, we’re never getting to a robust and reliable AI that can substitute for all human cognitive labour (say a test loss score of 0.01, plus fully generalisable) via LLMs on their own.

Dyson book – Analogue computers:

A very nice summary on Veritasium here of AC from just before (March 2022) the start of the current deep neural net / LLM ‘explosion’:

https://youtu.be/GVsUOuSjvcg?feature=shared

Surely AGI if/when it arrives will be:
1. Part digital and part analogue
2. Part classical (for general purpose computing) and part quantum (for very specific tasks, where quantum offers exponential speed ups)
3. Part formal logic, symbolic reasoning and programming; and part neural nets, weight adjustment, back propagation and inference.
And:
4. Where digital, then using a dynamically optimised mix of:
a. general-purpose chips: CPUs, GPPs and MPUs;
b. graphics & parallel processing: GPUs;
c. AI & machine learning chips: IPUs, NPUs, TPUs and LPUs;
d. embedded systems: MCUs, SoCs and ECUs;
e. signal processing: DSPs; and,
f. custom hardware: FPGAs and ASICs

Largest AI firms’ Capex has increased to 1.3x EBITDA on a trailing 12 months’ basis, compared to a 0.5x average for all Capex for the rest of the 100 largest companies in the S&P (Bloomberg). If the Mag. 7 are no longer going to be Capex light, with FCF returning to shareholders in buybacks, or held as cash, then valuations rest only on future growth.

Manchester and London IT (14% discount to NAV) currently holding >90% in biggest US AI plays (39% in Nvidia and 23% in Microsoft for Open AI and Azure cloud). That’s a high conviction allocation if ever I saw one. Puts even likes of PCT, SMT and ATT in the shade!

MS just estimated LLMs could deliver $920 bn in annual net benefits to S&P 500 firms, with ~$3 tn in LLM-related capital expenditures needed to realize these gains. Over time, this investment could generate $13–16 tn in new market value—a 24–29% boost to the index’s capitalisation. Sectors such as staples, retail, real estate, healthcare, and transportation may see savings that exceed anticipated 2026 pretax profits. The bank estimates about 90% of jobs will be affected to one degree or another by automation or by augmentation, mainly to enhance rather than replace roles, and less on the ’embodied AI’ front (robots) at this stage.