My yap
Essay

Will we get AGI by 2030?

This is part of my application to the Cambridge AI Safety Hub Alignment Fellowship, where we were requested to write a 200-400 word reflection written for a reader who is familiar with the piece.

This is the reading

Will we have AGI by 2030?

A first, eyepopping claim is “Length of tasks AI can do doubling every 4 months”.

Doubling.

A seemingly relevant idea is Moore’s Law, which states the number of transistors on a chip doubles roughly every two years, leading to faster, smaller and cheaper compute power. For fifty (over) years, this held. Now, physical constraints on transistor sizes could dislodge this trend.

Point is, extrapolation is a hard thing to do. We are at right now where Gordon Moore was in 1965. The key factors that drive AI progress are “larger base models, teaching models to reason, increasing models’ thinking time, and building agent scaffolding for multi-step tasks.” The first sounds familiar: larger base models means more parameters. (See the link to more transistors?)

Difference is, the scale is a lot faster, and it looks like even cost would not deter the labs from training better models.

It is a cause for concern how AI has been moving so rapidly, and coupled with our [humanity’s] gift of extrapolating curves and fearing the worst, a pause in AI research and development should be honoured by the frontier AI labs, as we as a tribe relearn what it means to be human again.

Are there physical limitations to teaching a base model reasoning and then providing it human feedback (RLHF)? Perhaps the bottleneck might become “can we find good data labellers”, as it becomes important to get accurate ground truths for the models to learn from. And honestly, I think we might run out of PhDs that are willing to be data monkeys doing data labelling.

But that begs the question: would AI get good enough to label data for itself?

I think that is the necessary breakthrough for AGI to happen: once AI (and I am using it as a broad term here) becomes competent to direct its own training, AI becomes infinitely close to human. In my opinion, our ability to draw links (unprompted) and derive insights (ie. a proxy for creativity) is (or was, depending on when you read this) a defining feature of what makes us human. It is scary for AI to creep closer to that every day.

Another concern is explainability: if we can’t even understand what AI is doing now, can we really understand it when its empowered enough to drive itself (AGI), and possibly, us?

The reading said we will eventually hit a crossroad by 2030: either progress slows by a lot, or we go full “rocket boom!”

Me? You’re asking me if we will get AGI by 2030? I think we should delay reaching the crossroad, so more individuals, government, and the world, can be more prepared for an inevitable decision.

Right sorry, I have to answer the question. I hope we don’t? But as a certified human (and therefore a curve line graph extender) I think we might, and with how much frontier labs are sandbagging their best models, we might already be there. We just don’t know yet.

Until then, maybe it’s time to think about what makes us human.

I am also very happy to say that I did not use AI for this at all. Yay?

My yap