Terence Tao's ChatGPT conversation about the Jacobian Conjecture counterexample

It’s endlessly fascinating to read the AI transcript of an expert who _really_ knows how to cut to the chase. It just shows how much you can potentially squeeze out of these models. I’m also surprised to see that even Terrence Tao seems to use it in a way that resembles, in progression, how I use llms in my area of expertise (emphasis on progression and usage patterns, not absolute skill, obv I don’t match that): short pointed questions that goes all in on the jargon and machinery of the field and steers the llm hard (eg no softballs). I’ve noticed that llms switch their tone and meet you basically more or less on your level.

This is the second ChatGPT shared conversation I've seen today that is truly fascinating.

The first one was someone proving another conjecture false by just repeatedly saying "keep going" to ChatGPT: https://x.com/DmitryRybin1/status/2079904005652893709

What a world we live in.

Terrance Tao's chatgpt conversation is really interesting for a variety of reasons:

1. The counter example wasn't just a brute force selection, the polynomial is structured in a very specific way that ends up getting the result.

2. Terry Tao's questions are very specific and prompts the AI in a useful way, that without high math training you are not going to get the same information out of it. Terry seems to see some aspects of the problem and counter example and uses AI to brute force some parts of it.

It's crazy how he suggests simplifications over and over and gets led through the finding. Absolutely bonkers how you can use AI to understand something and map it to your own mental map so efficiently, and of course he's most interested in generalizing or finding a simpler sub-result that would explain it.

Just awesome to see new knowledge hit an incredible mind like this. Having these "what if" discussions is what I miss most from JPL and academia.

tao transcricpt, maybe cannon , 100000 taos or more clearly now in data centers, agi marker.

Math has some of the most insanely dense and impenetrable nomenclature. I can generally keep my head mostly above water or at least near the surface reading from most STEM fields, perhaps leaning on google/wikipedia a bit, but man, mathematics just so quickly decouples from all common tractable understanding it's insane.

Sorry it's a bit of an aside, but I imagine many other otherwise "technical" folks feel the same unfamiliar sense of total loss like when encountering hard mathematics.

What was most remarkable to me from this transcript, was how strong of an equal the AI agent comes across compared to the user (Tao). And Tao is one of the top mathematicians of modern times.

Yes, Tao is guiding it to where he wants to go. But also, Tao is actively learning from it and relying on its explaining, analysis, and inference abilities. You can easily imagine this conversation having taken place between Tao and a PhD thesis student, or even another professor, explaining their results.

What can we imagine and predict about the future anymore? Maybe a year - or two model releases - from now, the AI assistant will be undeniably stronger than Tao, and not an equal anymore.

Jeez. While I obviously can't talk at all about the math, I've noticed a few things:

a) The model thinks on some questions while straight answers on others. (I wish I'd knew from the questions if this is somehow correlated to hard tasks or "inventive" tasks, but that's way out of my league).

b) The model sometimes pushes back. Again, I'd wish I knew if it was warranted, but I counted 2 instances where it said "yes, but with caveats", one where it said "mostly yes but with this correction" and one where it said "careful here, because x y z".

c) The model did q&a + pdf ingestion + code writing + more q&a + thinking + more q&a, for a looong while, while seemingly staying on topic (at least Terrence Tao seems to think they're still productive, so I'll trust that).

This is what model progress is, not number goes up on xBency or yBencher. Damn.