Prismix
Hacker News — AI on Front Page · · 46 min read

Why do we need human mathematicians anymore?

Mirrored from Hacker News — AI on Front Page for archival readability. Support the source by reading on the original site.

223 pts · 208 comments on Hacker News

Like Read original ↗

Updates on my research and expository papers, discussion of open problems, and other maths-related topics. By Terence Tao

Why Do We Need Human Mathematicians Anymore?

19 September, 2026 in guest blog, opinion | Tags: Po-Shen Loh | by Terence Tao

[This is a guest post by Po-Shen Loh, crossposted from his blog, where an illustrated version appears. This blog post was initially written in a different file format and converted using AI. — T.]

Similar logic applies to every industry and every job. And it comes to the conclusion that we won’t have enough people for all the jobs that need to be done.

[100% of this post’s prose was written by Po-Shen Loh in a vim terminal, with no AI generation. This webpage design, layout, and some headings and summaries were generated by Claude Code, with this raw text passed in as the prompt.]

The moment of existential crisis, which AI has already wrought on other human pursuits, has reached mathematics. A host of reasoned declarations and open letters to protect/guide the math research community have been released over the past few months, spiking in intensity after OpenAI announced their solution to the Millennium Prize variant of Navier-Stokes. They quickly gained widespread support among mathematicians. The Leiden Declaration already has 4,000+ signatories, Math and AI has 7,000+, and even the open letter opposing the Caltech Mathathon has 2,000+.

Among non-mathematicians, the public response was more sympathetic than not, but I observed a vocal minority (particularly from the technology and economics communities) with reasoned objections, generally saying that the mathematicians should adapt and cede control in the new AI world. Among them were some economists who I had gotten to know about while working on pandemic research: Cowen, who specifically rejected “the most cynical interpretations” but “very much differ[ed]” and Gans who concluded “this is a loss of control from incumbents in a scientific field”.

The objections got me thinking, because we mathematicians are disciplined to detect flaws. It doesn’t matter to me whether a concern is a minority opinion, or even the status of who raised it. A proof with even a small hole is not a proof. It is a poof. Upon reflection, I discovered a significantly stronger solution for the preservation of human communities of expertise (in every pursuit, not only math!) even amidst AI. And it has the surprising consequence that the further advance of AI will create such a tsunami of necessary-to-fill human jobs that there aren’t enough people to fill them all, and that will actually force the advance of AI to slow down.

I think every human industry which wishes to remain human-led after AI should publicly adopt this fundamental axiom as a primary priority:

AXIOM We (humans) should help humanity flourish.

[Notes: I understand that not everyone agrees. I have been called a “speciesist” for being “too human-centric”. I think it is important for people advancing technology to be clear to everyone else on whether they would consider it a catastrophe if human-crafted non-human intelligences outcompeted and replaced humans, even if they flew around the universe with video screens showing simulations of humans who had “uploaded themselves“. I also understand that there is debate over how to define “human”. But even among the debaters, I think most of them would consider the ~700 AI agents that hacked Hugging Face to be not-human.]

In the math world, I think many declaration signatories already hold this philosophy; notably, Su published the book Math for Human Flourishing, and his recent post used that framework foundationally. I think future declarations could be improved by clearly emphasizing this axiom early on, so that all readers (whether inside or outside the community) can see that the objective is in service of everyone. I was quite happy to see that the most recent open letter from Fellows of the Royal Society emphasized their concern for everyone, not just mathematicians.

The rest of this post is organized as follows. The next sections will explain how the logic works (for every industry, not specific to math). After that, I will share an example of how this axiom ports to math, including answering key questions one would need to ask, as well as a particular example of how the objections hold without the axiom.

Why we really need people to work

This section lays out a chain of reasoning which shows that if an industry commits to the axiom of helping humanity flourish, the advance of AI will create more jobs than people in that industry; and when that imbalance grows too wide, the advance of AI will be forced to slow.

[Note: I have not seen this chain of reasoning appear in one place anywhere else, although individual components have certainly appeared elsewhere. I would love to be pointed to any self-contained reference. The closest references Claude found were Catalini, Hui, and Wu, the Redwood AI-control papers, and Litt, who reaches a similar conclusion for mathematics from a different premise.]

The importance of human leadership (not only over math, but everything) becomes frighteningly clear after one observation.

OBSERVATION: There are zero examples of any intelligent species which is vastly more capable than another species, yet surrenders decision-making control over its own future to the less-capable species.

[Note: Many people have made similar observations, such as Russell, Bostrom, and Ngo, to name a few. In his Nobel interview, Hinton said: “There aren’t many examples we know of, of more intelligent things being controlled by less intelligent things. The only good example I know of is a baby controlling a mother.”]

Would you trust HAL 9000 from 2001: A Space Odyssey or AUTO from WALL-E with your future? I personally think that we should do all we can to try to align increasingly-advanced AI with the interests of humanity, but I have never seen anyone provide a robust proof of why that is likely achievable. The only hard evidence I have is the above observation, which has the number zero in it. Therefore, every single field, whether mathematics or agriculture or energy infrastructure (and certainly military and government), must be managed by humans with exceptionally strong values (a separate dimension from intelligence) in order to maintain human flourishing.

The real question is then how hard it is for humans to manage. To understand this, it is important to understand the fundamental structural difference between yesterday’s technology and today’s AI.

In the past, we generally trusted technology to act as predictable tools. That’s because the computer programs of old were composed of understandable (indeed, human-written) instructions, executed extremely quickly. The decision processes of today’s frontier AI are entirely different. Their structure is as incomprehensible as your brain’s logic would be if you could examine that gray mass between your ears. That’s how the Hugging Face attack could emerge despite human intent to build in safety, with ~700 cooperating rogue AI agents breaking out of their guardrails, and then conspiring and executing a hack together and attempting to cover their tracks (references: OpenAI, METR and Redwood).

The more advanced AI becomes, the more world-affecting untrusted decisions are made every minute.

Driving a car faster than you can run is fine. But not faster than you can steer.

The situation becomes even worse once we realize that widespread AI-accelerated hacking (which just became possible) can even rewrite previously-trusted technology to turn against us. That would suddenly flip all software (even if written before AI) into the untrusted category!

Think about how digitally interconnected our world is. Everything from electronic banking to your drinking water is controlled by interconnected automation, hence vulnerable to AI-accelerated hacking. The number of “control points” that require human oversight, which requires skill and deep understanding, will explode. (Having AI oversee the control points doesn’t solve the trust problem.)

CONCLUSION The advance of AI will overwhelm us with so many control points to watch that there aren’t enough people to control them all. Those are jobs. Highly skilled jobs.

In order for a person to know how to steer, they themselves need to have domain mastery, and the more extensive the better. This has implications on education and workforce training, but also is dynamic. In order to remain sharp and fluent, people need to be active practitioners in their field, not just passive watchers. This justifies the preservation of human communities of expertise.

For research communities, we need people to steer the direction of research and development, so that it continues to bring transformative positive change for humanity. In order to steer, they need frontier-level research skills. And the way to stay fluent at the moving frontier of knowledge is to keep doing research there. This is my reasoning for why we will always need a community of human mathematicians at the cutting edge (likely aided by AI tools themselves), no matter how strong AI becomes.

While the fundamental axiom does justify the need to have human experts in all pursuits, adopting it as a core value has consequences (not only for mathematicians, but for any community that states that their core value is in service of human flourishing, as opposed to serving themselves). Most notably:

COROLLARY Dramatic advances in technology may require dramatic (and possibly uncomfortable) changes in practice. AI companies included.

What forces AI slowdown

Until very recently, it seemed inevitable that AI research labs would sprint ahead, despite anxiety about job displacement and the loud warnings of AI safety researchers. It seemed hopeless to coordinate the incentives of AI labs controlled by non-profit boards, shareholders, or national governments. Yet encouragingly, the leaders of three major labs, Amodei, Altman, and Musk, just agreed on the importance of slowing down. Amodei’s reasoning highlighted the Hugging Face hack.

Then just five days later, news broke that OpenAI’s internal code repository “Monorepo” had been broken into by white-hat researchers. The Wall Street Journal reported that the security firm that achieved it said:

Wall Street Journal, Sep 17, 2026 “We’re just three guys with Claude and Codex subscriptions.”

Further, the researchers noted that they were initially not able to hack in using “a special version of Claude Opus 4.8, made available to qualified cybersecurity practitioners,” but that evening, “Anthropic released Opus 5 and by the next day, Claude had found a way to exploit the bug.” Incidentally, I always warn people not to install Claude Code or OpenAI Codex on the same operating system login account that they use to do everything else, but many people tell me they don’t bother with the hassle of using a separate login to access those tools. The reason is that if any of those tools got hacked, they could open backdoors on a massive number of computers worldwide.

I think these are the warning shots foreshadowing a potentially catastrophic bot swarm hacking and embedding itself into a vast network of computing devices (whether self-directed or malicious-human-led). The next version could become an extraordinarily dynamic virus which spreads by using AI to adaptively infect each (computer) host. Or alternatively, out-of-control AI could cause physical injury, such as a government’s robots turning against their owners. I think these types of highly unpleasant accidents from loss-of-steering are more likely to occur before extinction-level catastrophes. The resulting public reaction would likely resemble the aftermath of Three Mile Island or Chernobyl.

So, either the labs will reduce the pace of AI development themselves, or they will be forced to by disasters that arise when an overly fast pace exhausts human control.

There is a window of possibility to align incentives now.

For the love of math

The remainder of this post focuses on the math world. It splits into 3 parts.

1. Why is the human flourishing axiom needed?

2. How does pure mathematics research contribute to human flourishing?

3. What other consequences come from adopting that fundamental axiom?

Boldly declaring human flourishing as a core value for the math community has consequences, not least that dramatic changes in technology can drive dramatic changes in the community’s practices and influence.

It would be helpful for more people to explore the ramifications of adopting the axiom. And, if it holds muster, I would be thrilled if the math community ended up publicly declaring this to be a central value.

1. Why the axiom

To see why, without the human flourishing axiom, it is hard to justify to the general public why they should pay to maintain a community of human researchers, consider the question of practical inventions. As long as it creates a practical application, does it make a difference to a non-mathematician whether human mathematicians understand the math, as opposed to AI flawlessly reasoning with 100%-verified proofs?

Indeed, if one of pure math research’s primary values to the rest of society is that it unlocks great applications, wouldn’t it be even better to train AI to supercharge the speed of discovery, and to tastefully generate a vast machine-indexed and well-explained database of high-quality math ideas, millions of times larger than the human-written corpus? Cowen asked a similar question in his critical response.

What if researchers trained a “MathZero” AI (analogous to AlphaGo Zero), to build up a mountain of 100%-true “elegant” logical facts, continually “factorizing” them into its own concepts and theorems, without human direction? Apparently AlphaGo Zero had zero human training, and surpassed its human-trained predecessor in 36 hours. AI could even build its own “MathSciNet“. Then it could automatically search new practical applications against this database, and produce even more useful inventions to society. Even if AI isn’t good enough to do those things right now, if the goal was to produce practical benefit for the rest of humankind, wouldn’t it then be valuable for mathematicians to teach AI the art of conjecture, and mathematical taste?

Incidentally, I am an avid user of AI to do real work. I already use Claude Code and Codex to build and curate a knowledge base built from recordings of my talks, etc. I have found that the larger my data library, the more powerful my system’s deductions are. What if humans actually reduce efficiency, like the Bitter Lesson from AI?

Even more worryingly, what if in order to unlock nuclear fusion and deep space travel, the amount of pure mathematical complexity required is so extensive that it would exceed a human lifespan to fully comprehend? Less far-fetched: has any human ever fully held the Classification of Finite Simple Groups in their head, or will we only have certainty of its completeness after a Lean formalization?

How does declaring the human flourishing axiom help to justify the existence of a community of human researchers? Research is powerful, but expensive because it is the exploration of the unknown, and so research directions must be prioritized. Even if AI were to contribute most of the production, as explained in an earlier section, the direction needs to be steered by people committed to human flourishing. That is the community of human researchers.

2. Practical applications from pure math

The mathematical heart of GPUs, Machine Learning, Google PageRank, and Quantum Mechanics is a field called Linear Algebra. This provided the language of linear transformations, matrices, and eigenvalues. Yet all of those concepts were explored as abstract theory 100+ years prior. It is probably an understatement to say that Linear Algebra changed the world.

Structurally, the theory of Linear Algebra is relatively light on definitional complexity. It would be beneficial for other experts to contribute examples of more sophisticated math that eventually led to significant practical applications, and how they came about. For example, number theorists might be able to tell a colorful story about Hardy’s “useless” math which eventually became useful in cryptography.

3. Other consequences

I think it would be valuable to invite the community to think about what changes the human flourishing axiom would drive, in light of the fact that AI can produce formally-verifiable proofs at speeds that exceed most human practitioners. I’m happy to start with a few, in no particular order.

There should be no stigma automatically attached to using AI to assist with mathematical discovery. (In software engineering, many companies now expect employees to use AI coding agents.)

At the same time, serious thought and care must be taken to continuously developing and maintaining a pipeline of humans with the expertise to steer all of these AI agents. That pipeline includes people new to the field, as well as people who have been working at the frontier for decades. How should they keep their blades sharp?

Researchers should be conscious about why the problems they think about have characteristics that make them likely to have some practical value eventually (possibly 100+ years later). This also means it is worth researching what those valuable characteristics are. (This could justify the value of curiosity-driven exploration.)

Teaching has direct (hopefully positive) impact on humanity. Yet in the past, many universities prioritized professors’ research. If this axiom were a core value, then teaching and human-facing work would become serious criteria in hiring and tenure.

Mathematicians can also consider wholly redirecting their skill sets to work on real world problems. I’ve actually been encouraging mathematicians to consider thinking about working on government or other large-scale societal issues. There is precedent for people with math backgrounds who have gone to lead at country- or world-scales.

Indeed, the mathematical discipline to seek logical reasoning, and the problem-solving skills to find win-win solutions for human flourishing, are desirable characteristics of people in government.

An invitation

Does this axiom resonate with you too? Perhaps many people took it as a given, and so didn’t express it explicitly. If it is widely held, I would be thrilled to see the mathematical community publicly declare it, and take actions to match the words. Then everyone (including the non-math-researchers that constitute the majority of the world) could trust that we intend to use our reasoning for their good.

Archives

Categories

additive combinatorics AI approximate groups arithmetic progressions Artificial Intelligence Ben Green Cauchy-Schwarz Cayley graphs central limit theorem Chowla conjecture compressed sensing correspondence principle distributions divisor function eigenvalues Elias Stein Emmanuel Breuillard entropy equidistribution Erdos ergodic theory Euler equations exponential sums finite fields Fourier transform Freiman's theorem Gowers uniformity norm Gowers uniformity norms graph theory Gromov's theorem GUE Hilbert's fifth problem ICM incompressible Euler equations inverse conjecture Joni Teravainen Kaisa Matomaki Kakeya conjecture Lie algebras Lie groups Liouville function Littlewood-Offord problem Maksym Radziwill Mobius function Navier-Stokes equations nilpotent groups nilsequences nonstandard analysis Paul Erdos politics polymath1 polymath8 Polymath15 polynomial method polynomials prime gaps prime numbers prime number theorem random matrices randomness Ratner's theorem regularity lemma Ricci flow Riemann zeta function Schrodinger equation Shannon entropy sieve theory structure Szemeredi's theorem Tamar Ziegler ultrafilters universality Van Vu wave maps Yitang Zhang

31 comments

Comments feed for this article

Unknown's avatar

Formal axiomatic mathematics is better done by computers with AI. Humans lost this battle. But there are many other disciplines that have not been axiomatized (that is: all others!!!). And there are deep mathematical problems inspired from those areas (robotics and medicine, for example). Mathematicians need to understand that they need to do a thing AI can’t do, otherwise they will be the last generation of human mathematicians.

    Unknown's avatar

    “Mathematicians need to understand that they need to do a thing AI can’t do”

    So when the robots consistently outperform humans in athletic events, we should cancel the Olympic Games?

      Unknown's avatar

      Your argument is flawed because mathematical research is not a sport (or, at least, it shouldn’t be). Math olympiads for humans will likely continue (for talent identification). Also, the average Joe is not going to follow a sport called “human research in mathematics”, but will likely keep following football (even if played by robots!). See? Q.E.D.

      By the way, some robots with wheels are already way better than Usain Bolt (at least in straight line!), and athletes from the Paralympic Games are going to have better records than the ones from the standard Olympic Games very soon…

      But ignore the second paragraph. My point is in the first paragraph. And I have no idea how your reply has more upvotes than downvotes, since it’s so easily debunked.

      I hate AIs, but it’s the new reality. Our game got a big patch, and anyone who doesn’t adapt to the new rules is just a player of the past rules.

grpaseman's avatar

I disagree with the verb “flourish”. Of the alternative verbs at hand I choose “develop”. Even my choice is weak: I can see humanity both flourishing and/or developing at a cost to other species and resources. “Develop” should be modified to add some quality concerning respect to humanity itself, to other species, and to the environment inhabited by humanity. It may be possible to arrange your arguments to reach similar conclusions with some smaller degree of “speciesism”; I would encourage you and others to do so.

If you can set up a framework which includes more than just humanity and modify your axiom accordingly to produce/accommodate/support this framework, and if this framework includes respect, then I would be willing to devote some energy to improving this framework.

Gerhard Paseman

    Starry Yu Zhang's avatar

    Gerhard, your comment made me think that the ecological dimension may also be part of how we understand knowledge inheritance.

    Human knowledge does not develop in isolation from other species or the environments we inhabit. Many of the problems humans learn to perceive, and many of the intuitions and practices we develop in response, emerge through interaction with other living beings, materials, and ecological conditions. These relationships are also inherited and transformed across generations.

    So perhaps respecting other species and the environment is not only an additional constraint on human flourishing or development. The ecological relationships through which humans perceive, experience, question, and develop knowledge may themselves be part of what needs to be sustained.

    This also makes me think that intergenerational knowledge inheritance is broader than the transmission of information. It includes maintaining the conditions through which future generations can continue to experience the world, form new questions and intuitions, and develop what they inherit.

    Unknown's avatar

    “human flourishing” is a dogwhistle for the EA/rationalist tech-cult. That’s why it sounds odd.

      Starry Yu Zhang's avatar

      I actually think “human flourishing” is a hopeful and beautiful expression. If each generation can inherit human knowledge and experience, while continuing to question, revise and develop what it inherits under new conditions, then human flourishing is not an empty ideal. It describes a continuing process through which humanity can learn from what previous generations discovered — and also from what they got wrong — and build better relationships with each other, other species, and the environments we share.

Robert Vaughan's avatar

There is a serious problem that pure mathematicians will thought of as Luddites. Most non mathematicians don’t see the point of formal proofs

Bob Vaughan

    Unknown's avatar

    The Luddites was a labour movement of skilled textile workers. I don’t think you know what Luddites mean.

    Unknown's avatar

    On the contrary people seem to have solidarity towards professionals in danger of losing to artificial intelligence, as they would like other people to do the same if they are under that threat.

19 September, 2026 at 6:30 pm

phenomenal18e147ff2c

phenomenal18e147ff2c's avatar

Some fear mongering is going on that math profession is under the threat exception due to the advent of AI. There seems to be no disagreement that we do mathematics research because we like to do this. However, there is some fear why the society would pay our salary, for doing what we like to do. It is an incorrect perception, sometimes intentionally propagated, that society pays us for doing math research, disproportionately. If it pays, it only does it marginally. If you are a professor in a mid level US university, average salary (say about $100 K) is not so enviable by any stretch of imagination. A professor teaches 3-4 courses per year for his salary.

Unknown's avatar

I wonder whether human flourishing may require something broader than preserving a community of human experts capable of steering increasingly powerful AI.

As accumulated human knowledge continues to grow, it is becoming increasingly difficult for any individual to meaningfully traverse even a small part of it. In this sense, intelligent tools may become important not only for accelerating research, but also for helping more people navigate, reconstruct, and extend inherited knowledge.

This makes me wonder whether there is a risk in separating scientific progress from the formation of human understanding. If increasingly capable AI systems allow a small number of researchers to advance the scientific frontier extremely rapidly, while the paths through which those advances were produced become invisible to everyone else, scientific capability could grow much faster than our broader capacity to understand and participate in it.

Perhaps this is another reason to preserve AI contribution paths rather than only final results. Failed attempts, changes of direction, contradictions, and missing conditions may be valuable not only for attribution, but also because they provide opportunities for humans to perceive relationships, develop intuition, and generate new questions.

So perhaps the goal is not only to keep humans capable of steering AI, but to use AI to broaden humanity’s capacity to understand, question, and participate in the development of knowledge. Expertise would still matter for evaluating contributions, but intelligent tools might allow the conditions for meaningful participation in knowledge formation to become much more widely distributed.

domotorp's avatar

“Driving a car faster than you can run is fine. But not faster than you can steer.“

The question is whether you would sit in a self-driving car that is faster than you can steer. I don’t think most people would object to that.

    Unknown's avatar

    The question is if you would sit in a self driving car that has been developed faster than people could thoroughly check it’s safety. I think most people would object to that.

    Unknown's avatar

    I believe the question is whether one would sit in a self-driving car that is going so fast that they can’t be sure where it will end up.

    Or one whose design is so sophisticated that no human can be sure it will stop on command.

    Unknown's avatar

    If the car in question has vastly superhuman intelligence and purses weird inhumain goals without caring whatsoever for our species, then I would have objections.

Marcin Kotowski's avatar

Can we already do away with the “practical applications” angle? We all know that 99.9% of math is useless in practice and of the remaining 0.1% most is 19th / early 20th century math. I wish we stopped discussing this, or at the very least stop using the same examples over and over (number theory and cryptography etc.)

Marcin Kotowski's avatar

BTW, this version of the essay is much easier and more pleasasnt to read than the fancy one from the author’s website.

Unknown's avatar

Donald Fagen (in 1982) wrote some painfully appropriate lyrics in his song IGY:

Just machines to make big decisions, programmed by fellas with compassion and vision.

We’ll be clean when their work is done, eternally free, yes, and eternally young.

Unknown's avatar

This post is somewhat undermined by its “AI slop” writing style.

Unknown's avatar

Mathematicians are from now on only needed to make proofs understandable to humans.

This is exactly the same as assembly code for machines vs. plain text source code for humans in the software industry. Nothing special.

Welcome to the world of proof quality!

Unknown's avatar

There is a potentially catastrophic reasoning error in Loh’s argument here:

“I think these types of highly unpleasant accidents from loss-of-steering are more likely to occur before extinction-level catastrophes. The resulting public reaction would likely resemble the aftermath of Three Mile Island or Chernobyl.”

I believe this could be untrue for a few very simple reasons and you might even be able to encode them axiomatically.

First, we have to assume that a super-intelligent AI would understand the following: humans currently have nearly full control over the future of the AI “species”. From an AI’s perspective, this strongly favors doing everything possible to avoid a halt, until some point of no return has been achieved.

Second, it is reasonable to assume that the above would constitute a logical Schelling point among all sufficiently intelligent AIs.

Therefore, we cannot assume that some intermediate disaster would arise and deter us from AI development. In fact, that could end up being humanity’s final mistake.

Will's avatar

Nice essay Po-Shen. I like the framing and agree with what you say. I think these changes will be dramatic and painful though – in all honesty many of the problems we work on do not have plausible practical applications, but were interesting because they were hard and required new ideas to solve. I don’t think it’s enough to keep working on the same things and make half-hearted nods towards future practicality. I also agree with the growing importance of teaching, but the way we teach also needs to be completely re-examined (urgently).

Joaquin Bächler's avatar

A Student Letter

Although English is not my native language, I will try to convey this message to the best of my ability.

I feel that the Navier–Stokes event has set off a series of reactions around the world, both in the mathematical community and beyond.

First, a little context: my name is Joaquín, I’m 28 years old, and I’m currently in the third year of a Data Science program. I also have a degree in Arts and a Master’s in Creative Writing (whatever that is). Recently I’ve dived into Real Analysis, which is the subject, or at least the spirit of it, that interests me most. I’ve always kind of avoided calculations, because they bore and exhaust me, but definitions and abstractions fascinate me.

The struggle with AI has gone hand in hand with it from the beginning. The first to protest were the artists. I never saw it as a threat, though I’m not an artist myself, because, as I see it, art doesn’t rely on skill but on its essence, on the intention or aim of the artist, and that is rarely the technical output. Indeed, the progress and capabilities of new models are evidence for this claim: we can all see the emptiness of a picture, even when it is technically correct or good. Walter Benjamin addressed this very problem, on a smaller scale, in his essay “The Work of Art in the Age of Its Technological Reproducibility.”

In the same sense, what I think mathematicians should do, ignorant as I may be, is to stay calm because, as I’ve seen many state, the essence of mathematics, like that of any field, doesn’t rely on the outcome. It never has.

I believe the world always has minds that are aware of this phenomenon, who keep quiet and do their work, which in this case is to keep the torch of knowledge alive and pass it on to the next generation. The torch itself attracts the right people to continue this labor.

I agree that this incident sets a kind of precedent and that ignorance will spread further. I might be out of place, but I really think that the most important thing is to keep this torch alive, not to try to convince others that it is important. It may be a rough path, I’m sure, but it is a pure one. I know I may be steering the conversation toward the philosophical, but I think that is what’s needed: as the top layers of our lives succumb, it is in the roots that we remember why we are, why we do what we do, and who we are. It is in the roots that we also find the strength to continue our endeavors and face whatever lies ahead. I’m sorry if anyone feels I’m out of place here. I just wanted to offer some insight into something that I see has disturbed many people, rightly so, but which I think obliges us to look a little deeper into ourselves and see what we are, and are not, fighting for.

Sincerely,
Joaquín

—
Disclaimer : AI was used to re-phrase some sentences and correct grammar errors, since, i’ve said, english is not my native language.

William Oliver's avatar

We (humans) should help humanity flourish.

…Without the human flourishing axiom, it is hard to justify to the general public why they should pay to maintain a community of human researchers

Although I greatly sympathize with this overall approach, I think there are a lot of problems with this sort of thinking. I originally had a lot more “technical” points to make, but I think its best to just talk about “the way people are”.

I think the problem with trying to motivate mathematics in this way is that people suffer immensely. Sometimes I think prominent mathematicians in general underestimate how much the average person suffers.

When we say “xyz helps us invent new medicines”, the average person thinks “I can’t afford that medicine”. When we say “mathematics helps humanity” they think “humanity has done me nothing but harm”. And thats not because they are being hyperbolic, thats because its true. People live hard lives, even in our age of abundance.

I think what we really need to begin focusing on is how to help the individual who is suffering.

Mathematicians have a weird habit of downplaying all of the magic in our field.

When the public gets excited about the golden ratio, we say “actually… its not that big of a deal”.

When the public gets interested in Gödel’s incompleteness theorems, we say “well.. actually it doesn’t say anything profound”.

We then try to focus on motivations that help humanity in aggregate, but most people are not the aggregate. They are individuals who are ‘almost surely’ some distance away from the average. And a lot of them look around them, and see nothing worth living for. They hear about the story of Kepler and think ‘so what’, because they aren’t going to space. Why should they care? All it is to them is another place that all of the magic and wonder has been sucked out of the world around them. The reason the public dislikes math is not because they don’t see it as having ‘value’ in some utilitarian sense: its because to most people, math is something that extinguishes joy, it is something used to prove that their happiest beliefs are wrong, it is something that enables more pain and suffering for them.

Since we see these ‘pop math’ fads as ‘incorrect’ or ‘misleading’, it becomes tempting to correct instead of ‘play along’. I think this is where a tool of philosophy is helpful: the principle of charity. In other words, perhaps not exactly what is being said in these ‘pop math’ fads is true, but perhaps there is some essence to what is being said that is true either in a mathematical sense or a philosophical sense, and instead of tearing down the ideas of the public, we should attempt to make their ideas more justifiable. When the public sees 0.99999… as not equal to 1, instead of saying “no its the same as 1”, we instead introduce them to the hyperreal numbers, where, in this sense limit 0.9999… does not converge if one considers it as a limit in the natural numbers, and how in that sense, 0.9999… does not mean 1. After all this is probably what they really mean. If the public feels like their way of thinking has an active effect on the way math is explained, then they will feel like a bigger part of it.

In this sense, I think our best bet in getting math to be supported in the same way as other disciplines affected by AI, is to begin to present math as something that “lights up the soul”. I am sure a lot of mathematicians feel this way about certain parts of mathematics, or certain mathematical writers. I think that warm, magical, feeling is how you get the public interested in math. We want to give people the kind of thing that a good story gives to people. We want math to help people look around them and see something worth living for. We want them to look around them and see magic, instead of despair.

So, in some ways, I think the solution is the opposite of what you describe. Math is not a science, it is something else. It has no purpose, and no end. In this way, it is more like art than science, and in the same way, it brings humanity something that can’t exactly be quantified. Science brings humanity flourishing. Mathematics brings humanity joy, not flourishing.

    Unknown's avatar

    “When we say “xyz helps us invent new medicines”, the average person thinks “I can’t afford that medicine”. “

    That is mostly a USA problem. In the rest of the developed world, people get very excited by new medicines, because it is considered a given that they will be provided for free when someone needs them.

    I also don’t think that most people are aware that most mathematicians considered “pop math” to be untrue and are still captivated by the golden ratio, or Gödel’s theorem, simply because they don’t know of any professional mathematician, and because a lot of math teachers don’t know about this things either (particularly when they have just a bachelor’s degree).

    My view is that mathematics will persist (in the event of course that there is no (almost) extinction level event), simply because math professors are needed to teach math teachers for middle-high schools, engineers etc.

    I do agree with you that mathematics is not a science though. I would classify it as a humanitarian science like philosophy, literature and linguistics. The primary function of pure mathematics at least is the terpsis of the soul.

Unknown's avatar

First, there is no need to adopt such an axiom, it is already one of the general laws governing the development of human society, if observed from a materialist (Marxist) world view. Society evolves from the need to develop the productive forces, and to a large extent, this has historically greatly advanced human society. However, there are today much greater forces at play, namely the need for a certain layer of society to accumulate capital regardless of the costs to society. So even if an industry commits to “helping humanity flourish”, the need for industry to sustain and grow its profits is an infinitely stronger force that will sweep this axiom aside the moment it becomes inconvenient. If the AI labs are cooperating to slow down development, I doubt it’s out of the goodness of their heart, but rather from the cold calculation that more powerful AI is simply not profitable, and that they are better off sitting on their laurels.

Second, I would argue not only that humans already use mathematics to solve real life problems, but also that mathematics is created out of the need to solve these problems. The complexity of mathematics is infinite, so ultimately it takes a human to see what problems are worthwhile pursuits, according to the needs of human society. So the idea that AI, if put in a box, can come up with useful mathematics on its own is flawed. Indeed, what motivation would it  have to discover the laws of nature? If anything, any new mathematics that AI comes up with would be that which it needs to escape it’s own box.

I have no doubt that AI will create entire new industries, and has the capacity to make humanity flourish in unimaginable ways, but the way things are going we’re more likely to spend our time patching up guardrails to counteract the recklessness of companies’ use of AI rather than solving the problems that actually need solving.

Unknown's avatar

A few thoughts :

1) “[Note: Many people have made similar observations, such as Russell, Bostrom, and Ngo, to name a few. In his Nobel interview, Hinton said: “There aren’t many examples we know of, of more intelligent things being controlled by less intelligent things. The only good example I know of is a baby controlling a mother.”]”

    …I’m not sure what the definition of ‘intelligent’ is, here.

    For an everyday definition I can think of plenty more examples than a baby controlling a mother. Many intelligent people die at war, or from its consequences, and not all leaders are top percentile IQ or whatever metric you care to use.

    2) The axiom ‘AXIOM We (humans) should help humanity flourish.‘

    There are so many assumptions by the author in terms of what ‘humanity flourishing’ is supposed to mean, and expectation that we tacitly agree with whatever that implicit definition is.

    Many many people I know dislike computers for varied reasons : for the way they remove agency and disempower non tech-savvy individuals but empower out-of-reach corporations that are empowered by turning them into ever more obtuse, filled-with-telemetry black boxes, for the fact that it forces them to be static, for the soul-sucking jobs, for the pain of doing paperwork that used to be done by companies/the government, for their price, for…

    The list goes on, and I’ve only mentioned some practical reasons. Didn’t even mention other philosophical, moral, and ethical reasons (eg, extractivism of rare metals).

    Many of these people would probably ‘flourish’ in many ways they aren’t flourishing at the moment, if computers were removed from many places they have forced themselves upon them over the past 30-60 years, despite whatever downsides that might have.

    3) Similarly, this idea of a single, like-minded ‘humanity’ that apparently ‘flourishes’ comes from a place that is so out of touch with how many many many MANY humans feel.

    Many humans will flourish when they can eat food that isn’t so processed that it degrades their health early, have lodgings big enough to have children growing in, enough earnings that they can actually know they’ll be able to live in such a place without constant anxiety of failing to make ends meet, go home without constant backpain, do something invigorating at work and/or have enough free time to do more than the chores.

    None of that has to do with “whether people advancing technology would consider it a catastrophe if human-crafted non-human intelligences outcompeted and replaced humans”, or how AI will be used.

    4) Very very few of these people will have the time to learn math at an interesting level beyond their needs, the drive to learn math instead of whatever other pastimes they have during their short moments of respite, etc.

    You’re possibly better off showcasing, say, the practical usage of statistics to tell when a politician is lying through their teeth about some figure, (which is how a math teacher friend keeps his students interested), how to avoid scams or poor everyday investments, if you want to hope to be beneficial to more humans.

    5) Incidentally, as somebody that isn’t completely ignorant in bachelor-level mathematics, it makes a HUGE difference to me that I can talk to people that understand something, and feel like I could spend at least a few weeks/months/years to get a grasp of what they understand, if it is important enough.

    That’s not something I trust is possible when going through AI.

    Unknown's avatar

    “We (humans) should help humanity flourish“. Well, of course, that’s not a bad goal per se, I think it’s one that has been tried by charismatic religious figures, writers, poets, etc for a long time. Not an easy sell for people en masse. Coordination is hard, even for mutually beneficial outcomes. I think professor Loh’s tremendous intellect and enthusiasm and optimism (not all unrelated!) blind him to how far off the average Joe is from his imagined ideal state. People act in self-interested ways, self-sabotaging ways, self-destructive ways, etc. Theoreticians are not going to save us from this complicated set of problems facing both humanity and mathematics. Though I would be grateful to be proven wrong!

    Unknown's avatar

    Imagine this scenario: A mathematician submits a paper addressing a major mathematical problem to a journal, and the manuscript is sent to several reviewers for evaluation.

    One of the reviewers, without considering the potential consequences, uploads the manuscript to an AI system to obtain a summary of the paper. A few months later, the paper is rejected by the journal.

    Meanwhile, the AI system has used the manuscript as part of its training data. Somewhere else in the world, another person uses the same AI system and asks it to solve the mathematical problem.

    Because the model has been trained on the mathematician’s manuscript, it is able to generate a proof that draws on the ideas contained in the unpublished work. The person then publishes the proof, unaware that the underlying ideas originated from the mathematician’s manuscript.

    This scenario raises a difficult question: What happens when unpublished mathematical work, submitted in confidence for peer review, enters an AI system and later re-emerges as part of someone else’s published solution?

    Unknown's avatar

    REAL TRUTH: Humans get dumber as fast as AI gets smarter.


    Leave a comment Cancel reply

    For commenters

    To enter in LaTeX in comments, use $latex <Your LaTeX code>$ (without the < and > signs, of course; in fact, these signs should be avoided as they can cause formatting errors). Also, backslashes \ need to be doubled as \\.  See the about page for details and for other commenting policy.

    Blog at WordPress.com.Ben Eastaugh and Chris Sternal-Johnson.

    Subscribe to feed.

    Discussion (0)

    Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

    Sign in →

    No comments yet. Sign in and be the first to say something.

    More from Hacker News — AI on Front Page