But the currently fashionable practice of pointing a powerful AI tool at the task of answering a problem X, unguided by any human expert in the field X resides in, has created an unprecedented divergence between the production of answers, and the production of insight, to the point where the two questions have become _negatively correlated_: further optimization in the direction of generating answers for X can now _decrease_ the amount of insight one could have gained from the study of X, thanks to factors discussed in previous posts, such as contamination of reasoning with the solution, degradation of the signal-to-noise ratio, or in reduction of incentives to analyze the problem further.
A priori, there is no reason that AI use _necessarily_ has to perform this tradeoff. For instance, the use of AI for literature search is already well established as a basic way to gain initial insight into a problem, or to connect any newly discovered technique to the literature. But the current incentive to race for priority for solutions above all other concerns works against such more deliberate use cases. (2/3)
Terence Tao
npub1gv…2sduy
2026-09-03 18:53:06 UTC
in reply to nevent1q…c3sk
Author Public Key
npub1gvea966levf836g93xsdnkde8mmz58qyznf9c82jg02hq35f46lsn2sduyPublished at
2026-09-03 18:53:06 UTCEvent JSON
{
"id": "c12148aaff7817684f1262f798d5e171a491e95960da7b0047d2da6051d6f56d",
"pubkey": "4333d2eb5fcb1278e90589a0d9d9b93ef62a1c0414d25c1d5243d5704689aebf",
"created_at": 1788461586,
"kind": 1,
"tags": [
[
"proxy",
"https://mathstodon.xyz/@tao/117208618508728654",
"web"
],
[
"e",
"105db437f20d8ca6cea55c79284543e1bcea96ad1ff96d731187ea294e83908b",
"",
"root",
"4333d2eb5fcb1278e90589a0d9d9b93ef62a1c0414d25c1d5243d5704689aebf"
],
[
"p",
"4333d2eb5fcb1278e90589a0d9d9b93ef62a1c0414d25c1d5243d5704689aebf"
],
[
"proxy",
"https://mathstodon.xyz/users/tao/statuses/117208618508728654",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mathstodon.xyz/users/tao/statuses/117208618508728654",
"pink.momostr"
],
[
"-"
]
],
"content": "But the currently fashionable practice of pointing a powerful AI tool at the task of answering a problem X, unguided by any human expert in the field X resides in, has created an unprecedented divergence between the production of answers, and the production of insight, to the point where the two questions have become _negatively correlated_: further optimization in the direction of generating answers for X can now _decrease_ the amount of insight one could have gained from the study of X, thanks to factors discussed in previous posts, such as contamination of reasoning with the solution, degradation of the signal-to-noise ratio, or in reduction of incentives to analyze the problem further. \n\nA priori, there is no reason that AI use _necessarily_ has to perform this tradeoff. For instance, the use of AI for literature search is already well established as a basic way to gain initial insight into a problem, or to connect any newly discovered technique to the literature. But the current incentive to race for priority for solutions above all other concerns works against such more deliberate use cases. (2/3)",
"sig": "958b88482a0dfa97ff67315959dc8521b452a6bda67fa8428b761738b87a4aa7d37cf6b9c0a472897f165a3c1333f20eaa83e71470fd509b7bce2265052e38cc"
}