"Hey, I proved the Collatz Conjecture!", proompter says one day before severe bugs disclosed in theorem provers
infosec.exchange/@0xabad1dea/117002106099986943
1 Comments
Comments from other communities
Extremely funny. There’s a similar situation in math where almost everybody is implicitly using one set of axioms (ZF/ZFC) because they’re highly expressive, intuitive, and not known to be inconsistent. There’s a hypothetical scenario where somebody discovers a proof that ZF(C) is inconsistent, rendering the vast majority of mathematics “invalid.” But nobody is really concerned because on a purely intuitive level, you should be able to transfer the vast majority of mathematics to a different formal basis without issue, since it’s all generally done at a different level of abstraction anyway. You’d have to somehow accidentally exploit the inconsistency of ZF(C) to get bit, which is unthinkable. … Unless you’re an LLM, I guess.
Next step we just need an LLM social engineer lean maintainer and introduce a bug they can then exploit.
Human Web Collective