Can AI detect AI writing? And why do people care?

Summary

Substack recently rolled out AI detection, so readers can now see what percentage of writing was generated by AI, assisted by AI, or written by a human.

Eric ran across the feature while prepping show notes for the podcast and went down a rabbit hole testing it. The results were both expected and surprising:

• Detection seemed directionally accurate
• Newer, more powerful models seem to generate somewhat less AI-coded prose
• With human-anchored inputs, even light editing of AI output triggered human detection

But the experiment raised a deeper question: if valuable content is valuable content, why do people care? Most code and writing were bad before, so will AI raise the floor? Eric asks those questions to John, and their conversation covers:

• The spectrum of subjectivity and where AI output is acceptable
• The economics of Substack's two-sided marketplace and how AI affects the perception of value
• Our core need for human connection

They end with timeless advice: use AI to create leverage, but don't outsource the work that makes human communication and connection valuable.

Key takeaways

• Tolerance for AI output falls on a spectrum: There isn't a binary answer for whether AI writing is good or bad. We're more OK with accurate instructions in documentation or a manual generated by AI than we are with thought leadership we pay for.
• Human effort signals value: We are still willing to pay more for a product we know someone has invested time and thought in, and that's the economic incentive Substack is trying to preserve with AI detection.
• We are more sensitive when AI output is the product: AI can write code, but that's a means to an end. In writing, words are the product, and we are much more sensitive to machine-generated output.

Notable mentions and takeaways

• Substack released AI detection, saying "human perspective is the essential ingredient of culture."
• Pangram is the AI detection service that Substack uses.
• Last fall, Paul Graham posted on X that "The reason AI coding works so well is that the source code of the median app was already slop before LLMs."