SlopShape: Identifying AI-Generated Commercial Web Content
A research paper that detects AI-generated commercial blog content from structural signatures alone, such as ordering and evidence, rather than wording, reaching 98 macro-F1 and identifying which model wrote it.
This paper asks whether AI-generated text can be identified from structural signatures, how information is presented, in what order, with what evidence, and in what voice, rather than from word-level style. It replicates an earlier method shown to work for AI-generated fiction, applying it instead to commercial content: 2,250 pre-ChatGPT human blog posts from 268 company domains compared against 11,250 AI-written mirrors from five frontier models.
A 214-feature instrument, scored by an LLM and validated against a human gold-annotation session, detects AI-written posts from 187 structural features alone at 98.0 macro-F1 on companies held out of training, essentially unchanged even when every AI post is reworded by its own model afterward. The same structural signal also attributes a post to its source model correctly 79.3 percent of the time, against a 16.7 percent chance rate, while human posts occupy distinct, rarer structural patterns.
The authors release the pipeline, the feature instrument, prompts, code, and aggregate artifacts alongside the paper.