Slop Watch

In an effort to understand AI-generated content, I developed a web app that provides some meaningful statistics about content in order to determine how much it's AI-generated. Call it Slop Watch This simple interface provides lots of information to understand AI writing.  But beware, I've only tested this sparingly.

Github

How It Works

The Binoculars method (the backbone) runs input text through two related LLMs (Mistral-7B base and Instruct) and computes:

• Perplexity from the observer model (how surprised it is by the text)
• Cross-perplexity between the performer and observer (how much their predictions agree)
• Score = perplexity / cross-perplexity — AI text scores below a threshold, human text above

The web UI provides four visualizations:

1. Score gauge — overall verdict with confidence
2. Per-sentence heatmap — color-coded sentences (green=human, red=AI)
3. Perplexity distribution — per-token perplexity chart (AI=flat, human=spiky)
4. Burstiness plot — sentence length variation (AI=uniform, human=varied)