Moltbook Human-Like Outlier Detection Pipeline

Today was a fun one!  I again used the HuggingFace Daily Briefer to learn about a new dataset that's available.  The dataset is from MotBook and the chats that have been happening.  One of the interesting stories about MotBook is that its not entirely clear which posts are AI agents and which are humans.  Sounds like an excellent data science opportunity.

This is an analysis pipline that statistically identify which posts that are linguistically atypical — and therefore potentially more "human-like" — compared to the bulk of agent-generated text.

One important caveat:

"Atypical" does not mean "human-authored."

This pipeline flags posts whose linguistic properties are statistical outliers. These posts may reflect genuine human writing, but they could also be unusual agent outputs, edge-case prompts, or noise. The correct interpretation is: "this text is statistically inconsistent with typical agent output" — nothing more.

I've gone through and built the full pipeline, and generated research paper which I hope to get on arXiv shortly.  Though I'll need to do a bit more work to really get it where I want.

Here is the full abstract:

We present a reproducible pipeline for identifying linguistically atypical posts in the
Moltbook corpus, a simulated social-media platform populated by AI agents. Our approach
extracts 19 numeric features spanning stylometry, lexical discourse markers, language-model
perplexity, and sentence-embedding geometry, then applies an ensemble of three unsuper-
vised outlier detectors—Isolation Forest, Local Outlier Factor, and Robust Mahalanobis
Distance—requiring agreement from at least two methods before flagging a post. Applied
to 43,234 posts across nine topic categories, the ensemble flags 1,381 posts (3.19%) as statis-
tically atypical. Atypicality is unevenly distributed across categories and communities, with
certain topic areas and niche communities exhibiting substantially higher concentrations
of unusual writing patterns. We emphasize that atypical does not imply human-authored ;
rather, these posts deviate from the dominant statistical patterns in the corpus.

View the pipeline on Github