Depression From Text: Why 3,067 Studies Came Down to 11

The headline number from this meta-analysis looks reassuring: across the models it could pool, machine learning detects depression from written text with a correlation of r = 0.605. The number that should actually shape your judgement sits a few lines down — a prediction interval of 0.140 to 0.851. That is the range a careful reader should expect the next study to land in: somewhere between barely above a coin toss and very nearly perfect. A strong average sitting over a spread that wide is the whole story here.

What makes this synthesis worth reading is not the pooled figure but the door policy that produced it. Most published work on detecting depression from text trains its models on weak labels — keyword matching, membership of an online forum, a status someone asserted about themselves. A model fitted to those learns the texture of how distressed people write on the internet, not the clinical syndrome. The authors threw all of that out and admitted a study only if the depression label it trained against was either a clinician's diagnosis or the PHQ-9 (Patient Health Questionnaire-9), a validated severity scale. Other common instruments, such as the BDI-II and CES-D, were excluded on purpose, to keep the labels comparable across studies.

The screening funnel

That single rule does most of the work. The team began with 3,067 records, deduplicated to 1,947, read 451 in full, and kept 11 — together contributing 15 independent models. By their own count the strict label criterion excluded 57.5 percent of otherwise eligible studies. The ratio of 3,067 to 11 is, in plain terms, the most honest figure in the paper: of more than three thousand publications on this topic, eleven cleared a basic bar for what they called depression in the first place.

How they did it

This is a systematic review and meta-analysis, pre-registered on PROSPERO (CRD420251056902) and reported to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 standard. The authors searched four databases — PubMed, Scopus, IEEE Xplore and Web of Science — from January 2014 onward for models trained on participant-generated text: essays, diaries, social-media posts, messages, chat logs and clinical transcripts. Effect sizes were pooled as correlations in a random-effects model with the Hartung-Knapp-Sidik-Jonkman correction, with subgroup and meta-regression analyses to probe the variation. It is a synthesis of existing development studies, not new prospective data, so it inherits whatever weaknesses sit inside its eleven inputs.

What the evidence supports

The pooled effect is genuinely large: r = 0.605 (95 percent confidence interval 0.498 to 0.693). Detecting depression from properly labelled text is, in aggregate, not a marginal task. The subgroup results then read like a methods textbook confirming itself. Embedding-based text representations beat traditional hand-built features (r = 0.741 versus 0.514); deep architectures beat shallow ones (0.731 versus 0.486); and models trained on a clinician's diagnosis edged out those trained on the self-report PHQ-9 (0.688 versus 0.500). None of this is surprising, and the lack of surprise is the point — the data behave as theory predicts they should.

The quietly most useful result concerns reporting, not architecture. Studies that scored higher on TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) also performed better, and the association held up in meta-regression (β = 0.085, P < 0.001). This does not mean transparency causes accuracy. It means the studies that documented their methods properly were also the ones that worked — reason enough to treat full TRIPOD reporting as a precondition for taking a model seriously rather than a courtesy.

What it does not support

Return to the heterogeneity, because it is the load-bearing caveat. With I² at 85.9 percent, most of the variation between models is real rather than noise, and that is precisely why the prediction interval runs from 0.140 to 0.851. The pooled r describes the field on average; the interval describes what you can expect from any one tool. This synthesis cannot tell you in advance whether the model in front of you sits near the floor or near the ceiling. A high average over a very wide spread is not a licence to trust a single instrument.

Two further limits the authors name themselves. They graded the overall certainty of the evidence as moderate under GRADE, pulled down by exactly that heterogeneity. And they excluded the newest methods by design — large language models and prompt-based approaches fell outside the window — so this is a clean read of a maturing literature, not a verdict on the systems moving fastest right now. To which add the limit every diagnostic-accuracy reader already knows: each effect here is discrimination measured in a development sample. Not one of the eleven studies reports the positive predictive value at the prevalence of an unscreened population, where most of the people a model flags would not, in fact, be depressed.

Why it matters here

Digital screening for depression keeps surfacing in European health-system debates, and the easy mistake is to read three thousand publications as a sign the field is mature. This paper is the corrective: eleven of those publications met a basic standard for the label, and even those scatter from near-random to near-perfect. What it leaves you with is a short list of questions for anyone offering such a tool. What were the training labels — a clinical diagnosis, the PHQ-9, or a proxy? How large and how representative was the development sample? Has it been validated externally, in a population and a language like the one you serve, at the prevalence you will actually meet? The honest answer to the last two is usually no — and that gap, not the pooled correlation, is what separates a strong meta-analytic average from a system you would let near a patient.

Source: Zhang S, Zhang C, Zhang J. Text-Based Depression Estimation Using Machine Learning With Standard Labels: Systematic Review and Meta-Analysis. Journal of Medical Internet Research 2026;28:e82686. A pre-registered systematic review and meta-analysis of eleven development studies; the authors rate the certainty of evidence as moderate under GRADE because of substantial heterogeneity, and exclude large language models by design. Identified via PubMed. The authors declare no conflicts of interest; the work was funded by a Chinese Ministry of Education industry-collaborative education programme, a Guangdong provincial key laboratory, and Southern University of Science and Technology startup funds.