BLIP-Driven Multi-modal Deep Learning Framework for Temporal Depression Prediction from Social Media Posts

Abstract: Depression remains one of the most widespread mental health conditions worldwide, and social media has become an unusually rich window into how people express emotional and behavioral change. Most existing detection systems however still look at a single channel either the wording of a post or a single accompanying image and largely ignore how a person’s posting behavior shifts over time. This paper addresses that gap with a temporal, multi-modal pipeline that reads depression risk not from one post in isolation but from a sequence of a user’s posts, combining what is written, what is pictured, the emotional tone of both, and how these change across time. Images are converted into descriptive captions using BLIP, which are then encoded alongside raw visual features and a set of behavioral and temporal signals processed through a GRU. When these three streams text, image, and temporal-behavioral are fused, the resulting model reaches 71% classification accuracy. The results support the view that behavioral and temporal cues, even when individually weak predictors, meaningfully sharpen a multi-modal system’s ability to separate depressed from non-depressed posting patterns.

Keywords: Depression Detection; Social Media Analytics; Multi-modal Fusion; Temporal Behaviour Modelling; Image Captioning; Sentiment Analysis; BERT; GRU.