A machine learning pipeline that processes a social media dataset and compares multiple models — K-NN, SVM, and CNN — for classifying posts as positive, negative, or neutral.
This project was built for a COE292 (AI) course and focuses on building and comparing three different machine learning classifiers for sentiment analysis on social media data. The dataset contains thousands of social media posts labeled as positive, negative, or neutral.
Rather than picking one model and optimizing it, the project intentionally implements three fundamentally different approaches — a distance-based model (K-NN), a margin-based model (SVM), and a deep learning model (CNN) — to understand when each performs better and why.
The primary goal was comparative: which model performs best on this specific dataset, and under what conditions? The secondary goal was to build a proper ML pipeline from scratch — preprocessing, feature extraction, training, evaluation — rather than just calling a pre-trained model.
Social media text is noisy — it contains hashtags, emoji, slang, misspellings, and mixed languages. The preprocessing pipeline handles cleaning, tokenization, stopword removal, and vectorization (TF-IDF for K-NN/SVM, embedding layers for CNN).
K-NN, SVM, and CNN are all trained on the same dataset split and evaluated on the same held-out test set. Accuracy, precision, recall, and F1 scores are reported per class and overall, enabling a fair apples-to-apples comparison.
Confusion matrices and per-class performance charts make it clear where each model struggles. The CNN, for example, handles ambiguous "neutral" posts better than K-NN but requires significantly more compute to train.
Class imbalance was the biggest issue — the dataset had far more neutral posts than positive or negative ones. Naive training produced a model that predicted "neutral" for almost everything and still achieved decent accuracy. Resampling and class-weighted training were needed to get meaningful results on the minority classes.
The CNN also required significant hyperparameter tuning — learning rate, embedding size, kernel size, dropout rate — which was time-consuming without GPU access.
Accuracy is a misleading metric on imbalanced datasets. This project made that lesson visceral — a model that ignores the minority class entirely can look great on accuracy while being completely useless in practice. F1 score and confusion matrices tell the real story.
The comparison also showed that "better" depends entirely on context. SVM consistently outperformed K-NN with a fraction of the inference time. The CNN beat both on F1 but took 10x longer to train. Choosing a model is always a trade-off, not a universal ranking.