Sentiment & Content Insights with Small LLMs

The bridge from classical ML to modern NLP pipelines: TF-IDF + logistic regression as an honest baseline, a conceptual comparison with zero-shot classification via a small Hugging Face model – and the question of when the LLM leap really pays off.

1. Why Sentiment in Marketing Is a Hard Problem

Social media texts violate every assumption of classical NLP methods:

• Short sequences (often <50 tokens) – little context per example.
• Sarcasm and irony – surface lexica are fooled.
• Emojis, hashtags, slang – out-of-vocabulary words dominate.
• Domain drift – today's memes are obsolete in three months.

Nonetheless a compact, transparent TF-IDF baseline is often surprisingly competitive – and it is the yardstick against which every more complex model must be measured.

2. TF-IDF + Logistic Regression: The Honest Baseline

What TF-IDF does

• Term frequency (TF): how often does a word appear in the document?
• Inverse document frequency (IDF): how rare is it in the corpus? Common words ("the", "and") are down-weighted.
• Result: a sparse vector that quantifies the importance of a word for a document relative to the corpus.

❌ Naive: bag-of-words without weighting

Pure term counting without IDF lets frequent, semantically empty words dominate and makes the model needlessly fragile against stop words.

✅ Functional: TF-IDF + LogReg in one pipeline

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# Small, representative example corpus (EN)
texts = [
    "Great product, clear recommendation!",
    "Absolutely thrilled with the quality",
    "Awesome service, fast delivery",
    "Finally something that delivers what it promises",
    "Super happy with the purchase",
    "Terrible, never again.",
    "Totally disappointing and expensive",
    "Poor quality, broke after 2 days",
    "Shipment never arrived, money gone",
    "I hate this product",
    "It was okay, nothing more.",
    "Mediocre, could have been better",
] * 5  # duplicate to 60 examples so train/test split is meaningful

y = ([1] * 5 + [0] * 5 + [0] * 2) * 5

X_tr, X_te, y_tr, y_te = train_test_split(texts, y, test_size=0.3, random_state=0)

clf = make_pipeline(
    TfidfVectorizer(ngram_range=(1, 2), min_df=1),
    LogisticRegression(max_iter=1000),
).fit(X_tr, y_tr)

print(f"Test accuracy: {clf.score(X_te, y_te):.3f}")
print()

# Predictions on unseen examples
new = ["surprisingly good", "sadly very disappointing", "meh"]
for t, pred, p in zip(new, clf.predict(new), clf.predict_proba(new)[:, 1]):
    print(f"  '{t}' -> label={pred}  P(positive)={p:.2f}")

# Interpretability: strongest features
tfidf = clf.named_steps["tfidfvectorizer"]
logreg = clf.named_steps["logisticregression"]
features = tfidf.get_feature_names_out()
coefs = logreg.coef_[0]
order = coefs.argsort()
print("\nTop-5 negative features:")
for i in order[:5]:
    print(f"  {features[i]:25s}  {coefs[i]:+.2f}")
print("Top-5 positive features:")
for i in order[-5:][::-1]:
    print(f"  {features[i]:25s}  {coefs[i]:+.2f}")

Interpretability is not a bonus here, it is a main argument: marketing stakeholders can inspect the learned words directly – a neural model swallows this sanity check.

3. Failure Modes of the Baseline

Despite its strengths, TF-IDF + LogReg has clear blind spots:

This is where the LLM leap comes in.

4. Zero-Shot with Pre-Trained Models (conceptual comparison)

Pre-trained language models carry world knowledge – negation, context and sarcasm are implicitly learned during training. The conceptual code block:

# CONCEPT (outside the Pyodide playground, since transformers won't load):
from transformers import pipeline

clf = pipeline(
    "sentiment-analysis",
    model="distilbert-base-uncased-finetuned-sst-2-english",
)
clf(["not bad", "Yeah, just great. *eye roll*"])
# [{'label': 'POSITIVE', 'score': 0.78}, {'label': 'NEGATIVE', 'score': 0.82}]

5. Discussion: The Discipline of the Baseline

"If your fancy model can't beat a simple baseline, it's not the baseline that's broken." – Empirical rule of thumb from ML engineering

In every production sentiment project both paths should be evaluated in parallel. Concretely:

1. Train TF-IDF + LogReg in 20 minutes.
2. Apply a pre-trained zero-shot model in 20 minutes.
3. Compare both on the same holdout.
4. If the LLM is only 1–2 percentage points better: the baseline wins via maintainability, latency and privacy.
5. If the LLM systematically handles negation and sarcasm more correctly and that is business-relevant: fine-tuning is worth it.

6. Wrap-up Quiz

7. Further Reading

• This series concludes with article 05. Cross references: recommendation-systems and large-language-models-llms.
• In refinement: comparison on a real tweet dataset (sentiment140, germeval-2017), fine-tuning example with transformers.Trainer.

Sources

• Generative AI Foundations in Python – chapter on classical vs. neural NLP.
• Machine Learning with PyTorch and Scikit-Learn (Sebastian Raschka et al.) – chapter on text classification.
• Practical Natural Language Processing (Vajjala et al.) – chapter on baselines.