labs/Lab-4/Lab-4-Solution/01_SMS_Dataset_Inspection_Solution.ipynb

Module 4 — Lab 1: Inspecting the SMS Spam Dataset

Topics: text data, duplicates, conflicting labels, class distribution

Solution version: This notebook contains the complete solution. Run the cells in order and compare the results with your Lab notebook.

Learning objectives

After completing this lab, you will be able to:

• load the original UCI tab-separated text file;
• audit missing values, exact duplicates, and conflicting labels;
• create a binary target without modifying the message text;
• describe class imbalance and message-length differences;
• explain why aggressive cleaning may remove useful security signals.

In [ ]:

from pathlib import Path
import pandas as pd

DATA_PATH = Path("data/SMSSpamCollection")

if not DATA_PATH.exists():
    raise FileNotFoundError(
        "Missing data/SMSSpamCollection. Follow data/README.md to download "
        "and extract the official UCI SMS Spam Collection."
    )

1. Load the original collection

The source file has no header. Each row contains a label, one tab character, and the message text.

In [ ]:

raw_df = pd.read_csv(
    DATA_PATH,
    sep="\t",
    header=None,
    names=["label", "message"],
    encoding="utf-8",
)

print("Raw shape:", raw_df.shape)
print(raw_df["label"].value_counts(dropna=False))
raw_df.head(10)

2. Audit the records

Exact duplicates can cross a random train-test split and make evaluation too optimistic. A more serious problem occurs when identical text has different labels. Audit both conditions before modeling.

In [ ]:

audited_df = raw_df.copy()
audited_df["label"] = audited_df["label"].astype("string").str.strip().str.lower()
audited_df["message"] = audited_df["message"].astype("string").str.strip()
audited_df = audited_df.dropna(subset=["label", "message"])
audited_df = audited_df[audited_df["message"].ne("")]
audited_df = audited_df[audited_df["label"].isin(["ham", "spam"])]

exact_duplicate_count = int(audited_df.duplicated(subset=["label", "message"]).sum())
label_counts_per_message = audited_df.groupby("message")["label"].nunique()
conflicting_messages = label_counts_per_message[label_counts_per_message.gt(1)].index

print("Exact duplicate rows:", exact_duplicate_count)
print("Conflicting message texts:", len(conflicting_messages))

3. Build the clean classroom table

Remove ambiguous messages before exact duplicates, then map ham → 0 and spam → 1. Keep the original text intact.

In [ ]:

clean_df = audited_df[~audited_df["message"].isin(conflicting_messages)].copy()
clean_df = clean_df.drop_duplicates(subset=["label", "message"]).reset_index(drop=True)
clean_df["target"] = clean_df["label"].map({"ham": 0, "spam": 1}).astype(int)

print("Clean shape:", clean_df.shape)
print(clean_df["target"].value_counts().sort_index().rename(index={0: "ham", 1: "spam"}))

4. Describe the classes and message lengths

Length is used only for exploration. The later models will learn from the message text itself.

In [ ]:

clean_df["character_count"] = clean_df["message"].str.len()
clean_df["word_count"] = clean_df["message"].str.split().str.len()

length_summary = clean_df.groupby("label").agg(
    message_count=("message", "size"),
    median_characters=("character_count", "median"),
    median_words=("word_count", "median"),
)
class_distribution = clean_df["label"].value_counts(normalize=True)

print(length_summary)
print("Class proportions:")
print(class_distribution)

Security note: do not over-clean

URLs, phone numbers, currency symbols, punctuation, unusual capitalization, and deliberate misspellings may contain spam evidence. Begin with the vectorizer defaults. Add normalization only when an experiment demonstrates that it improves validation performance without hiding operationally important signals.

Self-check

In [ ]:

assert list(raw_df.columns) == ["label", "message"]
assert clean_df["message"].notna().all()
assert clean_df["message"].ne("").all()
assert set(clean_df["label"].unique()) == {"ham", "spam"}
assert set(clean_df["target"].unique()) == {0, 1}
assert not clean_df.duplicated(subset=["label", "message"]).any()
assert not clean_df["message"].isin(conflicting_messages).any()
assert class_distribution.loc["ham"] > class_distribution.loc["spam"]
assert {"character_count", "word_count"}.issubset(clean_df.columns)
print("✓ Lab 1 completed")

Suggested answers

1. Why should exact duplicates be removed before the train-test split?
   A duplicate can otherwise appear in both partitions, allowing the model to receive nearly identical training and test evidence and inflating the reported score.
2. Why might removing URLs and currency symbols reduce detection quality?
   Those tokens often carry direct spam intent. Removing them can discard discriminative evidence rather than remove harmless noise.
3. Why is accuracy alone risky for this dataset?
   The ham class is much larger, so a model can obtain high accuracy while missing many spam messages. Spam precision, recall, F1, and the confusion matrix are also required.

In [ ]: