{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1727969b",
   "metadata": {},
   "source": [
    "# Module 2 — Lab 6: Capstone — Investigating Suspicious Logins\n",
    "\n",
    " \n",
    "**Topics:** Pandas, NumPy, Matplotlib, export, analyst summary\n",
    "\n",
    "\n",
    "> **Solution version:** This notebook contains the complete solution. Run the cells in order and compare the results with your Lab notebook.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "467cdb20",
   "metadata": {},
   "source": [
    "## Mission\n",
    "\n",
    "As a junior security analyst, you must build a complete workflow:\n",
    "\n",
    "1. load and clean authentication logs;\n",
    "2. identify IP addresses with repeated failed logins;\n",
    "3. identify off-hours activity;\n",
    "4. visualize the results;\n",
    "5. export the data and write a short conclusion.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "525cb519",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "DATA_PATH = Path(\"data/auth_log.csv\")\n",
    "OUTPUT_DIR = Path(\"output\")\n",
    "OUTPUT_DIR.mkdir(exist_ok=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d94da21b",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "## 1. Write a reusable cleaning function\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c33bbbcb",
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "outputs": [],
   "source": [
    "def clean_auth_log(df):\n",
    "    result = df.copy().drop_duplicates()\n",
    "    result[\"timestamp\"] = pd.to_datetime(\n",
    "        result[\"timestamp\"], errors=\"coerce\", utc=True\n",
    "    )\n",
    "    result = result.dropna(subset=[\"timestamp\", \"source_ip\", \"status\"])\n",
    "    result[\"source_ip\"] = result[\"source_ip\"].str.strip()\n",
    "    result[\"username\"] = result[\"username\"].fillna(\"unknown\").str.strip().str.lower()\n",
    "    result[\"status\"] = result[\"status\"].str.strip().str.upper()\n",
    "    result = result[result[\"status\"].isin([\"SUCCESS\", \"FAILED\"])]\n",
    "    result[\"hour\"] = result[\"timestamp\"].dt.hour\n",
    "    result[\"is_off_hours\"] = (result[\"hour\"] < 6) | (result[\"hour\"] >= 22)\n",
    "    return result.sort_values(\"timestamp\").reset_index(drop=True)\n",
    "\n",
    "\n",
    "raw_df = pd.read_csv(DATA_PATH)\n",
    "clean_df = clean_auth_log(raw_df)\n",
    "print(f\"Raw rows: {len(raw_df)} | Clean rows: {len(clean_df)}\")\n",
    "clean_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5205cafa",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "## 2. Analyze failures and rank IP addresses\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "085dd483",
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "outputs": [],
   "source": [
    "failed_df = clean_df[clean_df[\"status\"] == \"FAILED\"].copy()\n",
    "failed_by_ip = (\n",
    "    failed_df.groupby(\"source_ip\")\n",
    "    .size()\n",
    "    .sort_values(ascending=False)\n",
    "    .rename(\"failed_attempts\")\n",
    ")\n",
    "\n",
    "THRESHOLD = 3\n",
    "suspicious_ips = failed_by_ip[failed_by_ip >= THRESHOLD]\n",
    "\n",
    "print(\"Failed attempts by IP:\")\n",
    "print(failed_by_ip)\n",
    "print(\"\\nSuspicious IPs:\")\n",
    "print(suspicious_ips)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5967176b",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "## 3. Calculate additional investigation metrics\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7cab677c",
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "outputs": [],
   "source": [
    "status_counts = clean_df[\"status\"].value_counts().reindex([\"SUCCESS\", \"FAILED\"], fill_value=0)\n",
    "off_hours_failed = failed_df[failed_df[\"is_off_hours\"]]\n",
    "top_ip = failed_by_ip.index[0]\n",
    "top_ip_count = int(failed_by_ip.iloc[0])\n",
    "failure_rate = np.round((status_counts[\"FAILED\"] / status_counts.sum()) * 100, 2)\n",
    "\n",
    "print(f\"Top suspicious IP: {top_ip} ({top_ip_count} failures)\")\n",
    "print(f\"Failure rate: {failure_rate}%\")\n",
    "print(f\"Off-hours failed attempts: {len(off_hours_failed)}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c472f43a",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "## 4. Visualize the findings\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4816f960",
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(11, 4))\n",
    "\n",
    "status_counts.plot(\n",
    "    kind=\"bar\",\n",
    "    ax=axes[0],\n",
    "    color=[\"#2a9d8f\", \"#e76f51\"],\n",
    "    title=\"Authentication status\",\n",
    ")\n",
    "axes[0].set_xlabel(\"Status\")\n",
    "axes[0].set_ylabel(\"Number of attempts\")\n",
    "axes[0].tick_params(axis=\"x\", rotation=0)\n",
    "\n",
    "failed_by_ip.head(5).sort_values().plot(\n",
    "    kind=\"barh\",\n",
    "    ax=axes[1],\n",
    "    color=\"#e76f51\",\n",
    "    title=\"Top IPs by failed logins\",\n",
    ")\n",
    "axes[1].set_xlabel(\"Failed attempts\")\n",
    "axes[1].set_ylabel(\"Source IP\")\n",
    "\n",
    "fig.suptitle(\"Authentication Log Investigation\")\n",
    "fig.tight_layout()\n",
    "fig.savefig(OUTPUT_DIR / \"auth_log_analysis.png\", dpi=150, bbox_inches=\"tight\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "933a5898",
   "metadata": {
    "tags": [
     "exercise"
    ]
   },
   "source": [
    "## 5. Export results and write a short report\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc68ed13",
   "metadata": {
    "tags": [
     "solution"
    ]
   },
   "outputs": [],
   "source": [
    "clean_df.to_csv(OUTPUT_DIR / \"capstone_clean_auth_log.csv\", index=False)\n",
    "suspicious_ips.to_csv(\n",
    "    OUTPUT_DIR / \"capstone_suspicious_ips.csv\",\n",
    "    header=True,\n",
    ")\n",
    "\n",
    "summary = f'''SECURITY ANALYSIS SUMMARY\n",
    "=========================\n",
    "Clean events: {len(clean_df)}\n",
    "Failed events: {int(status_counts['FAILED'])}\n",
    "Failure rate: {failure_rate}%\n",
    "Top suspicious IP: {top_ip} ({top_ip_count} failed attempts)\n",
    "Off-hours failed attempts: {len(off_hours_failed)}\n",
    "Decision threshold: {THRESHOLD} failed attempts\n",
    "\n",
    "Recommended action:\n",
    "- Review events from {top_ip}.\n",
    "- Correlate off-hours failures with endpoint and identity logs.\n",
    "- Validate whether the account owners recognize these attempts.\n",
    "'''\n",
    "\n",
    "(OUTPUT_DIR / \"capstone_summary.txt\").write_text(summary, encoding=\"utf-8\")\n",
    "print(summary)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "023ea53a",
   "metadata": {},
   "source": [
    "## Self-check\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "560fa29e",
   "metadata": {},
   "outputs": [],
   "source": [
    "assert len(raw_df) > len(clean_df)\n",
    "assert clean_df.duplicated().sum() == 0\n",
    "assert failed_by_ip.index[0] == \"198.51.100.14\"\n",
    "assert top_ip_count >= THRESHOLD\n",
    "assert len(off_hours_failed) >= 1\n",
    "assert (OUTPUT_DIR / \"auth_log_analysis.png\").exists()\n",
    "assert (OUTPUT_DIR / \"capstone_summary.txt\").exists()\n",
    "print(\"✓ Capstone lab completed\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dedf0889",
   "metadata": {},
   "source": [
    "## Report questions\n",
    "\n",
    "1. Which IP address should be investigated first, and what evidence supports that decision?\n",
    "2. In what situations could a threshold of three failed attempts produce a false positive?\n",
    "3. Which additional log sources should be reviewed before deciding to block an IP address?\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}