Automation

Automating Data Collection and Reporting with Python and AI

Published Nov 9, 2026

Collect a bounded CSV source, compute totals in Python, and use AI to narrate only checked results.

Overview

For learners who want to complete this practical exercise with inspectable sample data and explicit checks.

Separate numeric computation from AI narrative.
Check source schema, size, and timeouts.
Produce a repeatable Markdown report.

You will build: report.md with Design=3, Engineering=3, 6 total hours, and a grounded narrative.

Prerequisites

Python 3.10+, an editor, and optionally an authorized HTTPS CSV URL.

Default mode uses an offline fixture. Optional --url reads an authorized CSV without scraping or credentials. AI chat for narration is optional and may have fees/limits.

Step-by-step tutorial

Conceptual practice workflow; not measured results.

Step 1. Define the schema and sample

The schema is team and hours, with each row containing non-negative integer training hours. Start with the fixture so the answer is known: Design is 2+1=3, Engineering is 3, and the total is 6. Do not delegate arithmetic to a model when code can compute it exactly.

Step 2. Write a pipeline with source checks

Save this as report.py. Pass --url only for a source you control or are authorized to read. HTTPS checks and byte limits bound this exercise but do not make it safe as a public URL-fetching service.

PYTHON

import argparse
import csv
import io
from pathlib import Path
from urllib.parse import urlsplit
from urllib.request import Request, urlopen

SAMPLE = "team,hours\nDesign,2\nEngineering,3\nDesign,1\n"

def read_source(url=None):
    if url is None:
        return SAMPLE
    if urlsplit(url).scheme != "https":
        raise ValueError("use an approved HTTPS CSV source")
    request = Request(url, headers={"User-Agent": "TrainingReport/1.0"})
    with urlopen(request, timeout=15) as response:
        data = response.read(100001)
    if len(data) > 100000:
        raise ValueError("source exceeds 100 KB")
    return data.decode("utf-8-sig")

def aggregate(text):
    rows = csv.DictReader(io.StringIO(text))
    if not {"team", "hours"} <= set(rows.fieldnames or []):
        raise ValueError("team and hours columns required")
    totals = {}
    for row in rows:
        team = row["team"].strip()
        hours = int(row["hours"])
        if not team or hours < 0:
            raise ValueError("invalid team or hours")
        totals[team] = totals.get(team, 0) + hours
    return totals

def write_report(totals):
    lines = ["# Training hours", "", "| Team | Hours |", "| --- | ---: |"]
    for team, hours in sorted(totals.items()):
        safe = team.replace("|", "\\|").replace("\n", " ")
        lines.append(f"| {safe} | {hours} |")
    lines += ["", f"Total hours: {sum(totals.values())}"]
    Path("report.md").write_text("\n".join(lines) + "\n", encoding="utf-8")

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--url")
    args = parser.parse_args()
    totals = aggregate(read_source(args.url))
    write_report(totals)
    print("Total hours:", sum(totals.values()))

Step 3. Run the fixture and check the numbers

SHELL

python3 report.py

Expected result

Total hours: 6

Open report.md and verify the two group totals of 3 and 3. Try a missing column or hours=-1 in SAMPLE; processing must fail before a new report is written. In a real system, label archived outputs with their actual source/access time rather than silently treating an old report as fresh.

Step 4. Try an authorized CSV source

Host the fixture at an authorized HTTPS location and run python3 report.py --url YOUR_APPROVED_HTTPS_CSV_URL after replacing the placeholder with its real URL. Compare with the fixture. Use no fabricated sample URL or query-string key, and do not hammer a failing source.

Step 5. Add an AI narrative and verify it

PROMPT · INSTRUCTIONS

Checked data: Design=3 hours, Engineering=3 hours, total=6 hours. This is a training fixture, not real business data. Write three bullets describing it and note that a single batch establishes no trend, cause, or effectiveness. Do not change any number.

Place the checked narrative below the table and keep it separate from computed data. Never overwrite numeric values with model text. Before scheduling, add logs, bounded retries, and a way to identify previously processed batches.

Verification

The fixture yields 3/3 and a total of 6, and invalid data is rejected.
The narrative preserves every number and infers no trend.

Troubleshooting

The endpoint returns HTML instead of CSV

Cause: The URL is a web page or login screen.

Fix: Use the actual CSV URL and verify team,hours headers and access.

Duplicate rows distort the report

Cause: The current schema has no event ID for deduplication.

Fix: Do not guess which rows are duplicates; add source IDs and a tested deduplication rule before real use.

Exercises & extensions

Add tests for zero data rows and a team containing |.
Add event IDs and valid/rejected-row logs for your own source.

This script implements no scraping, automatic schedule, or continuous-update guarantee. Untrusted URL input requires separate SSRF, redirect, and network-permission design.