Turn JSONL web logs into a small Python report

A pile of request logs is not yet an operational answer. This project turns a narrow, documented input format into a short report: how many records are valid, how many were rejected, and what share of the valid records contain server-error status codes.

The input format is JSON Lines: one JSON object per line with an integer status field. It is a fixture format for this tutorial, not a promise to parse the default logs from every proxy or framework. Export or transform your own logs before using this reader.

1. Start with an answer you can predict

Save the following as example.jsonl in UTF-8 without a byte-order mark. Four records make it easy to calculate the expected result by hand.

{"status":200,"path":"/"}
{"status":200,"path":"/health"}
{"status":404,"path":"/missing"}
{"status":503,"path":"/api"}

There are two successful responses, one client error, and one server error. Our server-error percentage is therefore one divided by four, multiplied by one hundred: 25 percent. We deliberately keep the 404 separate from the 503.

2. Validate before counting

A parser can decode a JSON value without establishing that the value is useful. A list, a string status, or a boolean status does not meet this project’s record contract. The Python JSON reference documents decoding errors and warns that untrusted input can consume resources. We bound each line and check the decoded shape.

"""Summarize bounded JSON Lines web-log records. Standard library only."""
import argparse
from collections import Counter
import json
from pathlib import Path
import sys

MAX_LINE_BYTES = 64 * 1024


def summarize(path: Path) -> dict:
    groups = Counter()
    valid = invalid = 0
    with path.open('rb') as stream:
        while True:
            raw = stream.readline(MAX_LINE_BYTES + 1)
            if not raw:
                break
            if len(raw) > MAX_LINE_BYTES:
                while raw and not raw.endswith(b'\n'):
                    raw = stream.readline(MAX_LINE_BYTES + 1)
                invalid += 1
                continue
            try:
                record = json.loads(raw.decode('utf-8'))
                if not isinstance(record, dict):
                    raise ValueError('record must be an object')
                status = record.get('status')
                if type(status) is not int or not 100 <= status <= 599:
                    raise ValueError('status must be an integer from 100 to 599')
            except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
                invalid += 1
                continue
            valid += 1
            groups[f'{status // 100}xx'] += 1
    errors = groups['5xx']
    return {'valid': valid, 'invalid': invalid,
            'status_classes': {f'{n}xx': groups[f'{n}xx'] for n in range(1, 6)},
            'server_error_percent': round(100 * errors / valid, 2) if valid else None}


def main() -> int:
    parser = argparse.ArgumentParser(description='Summarize JSONL records with an integer status field.')
    parser.add_argument('logfile', type=Path)
    args = parser.parse_args()
    try:
        result = summarize(args.logfile)
    except OSError as error:
        print(f'ERROR: {error}', file=sys.stderr)
        return 2
    print(json.dumps(result, indent=2))
    return 1 if result['invalid'] else 0


if __name__ == '__main__':
    raise SystemExit(main())

3. Run and inspect the report

python log_summary.py example.jsonl

Expect valid: 4, invalid: 0, status-class counts of 0, 2, 0, 1, and 1, and server_error_percent: 25.0. The report includes empty classes so consumers do not need to guess whether a missing key means zero.

Now add a line containing {"status": "503"}. The valid count remains four and the invalid count becomes one. The process returns status one to make rejected records visible to an automated workflow; it still prints the report. A missing or unreadable input file returns status two.

4. Defend the denominator

An error rate is only as meaningful as the records it covers. We divide server errors by valid records and report rejected input separately. Silently treating malformed records as successful requests would make a damaged dataset appear healthier. Treating an empty dataset as zero-percent failure would suggest evidence that does not exist; this report uses JSON null instead.

This is a record-based measure, not a measurement of unique users, business transactions, or availability over time. Retries can create several log entries for one user action. Missing files, sampling, or collection gaps can change the conclusion. Write down these caveats beside the number.

5. Handle size and privacy deliberately

The reader consumes one bounded line at a time and discards the remainder of an oversized line without decoding it. It counts that line once as invalid. This helps bound line memory, but total runtime still grows with the input size. Rotate large logs and control access to them.

The example does not print request paths, addresses, tokens, or user identifiers. Keep the report focused on the question you need to answer. If you add grouping by path, define how dynamic IDs and query parameters are normalized before publishing results.

Next exercise

Add a threshold option that returns a different status when the server-error share exceeds a chosen limit. Define how empty input and malformed rows affect that decision, then write fixtures for each branch before changing the code.

What was checked

The examples were exercised with fixture data using Python 3.14.7 and Node.js 26.7.0 on Windows. The guide targets Python 3.11+ or Node.js 24+ as specified. This is an example-project check, not a production audit or a benchmark.

Download the project files (ZIP) ↗

The ZIP contains readable source, sample log data, a README, and an MIT license. Inspect the files before running them. Nothing needs administrator access.