CI/CD Scheduling & Automation for Spatial Report Pipelines

A spatial report that renders perfectly on an analyst’s laptop is not yet a product. The moment the same PDF must be produced every night for two hundred jurisdictions, regenerated whenever an upstream dataset changes, and archived so that last quarter’s submission can be reproduced byte-for-byte, the interesting problems move out of the template and into the pipeline around it. This section owns that pipeline: how to package the rendering stack into a container that behaves identically everywhere, how to schedule and trigger it, how to fan the work out across parallel workers without corrupting shared state, and how to store the resulting documents so they are versioned, discoverable, and auditable.

The audience here is engineers who already know how to generate a report — the templating, map embedding, and layout mechanics are covered elsewhere on this site — and now need to run that generation unattended and at scale. The recurring theme is productionization rather than authoring: reproducible images over “works on my machine,” idempotent runs over fire-and-forget scripts, immutable artifacts over overwritten files, and observability over silent cron jobs that fail into a void. Everything below assumes Python 3.10+, a rendering engine such as weasyprint or reportlab, and a container runtime.

Foundational Architecture

The automation layer sits downstream of every content concern and upstream of every consumer. Its job is to take a rendering routine that is known to work and execute it reliably, repeatedly, and in parallel. Five stages recur in every mature deployment, and keeping them decoupled is what makes the system debuggable when a 3 a.m. batch fails.

Data Contracts at the Automation Boundary

The automation layer should never reach into a database or reproject a geometry itself. Those are content concerns. What it depends on is a stable run contract: a small, serialisable descriptor that names one unit of work. A good run descriptor carries the report identifier, a hash of the exact input data snapshot, the template revision, the target output location, and nothing else. Because the descriptor is pure data, it can be enqueued, logged, retried, and diffed. It is also the natural key for idempotency — two runs with the same descriptor must produce the same artifact.

from dataclasses import dataclass, asdict
import hashlib
import json


@dataclass(frozen=True)
class RunSpec:
    """One unit of report work. The fingerprint keys idempotency and storage."""
    report_id: str
    data_snapshot_uri: str      # immutable pointer, e.g. s3://.../2026-07-12/parcels.gpkg
    template_ref: str           # git sha or semver of the template bundle
    output_prefix: str

    def fingerprint(self) -> str:
        payload = json.dumps(asdict(self), sort_keys=True).encode()
        return hashlib.sha256(payload).hexdigest()[:16]

Orchestration Overview

Three moving parts turn a RunSpec into a stored PDF: a scheduler or trigger that decides when to run, a container image that defines how to run, and an artifact store that records what was produced. The scheduler is deliberately dumb — it dispatches descriptors and enforces timeouts and retries, but knows nothing about GIS. The image carries the entire rendering stack so the same bytes execute in CI, in a nightly cron job, and in a developer’s local reproduction. The store closes the loop by making every output addressable and immutable. The three child guides in this section each take one of these parts to depth: Headless PDF Rendering in Docker Containers for the image, Scheduled Batch Report Generation for the scheduler, and Report Artifact Storage & Versioning for the store.

Core Concepts

Reproducible Images Over “Works on My Machine”

A rendering pipeline is only as reliable as its least-pinned dependency. Spatial report images are unusually fragile here because the output depends on system libraries far below the Python layer: Pango and Cairo for text shaping, fontconfig for font resolution, GDAL for any geometry the render path touches. An unpinned apt-get install libpango1.0-0 today and next month can shape the same paragraph to different line lengths, silently pushing a table onto a new page. Reproducibility therefore means pinning three tiers together — the base image by digest, the system packages by version, and the Python dependencies by lockfile — and rebuilding the fontconfig cache as an explicit build step rather than relying on whatever the base layer left behind.

The practical test of reproducibility is not “does it build” but “does an image built today from the same inputs render pixel-identical pages to one built last week.” Treat the font set as source code. This is the exact territory that Installing WeasyPrint Dependencies in Slim Docker Images covers, and it interacts directly with the WeasyPrint PDF Rendering Pipeline choices made upstream.

Idempotent Runs

A batch job that runs on a schedule will be run twice — a retry fires after a network blip, an operator re-triggers a job they thought failed, two overlapping cron ticks collide. If the second run produces a different or duplicate artifact, the archive becomes untrustworthy. Idempotency is enforced at the storage key: derive the output path from the RunSpec fingerprint, and make the write conditional on absence (or on a matching content hash). A rerun with identical inputs then becomes a no-op, while a rerun with changed inputs writes a genuinely new version. This is why the run descriptor hashes the data snapshot, not the wall-clock time — the goal is that “the same report over the same data” is always the same object.

Secrets and Configuration Separation

Rendering pipelines routinely need database credentials, object-store keys, and per-tenant API tokens for map providers. None of these belong in the image or the repository. The workable rule is that the image is public-safe and the runtime injects everything environment-specific: connection strings, bucket names, and credentials arrive as environment variables or mounted secret files, never baked layers. Configuration that does vary the output — page size, locale, theme slug — belongs in the RunSpec or a checked-in config file so it is versioned alongside results. Keeping secrets and output-affecting config in separate channels is what lets you rotate a credential without invalidating a single archived report.

Observability for Unattended Jobs

The failure mode of a cron job is silence. A scheduled render that dies at 03:14 with no log, no metric, and no alert is indistinguishable from one that never ran. Three signals make unattended pipelines operable: structured logs that carry the report id and fingerprint on every line so a single run is greppable across workers; run metrics — count, duration, page count, output bytes — emitted per run so a batch that renders but produces suspiciously small PDFs is visible; and exit codes that are honest, so the scheduler’s own retry and alerting can react. Emit these from the render routine itself, not from a wrapper, so they survive whether the job runs in CI, cron, or a task queue.

Implementation Patterns

A Reproducible Render Image

The Dockerfile is the single source of truth for the rendering environment. Pin the base by tag (ideally by digest in production), install the shaping and font stack explicitly, and copy a lockfile before the application code so dependency layers cache independently of source changes.

FROM python:3.12-slim-bookworm

# Text shaping + rasterisation stack for WeasyPrint, plus a known font set.
RUN apt-get update && apt-get install -y --no-install-recommends \
        libpango-1.0-0=1.50.* \
        libpangocairo-1.0-0 \
        libcairo2 \
        libgdk-pixbuf-2.0-0 \
        libffi8 \
        fonts-dejavu-core \
        fonts-noto-cjk \
    && fc-cache -f \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY requirements.lock ./
RUN pip install --no-cache-dir -r requirements.lock

COPY src/ ./src/
ENTRYPOINT ["python", "-m", "src.render_cli"]

The fc-cache -f line is not optional cleanup — it rebuilds fontconfig’s index so the fonts installed above are actually resolvable at render time. Pinning the CJK font here means multilingual labels behave the same in CI as in production, which ties back to Embedding CJK Fonts for Multilingual Map Labels.

An Idempotent Render Entry Point

The container’s entry point takes a RunSpec, checks whether the target artifact already exists, renders only if needed, and exits with a meaningful code. Keeping this logic thin and typed makes it identical whether invoked by cron, a CI step, or a Celery task.

import logging
import sys
from pathlib import Path

log = logging.getLogger("render")


def execute(spec: RunSpec, store: "ArtifactStore", render_fn) -> int:
    """Render one report idempotently. Returns a process exit code."""
    key = f"{spec.output_prefix}/{spec.report_id}/{spec.fingerprint()}.pdf"
    if store.exists(key):
        log.info("skip report_id=%s fp=%s reason=already_present",
                 spec.report_id, spec.fingerprint())
        return 0
    try:
        pdf_bytes = render_fn(spec)                 # calls WeasyPrint/ReportLab
    except Exception:                               # noqa: BLE001 - top-level guard
        log.exception("render_failed report_id=%s fp=%s",
                      spec.report_id, spec.fingerprint())
        return 1
    if len(pdf_bytes) < 1_024:
        log.error("suspect_output report_id=%s bytes=%d", spec.report_id, len(pdf_bytes))
        return 2
    store.put(key, pdf_bytes, if_absent=True)
    log.info("rendered report_id=%s fp=%s bytes=%d key=%s",
             spec.report_id, spec.fingerprint(), len(pdf_bytes), key)
    return 0

The distinct exit codes matter: 1 for a render exception, 2 for output that rendered but looks wrong. A scheduler can retry a 1 and page a human on a 2.

A Scheduled Workflow

GitHub Actions is a natural home when the templates, schedule, and results live near the code. A matrix fans the batch across parallel jobs, each pinned to the same built image so no runner drifts.

name: nightly-spatial-reports
on:
  schedule:
    - cron: "15 2 * * *"      # 02:15 UTC nightly
  workflow_dispatch: {}        # manual re-trigger

jobs:
  render:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/org/spatial-render:${{ github.sha }}
    strategy:
      fail-fast: false
      matrix:
        shard: [0, 1, 2, 3]
    steps:
      - name: Render shard
        env:
          PG_DSN: ${{ secrets.PG_DSN }}
          ARTIFACT_BUCKET: ${{ secrets.ARTIFACT_BUCKET }}
        run: python -m src.batch --shard ${{ matrix.shard }} --of 4

fail-fast: false keeps a single bad jurisdiction from cancelling the other three shards — one region’s malformed geometry should not abort the whole night’s run. The scheduling and sharding mechanics get full treatment in Scheduling Nightly PDF Runs with Cron and GitHub Actions.

Bounded Parallel Rendering

Within a single worker or job, a process pool renders multiple reports concurrently. Rendering is CPU-bound and each engine instance carries non-trivial memory, so concurrency must be bounded and each process must own its own engine — sharing a WeasyPrint or reportlab context across processes is a reliable way to corrupt output.

from concurrent.futures import ProcessPoolExecutor, as_completed


def render_batch(specs: list[RunSpec], store, render_fn, workers: int = 4) -> dict[str, int]:
    """Render a shard of reports with bounded process-level parallelism."""
    outcomes: dict[str, int] = {}
    with ProcessPoolExecutor(max_workers=workers) as pool:
        futures = {pool.submit(execute, s, store, render_fn): s.report_id for s in specs}
        for fut in as_completed(futures):
            rid = futures[fut]
            try:
                outcomes[rid] = fut.result()
            except Exception:                       # noqa: BLE001
                log.exception("worker_crash report_id=%s", rid)
                outcomes[rid] = 1
    return outcomes

Memory, not CPU, is usually the ceiling — a worker holding a large geopandas frame plus a rendered page tree can spike hard. Parallel Batch Rendering with ProcessPoolExecutor works through sizing the pool against those limits.

Integration & Output Constraints

Productionizing the render step exposes constraints that never surface in interactive development. The largest is fonts. A slim base image ships essentially no fonts, so weasyprint falls back to a default face, line breaks move, and a table that fit on one page locally overflows onto two in CI. Because line-length changes cascade into pagination, this can also break the row-splitting guarantees described in Preventing Table Row Splits Across PDF Page Breaks. The fix is to install and pin the exact font set the templates assume and to rebuild the fontconfig cache in the image.

The second constraint is the display server. Tools that draw with a real GUI toolkit — headless QGIS, Mapnik in some configurations — expect an X server that a bare container lacks. The remedy is a virtual framebuffer (xvfb-run) around the offending process, or, better, replacing it with a pure-Python static-map path that needs no display at all. Running QGIS and Mapnik Headless with Xvfb covers when each approach is warranted.

The third constraint is engine selection under CI conditions. weasyprint is HTML- and CSS-driven and pairs naturally with the Jinja2 Templating & Theme Logic stack, but it pulls in the entire Pango/Cairo/fontconfig chain — a heavier, font-sensitive image. reportlab draws to the canvas programmatically with almost no system dependencies, producing a smaller, more hermetic image, at the cost of writing layout in Python instead of CSS. The trade-off is dissected in WeasyPrint vs ReportLab for Spatial PDFs; for pipelines that must minimise image surface and system-library risk, the ReportLab path documented under ReportLab Programmatic Layout is often the more CI-friendly choice.

Page geometry, bleed, and sizing must stay consistent no matter which engine and scheduler run the job — the Document Architecture & Layout Rules for Spatial Reports section defines those invariants, and the automation layer’s responsibility is only to reproduce them faithfully across every run.

Validation & Testing

Automation without validation just fails faster and more often. Three checks, run in the same CI that builds the image, catch the overwhelming majority of production regressions before they reach an archive.

Smoke Renders on Every Build

The cheapest, highest-value gate is a render of fixed sample data on every image build. It proves the whole stack — fonts, libraries, template, engine — actually produces a non-trivial PDF, and it fails the build loudly if a dependency bump broke the font chain.

import subprocess
import sys


def smoke_render(sample_spec: RunSpec, render_fn) -> None:
    pdf = render_fn(sample_spec)
    assert pdf[:5] == b"%PDF-", "output is not a PDF"
    assert len(pdf) > 10_000, f"suspiciously small PDF: {len(pdf)} bytes"
    print(f"smoke ok: {len(pdf)} bytes")


if __name__ == "__main__":
    sys.exit(0)

PDF Diff Regression

Content-hash comparison is too brittle for PDFs because embedded creation timestamps change every run. The durable technique is visual: rasterise each page and compare it to an approved baseline with a tolerance for anti-aliasing noise. Any real layout change — a shifted table, a re-flowed paragraph, a missing map frame — trips the threshold, while a re-run of unchanged inputs passes.

from pdf2image import convert_from_bytes
from PIL import ImageChops
import numpy as np


def page_diff_ratio(candidate: bytes, baseline: bytes, dpi: int = 100) -> float:
    """Return the max per-page fraction of differing pixels across a PDF."""
    cand = convert_from_bytes(candidate, dpi=dpi)
    base = convert_from_bytes(baseline, dpi=dpi)
    assert len(cand) == len(base), f"page count changed: {len(cand)} vs {len(base)}"
    worst = 0.0
    for c, b in zip(cand, base):
        diff = np.asarray(ImageChops.difference(c.convert("L"), b.convert("L")))
        worst = max(worst, float((diff > 12).mean()))   # 12 = anti-alias tolerance
    return worst


# In CI: fail the build if any page drifts more than 0.2% of its pixels.
assert page_diff_ratio(new_pdf, approved_pdf) < 0.002

Honest Exit Codes and Run Assertions

The scheduler can only react to what the process reports. A batch runner should aggregate per-report exit codes and fail the job if any report failed, while still attempting every report so one bad input does not mask the rest. Pair this with cheap post-render assertions — expected page count, non-empty output, presence of a required map frame — so a report that renders but is structurally wrong exits non-zero rather than being archived as valid.

def batch_exit_code(outcomes: dict[str, int]) -> int:
    """Non-zero if any report failed; log a per-code summary for alerting."""
    failed = {rid: code for rid, code in outcomes.items() if code != 0}
    if failed:
        for rid, code in sorted(failed.items()):
            log.error("report_failed report_id=%s exit=%d", rid, code)
        return 1
    log.info("batch_ok count=%d", len(outcomes))
    return 0

Guides in This Section

• Headless PDF Rendering in Docker Containers — building a reproducible image that carries the full WeasyPrint or ReportLab stack, including the font and shaping libraries slim bases omit.
• Scheduled Batch Report Generation — driving that image from cron, GitHub Actions, or Celery beat, with sharding, retries, and bounded parallel workers for large report sets.
• Report Artifact Storage & Versioning — writing immutable, content-addressed PDFs to object storage and artifact registries with a manifest so every output is reproducible and auditable.

Related

• Jinja2 Templating & Theme Logic for Automated Spatial Reporting — the rendering routine that this automation layer packages, schedules, and runs at scale.
• Document Architecture & Layout Rules for Spatial Reports — the page geometry and print invariants that every scheduled run must reproduce faithfully.
• WeasyPrint PDF Rendering Pipeline — the HTML-to-PDF engine whose system dependencies drive most headless CI complexity.
• Dynamic Map Data Embedding Workflows — the map and chart assets that must be generated before, or during, a containerised render.
• Table Pagination Strategies for Large Attribute Tables — the pagination behaviour most sensitive to font drift between local and CI environments.

Conclusion

Turning a working spatial report into a dependable pipeline is an exercise in removing surprises: pin the image so it renders identically everywhere, key runs to a data fingerprint so reruns are safe, and store outputs as immutable versions so any submission can be reproduced. Add the observability to see failures and the validation — smoke renders, visual diffs, honest exit codes — to catch regressions before they ship, and unattended generation stops being a liability. The three guides below take the container, the schedule, and the artifact store each to production depth.