Spatial Reporting Automated Docs & Maps

Versioning Generated PDFs in S3 Object Storage

Once a spatial report is rendered, the question is not where it goes but which version a stakeholder is looking at three months later during an audit. S3 answers this with object versioning, content-addressable keys, and Object Lock immutability — a combination that gives every nightly PDF a permanent, verifiable identity. This guide, part of the Report Artifact Storage & Versioning workflow, shows how to upload with boto3 using content-hash keys, enable bucket versioning, set lifecycle rules that expire stale versions, apply Object Lock for regulatory immutability, attach metadata tags, and hand out presigned URLs for distribution.

Content-addressed PDF versioning in S3 Flow from a rendered PDF through SHA-256 hashing into a content-addressed object key, uploaded via boto3 put_object to a versioned S3 bucket with Object Lock and lifecycle rules, then served to consumers through a time-limited presigned URL. rendered PDF bytes SHA-256 key reports/<hash>.pdf versioned bucket boto3 put_object Object Lock (WORM) lifecycle expiry presigned URL time-limited download

Prerequisites

  • Python 3.10+ with boto3>=1.34 (pip install boto3)
  • An AWS account and a bucket you can configure, plus IAM credentials via role or environment
  • A rendered PDF on disk — the output of the Parallel Batch Rendering with ProcessPoolExecutor batch, for example
  • Basic familiarity with S3 keys, versions, and IAM policies

Step 1: Enable Bucket Versioning

Versioning must be on before the first upload — objects written to a never-versioned bucket have a null version ID that cannot be recovered after overwrite. Enable it once, idempotently, at pipeline setup.

Python
# s3_setup.py
from __future__ import annotations
import boto3

s3 = boto3.client("s3")


def ensure_versioning(bucket: str) -> None:
    status = s3.get_bucket_versioning(Bucket=bucket).get("Status")
    if status != "Enabled":
        s3.put_bucket_versioning(
            Bucket=bucket,
            VersioningConfiguration={"Status": "Enabled"},
        )

With versioning enabled, overwriting reports/flood/latest.pdf no longer destroys yesterday’s file — the prior bytes are retained under a distinct version ID and remain retrievable for audits.

Step 2: Derive Content-Hash Object Keys

A content-addressable key — the SHA-256 of the file bytes — makes identical renders deduplicate automatically and gives every distinct report an immutable, self-verifying name. Two runs that produce byte-identical PDFs collapse to one object; any change produces a new key.

Python
# keys.py
from __future__ import annotations
import hashlib
from pathlib import Path


def content_key(pdf_path: str, prefix: str = "reports") -> tuple[str, str]:
    """Return (s3_key, hex_digest) for a rendered PDF."""
    data = Path(pdf_path).read_bytes()
    digest = hashlib.sha256(data).hexdigest()
    # Shard by first two hex chars to avoid hot key prefixes at scale
    return f"{prefix}/{digest[:2]}/{digest}.pdf", digest

Sharding by the first two hex characters spreads writes across key prefixes, which historically improved S3 request throughput and still keeps listings manageable. Pair the content key with a human-friendly pointer object (reports/flood/latest) whose body is just the hash, so consumers can resolve “the newest flood report” without listing.

Step 3: Upload With boto3 and Metadata Tags

put_object carries three kinds of self-description: the ContentType (so browsers render inline), user Metadata (arbitrary key/value pairs frozen onto the object), and Tagging (mutable labels usable in lifecycle and IAM conditions).

Python
# upload.py
from __future__ import annotations
import boto3
from pathlib import Path
from urllib.parse import urlencode

s3 = boto3.client("s3")


def upload_report(bucket: str, pdf_path: str, region: str, report_date: str) -> str:
    from keys import content_key
    key, digest = content_key(pdf_path)
    s3.put_object(
        Bucket=bucket,
        Key=key,
        Body=Path(pdf_path).read_bytes(),
        ContentType="application/pdf",
        ContentDisposition="inline",
        Metadata={                      # immutable, returned on every GET
            "sha256": digest,
            "region": region,
            "report-date": report_date,
        },
        Tagging=urlencode({             # mutable, queryable by lifecycle/IAM
            "class": "spatial-report",
            "retention": "long",
        }),
    )
    return key

Metadata is fixed for the life of the object version and is ideal for the provenance you never want to lose — the content hash and the source date. Tags are mutable and drive the lifecycle rules in the next step.

Step 4: Add Lifecycle and Object Lock Rules

Versioning accumulates every overwrite forever unless you expire noncurrent versions. A lifecycle rule trims old versions after a retention window, while Object Lock enforces write-once-read-many (WORM) immutability for reports under regulatory hold.

Python
# lifecycle.py
from __future__ import annotations
import boto3

s3 = boto3.client("s3")


def configure_lifecycle(bucket: str) -> None:
    s3.put_bucket_lifecycle_configuration(
        Bucket=bucket,
        LifecycleConfiguration={
            "Rules": [{
                "ID": "expire-old-report-versions",
                "Filter": {"Tag": {"Key": "retention", "Value": "long"}},
                "Status": "Enabled",
                "NoncurrentVersionExpiration": {"NoncurrentDays": 365},
                "AbortIncompleteMultipartUpload": {"DaysAfterInitiation": 7},
            }],
        },
    )


def lock_object(bucket: str, key: str, retain_until: str) -> None:
    """Apply governance-mode retention to a specific report version."""
    s3.put_object_retention(
        Bucket=bucket, Key=key,
        Retention={"Mode": "GOVERNANCE", "RetainUntilDate": retain_until},
    )

Object Lock requires the bucket to have been created with lock enabled — it cannot be added later — so provision compliance buckets deliberately. GOVERNANCE mode lets privileged users lift a lock; COMPLIANCE mode lets no one, including root, delete before expiry. The immutability discipline here parallels the checksummed, promotable bundles in Publishing Report Bundles to an Artifact Registry.

Step 5: Distribute With Presigned URLs

Consumers should never hold bucket credentials. A presigned URL grants time-limited GET access to a single object version, so you can email a link that stops working after an hour.

Python
# distribute.py
from __future__ import annotations
import boto3

s3 = boto3.client("s3")


def presign(bucket: str, key: str, version_id: str | None = None,
            expires: int = 3600) -> str:
    params = {"Bucket": bucket, "Key": key}
    if version_id:
        params["VersionId"] = version_id   # pin to an exact historical render
    return s3.generate_presigned_url(
        "get_object", Params=params, ExpiresIn=expires,
    )

Pinning VersionId is what makes an audit link stable: the URL resolves to the exact bytes rendered on that date even after newer versions land. These links are how a scheduled pipeline hands finished reports to downstream consumers described in Scheduling Nightly PDF Runs with Cron and GitHub Actions.

Key Parameters / Configuration Reference

Parameter API Value Effect
VersioningConfiguration.Status put_bucket_versioning Enabled Preserve prior bytes on overwrite
Metadata put_object dict Immutable provenance frozen per version
Tagging put_object urlencoded Mutable labels for lifecycle & IAM
NoncurrentVersionExpiration lifecycle NoncurrentDays: 365 Trim old versions after a year
Retention.Mode put_object_retention GOVERNANCE / COMPLIANCE WORM immutability strictness
ExpiresIn generate_presigned_url 3600 Link validity in seconds

Common Pitfalls

  • Enabling versioning after the first upload. Objects written before versioning have a null version ID that is silently lost on overwrite. Enable versioning at setup, before any put_object.
  • Confusing metadata with tags. User metadata is immutable and set only at write time; tags are mutable and drive lifecycle rules. Storing the retention class as immutable metadata means you can never change the policy.
  • Expecting to add Object Lock to an existing bucket. Lock must be enabled at bucket creation. A live bucket cannot be retrofitted; you must create a new lock-enabled bucket and migrate.
  • Presigning without a version ID for audit links. A bare-key presigned URL follows the current version and drifts as new renders land. Pin VersionId for any link that must reference a specific historical report.

Verification

Confirm versioning is active, the object round-trips with its metadata, and the presigned link resolves:

Python
import boto3, hashlib, urllib.request
from pathlib import Path

s3 = boto3.client("s3")
assert s3.get_bucket_versioning(Bucket="my-reports")["Status"] == "Enabled"

key = upload_report("my-reports", "out/flood.pdf", "coastal", "2026-07-12")
head = s3.head_object(Bucket="my-reports", Key=key)
assert head["Metadata"]["sha256"] == hashlib.sha256(Path("out/flood.pdf").read_bytes()).hexdigest()

url = presign("my-reports", key, version_id=head["VersionId"])
with urllib.request.urlopen(url) as r:
    assert r.status == 200 and r.headers["Content-Type"] == "application/pdf"
print("verified content-addressed, versioned, and distributable")

Matching the stored sha256 metadata against a freshly computed digest proves the upload was not corrupted, and a 200 from the presigned URL confirms the distribution path works end to end.