Object storage

The foundation supports Cloudflare R2, Amazon S3, and Google Cloud Storage behind one upload contract. The included product purpose is a public profile image. Uploads cross a private staging boundary before an immutable public object is created; new upload types require their own policy, ownership, and adversarial proof.

Provider selection

Guided AWS and GCP deployment provisions the selected native buckets, CORS, lifecycle policy, public access, and workload permissions. Azure remains a supported deployment target but has no Azure Blob adapter.

STORAGE_PROVIDER is one of disabled, r2, s3, or gcs. Every enabled provider requires:

STORAGE_BUCKET_NAME
STORAGE_STAGING_BUCKET_NAME
STORAGE_PUBLIC_URL

The destination and staging buckets must differ. The staging bucket must never be public. Provider requirements are additive:

Partial selected-provider configuration fails environment validation. disabled preserves the zero-key local path and returns a stable 503 from upload endpoints.

Included upload contract

PROFILE_PIC permits:

• authenticated owner only;
• image/jpeg, image/jpg, image/png, or image/webp;
• 1 byte through 5 MiB;
• a 15-minute presigned PUT to pending/<user-id>/<upload-id>.<extension>;
• a validated conditional copy to user-profiles/<user-id>/<upload-id>.<extension>;
• a public immutable-cache URL only after confirmed completion.

The profile image is public by design. Do not reuse this flow for private documents, identity data, financial files, or confidential customer uploads.

USER_FILE backs the Files module and permits:

• authenticated owner only; entity_id must equal the session user id;
• image/jpeg, image/jpg, image/png, image/webp, or application/pdf;
• 1 byte through 10 MiB;
• a 15-minute presigned PUT to pending/<user-id>/<upload-id>.<extension>;
• a validated conditional copy to user-files/<user-id>/<upload-id>.<extension> inside the staging bucket, outside the pending/ prefix that the one-day lifecycle rule expires;
• no public URL, ever. Delivery is a 5-minute presigned GET whose signature pins response-content-disposition: attachment and response-content-type to the stored MIME type, so a PDF cannot render inline from the bucket origin.

Private placement needs no new bucket and no new environment variable. The uploads row is the durable file record; the Files API speaks file_id (uploads.upload_id) only and never accepts or returns a storage key.

PDF bytes are checked for structure — a %PDF-1.<digit> header, and startxref before %%EOF within the final 1 KiB. That stops a renamed PNG from passing as a PDF. It is not sanitization and not malware scanning; attachment-only private delivery is the compensating control. Widening the MIME allowlist requires extending both the validation and the delivery story first.

Deleting a file transitions the owner-scoped row to deleted in one guarded statement, then deletes the object best-effort. A replayed delete, a cross-owner delete, and an unknown id are indistinguishable: all three return the same not-found response.

State machine

sequenceDiagram
  participant Browser
  participant API
  participant DB as Postgres
  participant Stage as Private staging bucket
  participant Public as Public asset bucket

  Browser->>API: purpose + owner target + MIME + exact byte size
  API->>API: session, limit, target ownership, lifecycle readiness
  API->>DB: create pending claim with private and public keys
  API-->>Browser: 15-minute staged PUT + upload ID + required Content-Type
  Browser->>Stage: PUT with signed Content-Length and Content-Type
  Browser->>API: confirm upload ID
  API->>DB: lock owner-scoped pending claim
  API->>Stage: read metadata and exact claimed object version
  API->>Public: publish the validated object version with immutable metadata
  API->>DB: complete once
  API->>Stage: delete temporary object
  API-->>Browser: public key and URL

The presigned URL is a temporary bearer credential that addresses only the private staging bucket. Its signature binds the exact declared Content-Length and Content-Type. Confirmation checks the stored byte count and MIME metadata, reads the complete provider-pinned object, and validates PNG chunks/CRCs, JPEG markers/scans, or WebP RIFF/chunks with bounded dimensions and no trailing polyglot payload.

S3 and R2 pin reads and copies to the same ETag. GCS pins them to the same object generation. An overwrite racing metadata, content validation, or publication therefore fails instead of publishing unvalidated bytes.

The public key is never disclosed before confirmation and never receives a presigned write URL. Published responses use Cache-Control: public, max-age=31536000, immutable. Concurrent or replayed confirmation completes at most once under a database row lock. At most four confirmation operations run concurrently per API process; excess requests receive a retryable 503.

Rejected and completed claims best-effort delete their staged object. A mandatory one-day lifecycle rule is the crash and abandonment backstop. The API verifies that policy before issuing an upload URL, refreshes the verification at least every five minutes, and returns 503 when it is absent or too permissive.

Provider operations

AWS and GCP deployment

Select the native provider during create-app or pnpm deploy:configure. OpenTofu creates two application-specific buckets per environment, configures exact frontend-origin PUT CORS, grants the API workload identity object access, makes only user-profiles/ publicly readable, and applies the one-day pending/ lifecycle rule.

AWS uses the JavaScript SDK default credential chain, so ECS receives temporary task-role credentials. GCS uses Application Default Credentials and grants the Cloud Run service account the object and iam.serviceAccounts.signBlob permissions required for V4 signed URLs. Neither native path stores an application access key.

Cloudflare R2

1. Create a private staging bucket and a public asset bucket per environment.
2. Give API credentials object read/write/delete access to both buckets and lifecycle-read access to staging.
3. Add app-pending-upload-expiry-v1 for the pending/ prefix with one-day expiry.
4. Configure staging CORS for exact app origins, PUT, and Content-Type.
5. Configure the public custom domain/base URL.
6. Set the shared storage values and all three CLOUDFLARE_* credentials atomically.

Keep the lifecycle rule scoped to the pending/ prefix. USER_FILE objects live in the same staging bucket under user-files/, deliberately outside pending/, so a bucket-wide expiry rule would delete users' files after a day. The adapter's readiness check rejects a rule whose prefix is not exactly pending/, so a mis-scoped rule fails closed rather than deleting data.

Do not share production credentials with previews or local development. Use separate buckets, credentials, lifecycle rules, and public bases per environment.

Demo data

scripts/seed-demo-data.sh inserts demo users and a USER_FILE library for the local demo account. Those rows describe objects that do not exist yet, so downloads fail until the bytes are written:

pnpm --filter @app/server seed:demo-objects

That command generates a real, valid PDF or PNG for every seeded row at the exact recorded byte size, then writes it through the same staged PUT and conditional private publish the product uses, so nothing bypasses the storage contract. It refuses to run against a production environment, and skips any object that already exists. Seeded rows are PDF and PNG only because those are the two formats the generator can produce as genuinely valid files.

Known limits

• Structural validation rejects malformed containers and trailing payloads but does not fully decompress pixels, strip EXIF, resize, moderate, or detect malware.
• Lifecycle deletion is asynchronous; application deletion remains the fast path.
• Public URLs remain valid until object deletion and may be cached externally.
• The baseline contains no generic file browser, quota ledger, multipart upload, private download authorization, Azure Blob adapter, or attachment domain.

Add higher-risk purposes only after specifying retention, private/public access, processing, malware/content controls, deletion, abuse/cost bounds, and support behavior.

Verification and diagnosis

Run the provider-independent suite:

bun test --cwd apps/server \
  src/domains/upload/service.test.ts \
  src/infra/storage/r2.test.ts
RUN_DB_INTEGRATION_TESTS=1 DATABASE_URL=postgresql://... \
  bun test --cwd apps/server src/domains/upload/service.integration.test.ts
vp check
pnpm verify:infra

Use disposable buckets for live provider rehearsals. Prove correct upload, wrong origin, wrong owner, wrong MIME header, spoofed bytes, wrong size, expiry, replay, concurrent confirmation, staging cleanup, lifecycle cleanup, publication, and public deletion.

Consult the optional Add an upload type playbook.