twimco.Let’s talk ↗︎
Menu
Field notes / Product & engineering

DynamoDB can bulk-put from S3 now. Retire the homemade loader before it fails mid-peak

On November 5, 2026 AWS made bulk put items generally available for DynamoDB: point at files in S3 and write into an existing table without babysitting a custom pipeline. Catalog refreshes and backfills finally have a managed path.

Editorial engraving of a sunny data studio where a backend lead works on a laptop from the sofa while a teammate arranges paper cards on a rolling whiteboard.

Amazon DynamoDB's November 5, 2026 bulk put items feature writes objects from S3 into an existing table with per-item condition expressions for insert-only or version-guarded writes. Standard tier covers most workloads; Accelerated is for time-sensitive jobs. Use it for daily refreshes, catalog enrichments, backfills, and migrations—after you decide overwrite rules, failure alerts, and who owns a partial load.

What AWS shipped on November 5

AWS announced general availability of bulk put items for Amazon DynamoDB on November 5, 2026. The feature writes data from Amazon S3 into an existing DynamoDB table. The problem it targets is familiar: other systems already land files in S3, and getting those rows into a live table usually meant a homemade pipeline that fought table capacity and failed in the least convenient hour.

With bulk put items you point at the S3 location of the input files and DynamoDB handles the write path. Each item can carry its own condition expression, so you can enforce insert-only behavior or version-guarded updates instead of blindly overwriting. AWS describes a Standard tier for most workloads and an Accelerated tier for time-sensitive jobs. Named use cases include daily refreshes, large catalog enrichments, backfills, and migrations. Standard tier is available in commercial AWS Regions and AWS GovCloud (US); Accelerated availability is regional—check the list before you promise a deadline.

A managed bulk put is still a write into production. Name the condition expression before you point it at S3.

Decision guide

Homemade loader vs managed bulk put

ConsiderationCustom pipeline painWhat bulk put asks instead
Hand-scaled workersWorkers you scale and restart by handPoint at an S3 prefix and let DynamoDB write
Capacity surprisesCapacity surprises mid-loadChoose Standard or Accelerated for the job shape
Silent overwritesSilent overwrites of live keysPer-item condition expressions for insert-only or version guards
Orphaned backfillsNobody owns a half-finished backfillName an owner for partial success and replay
Decision framework: assess these conditions against your own data path.

Why homemade loaders quietly fail

Custom S3-to-DynamoDB jobs accumulate sharp edges. Someone tunes worker count for last month's catalog size. Capacity mode surprises show up as throttling that looks like "the site is slow." A retry path double-writes. A partial success leaves half the assortment current and half stale, and the on-call thread argues about whether to rerun everything.

If your web applications depend on DynamoDB for catalog, session-adjacent state, or operational lookups, that homemade loader is part of the product—whether it has an owner or not. Bulk put does not remove ownership. It removes the excuse that you had no managed option.

A managed bulk put is still a write into production. Name the condition expression before you point it at S3.

Design the write rules first

Insert-only vs upsert. Some enrichments must never clobber a key a human just corrected. Condition expressions exist so you can say that in the load, not in a postmortem.

Version guards. If producers stamp a version or updated-at, require it. Bulk put without a guard is a fast way to revive yesterday's bad feed.

File layout. Agree what "done" means in S3: complete partitions, checksum sidecars, or a manifest. Pointing DynamoDB at a prefix that is still receiving files is how you load a half day.

Tier choice. Standard is the default story. Accelerated is for when the business clock is real—holiday assortment cutovers, not casual curiosity. Do not buy speed you will not monitor.

How the work moves

Before the first bulk put to production

  1. Shape the files

    Agree on S3 layout, item schema, and which keys are authoritative.

  2. Write conditions

    Choose insert-only vs version-guarded expressions per item class.

  3. Pick a tier

    Standard for routine refreshes; Accelerated only when the clock is real.

  4. Dress rehearsal

    Run against a non-prod table that mirrors production keys, then promote.

  5. Alert the partial

    Define who gets paged when a load finishes incomplete.

Illustrative workflow, not a client screenshot or measured result.

Operating the job like a release

Treat the first production bulk put like a cutover, not a cron experiment.

  1. Mirror key patterns in a non-production table and run the same S3 sample.
  2. Compare item counts and spot-check conflict cases your conditions should reject.
  3. Document who may start a load, who may cancel, and who decides to replay.
  4. Alert on incomplete completion the same way you alert on a failed deploy.
  5. Only then schedule the daily refresh that replaces the homemade workers.

Keep the old pipeline around for one cycle as a rollback path, then delete it on purpose. Abandoned loaders are how two writers fight over the same keys three months later.

What not to do

Do not invent cost savings for a board slide—AWS did not publish a single "average savings" number in the what's-new note. Do not skip conditions because "it's just a refresh." Do not run Accelerated on every job because the name sounds serious. And do not let the data producer and the table owner stay ambiguous; bulk put makes their handoff visible.

When the load feeds customer-facing availability or pricing, walk one SKU from S3 object to table item to storefront before you trust the schedule. The feature is managed. The promise to the shopper is still yours.

If you want help redesigning the write path as part of a broader application build, talk to us.

Catalog enrichments without melting the table. Assortment teams love a nightly enrichment file. Engineering loves not owning a brittle loader job. Bulk put sits between them only if the contract is written: which attributes may change, which keys are sacred, what happens when the file is late, and who pages when item counts disagree with yesterday.

For large catalog enrichments, stage in a parallel table or a prefixed keyspace when you can, then swap reads after validation. When you must write in place, condition expressions are the difference between a refresh and a corruption event. Insert-only for new keys, version guards for updates, and an explicit deny for keys marked human-override.

Migrations deserve a written freeze window. Bulk put does not care that merchandising is mid-edit in admin. Your freeze does. Put the freeze on the same calendar as the load, with a named person who can halt the job.

Accelerated tier will not save a bad file layout. If S3 objects arrive incomplete, you will accelerate incomplete data. Standard tier with a verified manifest beats a heroic Accelerated run against a moving prefix.

Observability you should demand on day one. Before the first production job, agree what finished emits: items attempted, items written, items rejected by condition, and bytes read from S3. If those counters are missing, you will discover failures through customer symptoms. Wire alerts to the same channel as deploy failures so the load is treated like a release.

Document idempotency expectations. Re-running a daily refresh should be safe because conditions make it so—not because someone hopes the file did not change. When producers republish a corrected file, the version guard should accept the newer stamp and reject the stale one.

For teams already deep into custom pipelines, migrate one job family at a time. Catalog enrichment first, then backfills, then migrations. Retire workers only after two clean cycles. Homemade loaders that stay just in case forever become a second writer with no owner.

When the enrichment feeds storefront availability, walk one SKU from the S3 object through the table to the page a customer sees. If that path crosses a cache, name who purges it after the load. Bulk put ends at DynamoDB; the customer promise does not.

Related work: Spin:Market

twimco.

We build alongside the people doing the work.
Bring us the problem you’re thinking about.

Related work

Spin:Market ↗︎