DynamoDB Test Data Generation with Realistic Partition Keys

When a test suite runs against a DynamoDB table, the shape of the data matters as much as the volume. A table that stores 10 000 items with a single partition key value will behave very differently from a table that spreads those items across 1 000 distinct partitions. Throughput throttling, hot‑key detection, and global secondary index (GSI) performance all depend on how realistic the partition‑key distribution is.

Yet many teams still generate test data by picking random UUIDs or sequential integers for the partition key. That approach is fast to write, but it hides the very problems you’re trying to catch: uneven access patterns, write‑capacity spikes, and the cost of adaptive capacity.

This post walks through a practical workflow for creating test data that mimics real‑world partition‑key distributions, validates the generated set, and integrates the process into a CI pipeline. The steps are tool‑agnostic, but we’ll point out where a free generator can shave off boilerplate.

1. Why Partition‑Key Realism Matters

Symptom	Root cause when partition keys are unrealistic
Throttling on a single partition	All writes hit the same hash value → write capacity exhausted on one partition.
Skewed GSI query latency	GSI partition key mirrors the base table’s key; a hot base partition creates a hot GSI partition.
Misleading capacity planning	Provisioned capacity looks sufficient in tests but fails in production where traffic is spread.
Incorrect TTL / TTL‑scan behavior	TTL scans run per partition; a single massive partition makes scans appear faster than they really are.

If your test data does not reproduce the cardinality and frequency of real partition keys, any performance or correctness assertion you make is suspect.

2. Decision Criteria for a Generation Strategy

Before writing code, decide which of the following dimensions drive your test‑data requirements.

Dimension	Questions to answer	Typical impact on design
Key cardinality	How many distinct partition‑key values exist in production?	Determines the size of the value pool you must sample from.
Frequency distribution	Is the distribution uniform, Zipfian, Pareto, or a custom mix?	Drives the sampling algorithm (uniform random vs. weighted).
Correlation with sort key	Do certain partition keys always pair with a narrow sort‑key range?	Requires joint generation of PK + SK rather than independent draws.
Attribute richness	How many non‑key attributes, and do they have constraints (enums, ranges, regex)?	Affects schema‑aware generators and validation rules.
Data volume per test run	Do you need 10 k items for unit tests, 10 M for load tests?	Influences batch‑write strategy and whether you stream or bulk‑load.
Refresh cadence	Nightly, per‑PR, on‑demand?	Determines whether you need a deterministic seed or a fast‑regen path.
Tooling constraints	Language, CI environment, existing data‑factory libraries?	Narrows the implementation choices (Python, Java, Go, CLI).

Trade‑off: A fully realistic distribution (e.g., a fitted Zipfian model) costs more CPU to generate and may require a statistical fitting step. A simpler “top‑N hot keys + long tail” model is often good enough for throttling and GSI tests while staying fast.

3. High‑Level Workflow

1. Profile production key distribution   → 2. Choose a statistical model
3. Implement a generator (code or config) → 4. Validate generated data
5. Load into DynamoDB (batch‑write)       → 6. Run tests / collect metrics
7. Automate in CI (seed, cleanup, repeat)

Each step has concrete deliverables; the checklist at the end of the post summarizes them.

4. Worked Example: E‑Commerce Order Table

4.1 Production Profile

Metric	Value (observed over 30 days)
Table name	`Orders`
Partition key	`customer_id` (string)
Sort key	`order_ts` (epoch ms)
Distinct customers	12 400
Top 100 customers	45 % of orders
Remaining customers	long‑tail, roughly Zipf α ≈ 1.2
Avg orders per customer per day	3.2
Non‑key attributes	`order_total` (decimal), `status` (enum), `items` (list of maps)

4.2 Choosing a Model

Hot‑key set – Explicit list of the 100 most active customer_ids with their relative weights (derived from the 45 % share).
Tail – Generate the remaining ~12 300 IDs using a Zipfian sampler (α = 1.2).
Sort key – For each selected customer_id, emit a Poisson‑distributed number of orders per day (λ = 3.2) and assign timestamps uniformly across the day.

This hybrid model captures the heavy‑hitter behavior without fitting a full distribution to the entire 12 k keys.

4.3 Generator Implementation (Python‑ish Pseudocode)

import random, uuid, datetime, json
from zipfian import Zipfian  # lightweight third‑party or custom


# 1️⃣ Load hot‑key weights from a CSV produced by the profiling step


hot_keys = []          # list of (customer_id, weight)
with open("hot_keys.csv") as f:
    for line in f:
        cid, w = line.strip().split(",")
        hot_keys.append((cid, float(w)))


# 2️⃣ Build a Zipfian sampler for the tail


tail_size = 12_300
zipf = Zipfian(n=tail_size, alpha=1.2, seed=42)


# 3️⃣ Helper to pick a customer_id according to the mixed model


def pick_customer():
    if random.random() < 0.45:               # 45 % hot‑key traffic
        return random.choices(
            [cid for cid, _ in hot_keys],
            weights=[w for _, w in hot_keys]
        )[0]
    else:
        idx = zipf.next() - 1                # 1‑based → 0‑based
        return f"cust_tail_{idx:06d}"


# 4️⃣ Generate a day’s worth of orders


def generate_day(day_start: datetime.datetime, target_items: int):
    items = []
    while len(items) < target_items:
        cid = pick_customer()
        # Poisson(λ=3.2) orders for this customer today
        n_orders = max(1, int(random.poissonvariate(3.2)))
        for _ in range(n_orders):
            ts = day_start + datetime.timedelta(
                milliseconds=random.randint(0, 86_400_000)
            )
            item = {
                "customer_id": cid,
                "order_ts": int(ts.timestamp() * 1000),
                "order_total": round(random.uniform(5.0, 500.0), 2),
                "status": random.choice(["PLACED","SHIPPED","DELIVERED","CANCELLED"]),
                "items": [
                    {"sku": f"SKU{random.randint(1, 5000):05d}",
                     "qty": random.randint(1, 5),
                     "price": round(random.uniform(1.0, 100.0), 2)}
                    for _ in range(random.randint(1, 4))
                ]
            }
            items.append(item)
    return items[:target_items]


# 5️⃣ Batch‑write to DynamoDB (25‑item batches)


def batch_write(items, table):
    for i in range(0, len(items), 25):
        batch = items[i:i+25]
        with table.batch_writer() as bw:
            for it in batch:
                bw.put_item(Item=it)

Key points in the code

Deterministic seed (seed=42) makes the run reproducible – essential for CI.
Hot‑key list is externalized; you can refresh it monthly without touching the generator.
Poisson order count mirrors the observed per‑customer daily variance.
Batch writer respects the 25‑item / 16 MB limit and automatically retries unprocessed items.

4.4 Validation Checks

Check	Method	Pass criteria
Partition‑key cardinality	`SELECT COUNT(DISTINCT customer_id) FROM Orders` (via PartiQL or Scan)	≈ 12 400 distinct keys
Hot‑key share	Aggregate `COUNT(*) GROUP BY customer_id ORDER BY COUNT DESC LIMIT 100`	Top 100 ≈ 45 % of rows
Sort‑key spread per PK	For a sample of 200 keys, compute `MAX(order_ts) - MIN(order_ts)`	≈ 1 day (86 400 000 ms)
Attribute constraints	Scan a random 1 % sample, assert enums, numeric ranges, non‑null fields	0 violations
Item size	`ByteSize(item)` for each generated item	≤ 400 KB (DynamoDB limit)
Write‑capacity simulation	Run a short‑burst write test (e.g., 5 k writes/s) and capture `ThrottledRequests`	< 1 % throttling at provisioned WCU

Automate these checks in a post‑load validation script that fails the CI job if any criterion is missed.

4.5 CI Integration Sketch (GitHub Actions)

name: dynamodb-test-data
on:
  workflow_dispatch:
  schedule:
    - cron: "0 2 * * *"   # nightly refresh


jobs:
  generate-and-load:
    runs-on: ubuntu-latest
    env:
      AWS_REGION: us-east-1
      TABLE_NAME: OrdersTest
    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - name: Install deps
        run: pip install -r requirements.txt
      - name: Generate data
        run: python generate_orders.py --days 7 --out /tmp/orders.json
      - name: Validate locally
        run: python validate_orders.py /tmp/orders.json
      - name: Load into DynamoDB
        run: python load_orders.py --file /tmp/orders.json --table $TABLE_NAME
      - name: Post‑load validation
        run: python post_load_checks.py --table $TABLE_NAME

The workflow is idempotent: the generator always produces the same JSON for a given seed, so re‑running the job simply overwrites the test table.

5. Tool Considerations

Category	Options	When it shines
Schema‑aware generators	`dynamodb-data-generator` (npm), `go-dynamodb-mock`	You already have a TypeScript/Go codebase and want compile‑time safety.
Statistical samplers	`numpy.random.zipf`, `scipy.stats.zipfian`, custom `Zipfian` class	You need a lightweight, dependency‑free sampler (as in the example).
Declarative data specs	YAML/JSON files fed to a generic engine (e.g., `datagen` CLI)	Non‑developers can tweak distributions without touching code.
Free web‑based generator	QA3’s test data generator at `/tools/test-data-generator`	Quick one‑off datasets, prototyping, or teams without a build pipeline.
CI‑native plugins	GitHub Action `aws-dynamodb-batch-write`, GitLab CI `dynamodb-local` service	Zero‑ops local DynamoDB for unit tests; real AWS for integration tests.

Choosing: If your team already maintains a Python test harness, the script above is often the lowest‑maintenance path. For polyglot teams or when you want a UI to explore distributions, the free web generator can produce a starter CSV/JSON that you then feed into your loader.

6. Common Pitfalls & Mitigations

Pitfall	Why it hurts	Mitigation
Using a single UUID for all items	All writes hit one partition → false confidence in capacity.	Enforce a minimum distinct‑key count in validation.
Ignoring sort‑key correlation	GSI queries that filter on sort key appear faster than reality.	Generate PK + SK jointly; validate per‑PK sort‑key range.
Hard‑coding weights	Production traffic shifts; tests become stale.	Pull hot‑key weights from a scheduled profiling job (e.g., Athena query on CloudTrail).
Generating >400 KB items	DynamoDB rejects the write; batch writer silently drops items.	Add a size check in the generator; truncate or split large maps.
Running validation only once	Schema drift goes unnoticed.	Gate every CI run with the post‑load validation step.
Assuming uniform read/write distribution	Autoscaling and adaptive capacity behave differently under skew.	Simulate read‑heavy and write‑heavy workloads separately in load tests.
Not cleaning up test tables	Costs accumulate; leftover data contaminates later runs.	Use a unique table suffix per run (`OrdersTest_<run_id>`) and a teardown step.

7. Checklist – From Profile to CI

8. Next Action

Pick one of the following concrete steps and execute it this week:

Run a quick profiling query against your production DynamoDB (or exported CloudTrail logs) to extract the top 200 partition keys and their request counts. Export the result to hot_keys.csv.
Create a minimal generator (≈ 50 lines) that reads hot_keys.csv, mixes in a Zipfian tail, and writes a 10 k‑item JSON file.
Validate the file with the checklist’s “local validation” row, then load it into a throwaway DynamoDB table (aws dynamodb batch-write-item …).

If you prefer a no‑code start, open the free test data generator at /tools/test-data-generator, choose “DynamoDB” as the target, paste a few hot‑key weights, and download a sample JSON. Feed that file into your existing loader and run the post‑load checks.

Either path gives you a reproducible, realistic dataset that will surface throttling, GSI skew, and capacity‑planning bugs before they hit production.

DynamoDB Test Data Generation with Realistic Partition Keys