Quality is not optional. It's our standard. Free QA tools for testers and developers.

GraphQL Test Data Generation for Queries and Mutations

QTQA3 Team

GraphQL Test Data Generation for Queries and Mutations

Generating realistic test data for GraphQL APIs is a recurring pain point for QA teams. Unlike REST, where each endpoint returns a fixed shape, a GraphQL schema can be queried in countless ways. A single query may request nested objects, fragments, variables, and directives, while a mutation can accept complex input types, enums, and custom scalars. If the test data does not respect the schema’s constraints—nullability, list sizes, enum values, custom validation rules—the test suite will flake or, worse, mask real defects.

This guide walks through a practical, evidence‑based approach to generating test data for both queries and mutations. It covers decision criteria, a repeatable workflow, a worked example, validation checks, common pitfalls, and tool considerations. The goal is to give you a process you can adopt today, not a sales pitch.


1. Why GraphQL Test Data Is Different

AspectRESTGraphQL
Response shapeFixed per endpointDetermined by the client query
Input validationUsually per‑endpointCentralised in the schema (type system, directives)
Nested dataRequires multiple callsSingle request can fetch deep graphs
Variable handlingQuery strings / bodyFirst‑class variables with type checking
Schema evolutionVersioned endpointsBackward‑compatible type additions/deprecations

Because the client decides the shape, a test data generator must understand the schema and honour the specific query or mutation being exercised. A generic JSON generator that only knows “string, number, boolean” will produce payloads that the server rejects or that exercise only a tiny slice of the schema.


2. Decision Criteria for a Test Data Strategy

Before picking a tool or writing scripts, answer the following questions. Your answers shape the workflow and the level of automation you need.

QuestionWhy It MattersTypical Answers
How many distinct queries/mutations are covered?Determines whether you need a one‑off script or a reusable pipeline.< 20 → ad‑hoc scripts; > 20 → schema‑driven generator
Do you need deterministic data for regression?Flaky tests often stem from random values that trigger different code paths.Yes → seedable generators; No → pure randomness acceptable
Are there custom scalars / directives (e.g., DateTime, @constraint)?Generators must know how to produce valid values.Yes → need extensibility points
Is the schema stable or evolving rapidly?Frequent schema changes demand a generator that can re‑sync automatically.Stable → static codegen; Evolving → CI‑integrated introspection
What is the test execution environment?Some generators run only in Node, others in JVM, others as CLI.Node/Jest, Java/TestNG, Python/pytest, CI containers
Do you need to simulate error responses (validation errors, permission denials)?Negative testing requires deliberately invalid data.Yes → generator must support “invalid‑by‑design” modes

If you answer “yes” to three or more of the “schema‑driven” rows, invest in a schema‑aware generator rather than hand‑crafted fixtures.


3. End‑to‑End Workflow

Below is a repeatable workflow that works for both queries and mutations. It can be implemented with a combination of open‑source libraries, a small amount of glue code, and (optionally) the free QA3 test data generator at /tools/test-data-generator for quick prototyping.

1. Introspect the GraphQL schema
2. Define the target operation (query/mutation) + variable shape
3. Generate a *valid* data set that satisfies the operation
4. Optionally generate *invalid* variants for negative testing
5. Persist the data (files, DB, in‑memory) for test consumption
6. Validate generated payloads against the schema (static + runtime)
7. Hook the data into your test runner (Jest, pytest, TestNG, …)
8. Automate regeneration on schema change (CI step)

3.1 Introspection



# Example using Apollo CLI (Node)


npx apollo service:download --endpoint=https://api.example.com/graphql schema.graphql

The resulting schema.graphql (or JSON introspection) is the single source of truth for all downstream steps.

3.2 Operation Selection

Create a manifest (YAML/JSON) that lists each operation you want to test, together with the variable types you intend to exercise.



# operations.yml


operations:
  - name: GetUserProfile
    type: query
    document: |
      query GetUserProfile($id: ID!) {
        user(id: $id) {
          id
          email
          profile { firstName lastName avatarUrl }
          posts(limit: 5) { id title publishedAt }
        }
      }
    variables:
      id: "User:123"


- name: CreatePost
    type: mutation
    document: |
      mutation CreatePost($input: CreatePostInput!) {
        createPost(input: $input) {
          id
          title
          content
          author { id email }
        }
      }
    variables:
      input:
        title: "Sample Title"
        content: "Markdown body"
        tags: ["graphql", "testing"]

3.3 Generation

A schema‑aware generator walks the selection set of the operation and produces values for every leaf field, respecting:

  • Nullability – non‑null fields get a value; nullable fields may be omitted or set to null based on a configurable “density” factor.
  • List sizes – configurable min/max (default 1‑3).
  • Enums – one of the defined values, optionally weighted.
  • Custom scalars – plug‑in functions (e.g., ISO‑8601 for DateTime, UUID for ID).
  • Directives – @skip, @include evaluated against variable values.

Example Generator Snippet (Node + graphql-code-generator)

// generate.ts
import { buildSchema, parse, validate } from 'graphql';
import { generateTestData } from 'graphql-test-data-generator'; // hypothetical lib


const schema = buildSchema(fs.readFileSync('schema.graphql', 'utf8'));
const ops = yaml.load(fs.readFileSync('operations.yml', 'utf8'));


ops.operations.forEach(op => {
  const ast = parse(op.document);
  const data = generateTestData({
    schema,
    operationAST: ast,
    variables: op.variables,
    options: {
      seed: 42,                     // deterministic
      listSize: { min: 1, max: 3 },
      nullableDensity: 0.2,         // 20% chance to include nullable fields
      customScalars: {
        DateTime: () => new Date().toISOString(),
        Email: () => `user${Math.random()}@example.com`,
      },
    },
  });


fs.writeFileSync(`test-data/${op.name}.json`, JSON.stringify(data, null, 2));
});

Running node generate.ts produces a JSON file per operation that can be fed directly to your test runner.

3.4 Negative Variants

For each operation, create a variant spec that flips a single rule:

VariantRule Change
missingRequiredFieldOmit a non‑null field
invalidEnumUse a value not in the enum list
oversizedListExceed max list size
malformedScalarPass "not-a-date" to DateTime

A small script can clone the valid payload and apply the mutation, giving you a ready‑made negative‑test corpus.

3.5 Persistence & Consumption

  • File‑based – simplest; commit generated JSON to the repo (or generate in CI and store as artifacts).
  • Database seed – if your tests run against a real DB, write a seed script that inserts the generated entities.
  • In‑memory mock server – tools like MSW or Apollo MockedProvider can ingest the JSON and serve it without a backend.

3.6 Validation

Two layers of validation catch regressions early:

  1. Static validation – run graphql --validate against the schema for every generated payload.
  2. Runtime validation – execute the operation against a test instance of the API (or a contract‑test mock) and assert that the response shape matches the selection set.


# Static validation example


npx graphql-cli validate --schema schema.graphql --documents test-data/*.json

3.7 CI Integration

Add a pipeline step that:

  1. Pulls the latest schema (or uses a committed snapshot).
  2. Runs the generator.
  3. Executes static validation.
  4. Fails the build if any payload is invalid.


# .github/workflows/graphql-test-data.yml


jobs:
  generate-test-data:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Download schema
        run: npx apollo service:download --endpoint=${{ secrets.GRAPHQL_ENDPOINT }} schema.graphql
      - name: Generate data
        run: node generate.ts
      - name: Validate payloads
        run: npx graphql-cli validate --schema schema.graphql --documents test-data/*.json

4. Worked Example: GetUserProfile Query & CreatePost Mutation

Below is a concrete walk‑through using the manifest from §3.2. The schema excerpt (simplified) is:

type User {
  id: ID!
  email: String!
  profile: Profile
  posts(limit: Int = 10): [Post!]!
}


type Profile {
  firstName: String!
  lastName: String!
  avatarUrl: String @deprecated(reason: "Use avatar")
  avatar: Media
}


type Media {
  url: String!
  width: Int
  height: Int
}


type Post {
  id: ID!
  title: String!
  content: String!
  publishedAt: DateTime
  author: User!
  tags: [String!]!
}


input CreatePostInput {
  title: String!
  content: String!
  tags: [String!]!
}


scalar DateTime

4.1 Generated Valid Payload for GetUserProfile

{
  "query": "query GetUserProfile($id: ID!) { user(id: $id) { id email profile { firstName lastName avatarUrl avatar { url width height } } posts(limit: 5) { id title content publishedAt author { id email } tags } } }",
  "variables": { "id": "User:123" }
}

Generated response stub (what the mock server would return):

{
  "data": {
    "user": {
      "id": "User:123",
      "email": "user0.7341@example.com",
      "profile": {
        "firstName": "Avery",
        "lastName": "Chen",
        "avatarUrl": "https://cdn.example.com/avatars/0.png",
        "avatar": {
          "url": "https://cdn.example.com/avatars/0.png",
          "width": 200,
          "height": 200
        }
      },
      "posts": [
        {
          "id": "Post:987",
          "title": "First post",
          "content": "Hello world",
          "publishedAt": "2024-03-12T14:22:00Z",
          "author": { "id": "User:123", "email": "user0.7341@example.com" },
          "tags": ["graphql", "testing"]
        },
        {
          "id": "Post:988",
          "title": "Second post",
          "content": "Another day",
          "publishedAt": "2024-03-13T09:10:00Z",
          "author": { "id": "User:123", "email": "user0.7341@example.com" },
          "tags": ["automation"]
        }
      ]
    }
  }
}

Notice:

  • avatarUrl is present even though it’s deprecated (generator includes deprecated fields by default; you can toggle this).
  • publishedAt uses the custom DateTime scalar formatter.
  • List posts contains two items (within the 1‑3 default range).

4.2 Generated Valid Payload for CreatePost Mutation

{
  "query": "mutation CreatePost($input: CreatePostInput!) { createPost(input: $input) { id title content author { id email } } }",
  "variables": {
    "input": {
      "title": "Generated Title 42",
      "content": "Generated markdown body",
      "tags": ["graphql", "testing", "ci"]
    }
  }
}

Expected response stub:

{
  "data": {
    "createPost": {
      "id": "Post:1001",
      "title": "Generated Title 42",
      "content": "Generated markdown body",
      "author": { "id": "User:123", "email": "user0.7341@example.com" }
    }
  }
}

4.3 Negative Variant: Missing Required title

{
  "query": "...",
  "variables": {
    "input": {
      "content": "Body without title",
      "tags": ["oops"]
    }
  }
}

Running this against the real API should return a validation error with a message like "Field 'title' is required". Your test asserts the presence of that error and the HTTP status 200 (GraphQL errors are delivered in the response body).


5. Validation Checklist

Use this checklist each time you add or modify an operation in the manifest.

  • Schema sync – Introspection file is up‑to‑date (CI step passes).
  • Operation coverage – Every field in the selection set appears in the generated payload (including nested fragments).
  • Nullability respected – No null on non‑null fields; nullable fields appear with configured density.
  • Enum values – All enum fields use values from the schema definition.
  • Custom scalars – Formatter functions exist for each custom scalar (DateTime, Email, JSON, etc.).
  • List bounds – Generated list lengths fall within min/max config.
  • Directives evaluated – @skip/@include logic matches variable values.
  • Deprecation handling – Decision documented (include / exclude deprecated fields).
  • Static validation – graphql-cli validate passes for every payload.
  • Runtime smoke test – At least one successful execution against a test environment.
  • Negative variants – At least one invalid payload per required field / enum / scalar.
  • Determinism – Same seed produces identical payloads (important for flakiness).
  • Version control – Generated payloads (or the generator script) are committed / archived as CI artifacts.

6. Common Pitfalls & Mitigations

PitfallSymptomMitigation
Generator ignores fragmentsMissing fields in payload, tests pass but production fails.Ensure the generator resolves fragments recursively (most libraries do this if you pass the full AST).
Random data breaks deterministic assertionsTests flake on CI, hard to reproduce locally.Use a fixed seed; store the seed in the manifest.
Custom scalar not implementedTypeError: Cannot read property 'toISOString' of undefined or server returns SCALAR_PARSE_ERROR.Register a formatter for every custom scalar; fallback to a safe default (e.g., ISO‑8601 for any DateTime‑like scalar).
Enum values change in schemaGenerated payload uses stale enum, server rejects.Regenerate on every schema change (CI step) and run static validation.
List size defaults too largePayload exceeds server‑side pagination limits, causing 500.Configure listSize.max to match known server limits (often documented in the schema via @maxLength directives).
Deprecated fields still generatedTests rely on deprecated fields that will be removed.Add a config flag includeDeprecated: false and audit tests for deprecated usage.
Mutation side‑effects not resetSubsequent tests

Read more

Postman Test Data Generation with CSV and JSON Files

A practical guide to “Postman Test Data Generation with CSV and JSON Files,” with worked scenarios, tool considerations, validation checks, and actionable advice for QA teams.

Generate API Test Data Directly from an OpenAPI Specification

A practical guide to “Generate API Test Data Directly from an OpenAPI Specification,” with worked scenarios, tool considerations, validation checks, and actionable advice for QA teams.

JSON Test Data Generator Guide for QA Engineers

A practical guide to “JSON Test Data Generator Guide for QA Engineers,” with worked scenarios, tool considerations, validation checks, and actionable advice for QA teams.