Synthetic Data

The Tuva Synthetic Data Generator creates deterministic, wholly synthetic longitudinal claims data shaped like public Aetna, Priority Health, and Health Care Cost Institute (HCCI) file layouts. It is intended for connector development, source profiling, mapping practice, and controlled evaluation of source-to-Tuva workflows.

The generated records contain no real patient data or payer claims extracts. Payer names describe the public file structures being modeled; the datasets are independent testing artifacts and are not official payer datasets or endorsed by the source organizations.

This generator produces source-shaped files for connector development. It is separate from the small and large synthetic Input Layer snapshots loaded by Tuva Core's integration project; generating files here does not publish or replace those data assets.

Available datasets

Dataset	Generated files	Published fields represented
Aetna	Medical/dental, pharmacy, and medical eligibility	178 + 71 + 63
Priority Health	Medical claims, pharmacy claims, and eligibility	163 + 105 + 82
HCCI 2.0	Enrollment, inpatient, outpatient, physician, and pharmacy claims	17 + 76 + 46 + 42 + 20

Each payer profile starts from an independent canonical population containing households, enrollment histories, demographics, chronic conditions, providers, facilities, professional and institutional utilization, multi-line claims, plan design, network status, adjudicated amounts, and prescription fills. The payer adapter then projects those records into exact checked-in column order.

Install

Python 3.11 or newer is required.

git clone https://github.com/tuva-health/synthetic-data-generator.git
cd synthetic-data-generator
python3 -m venv .venv
.venv/bin/python -m pip install -e '.[dev]'
.venv/bin/python -m pytest

The repository has no runtime dependency outside the Python standard library. The optional dev extra installs the test runner.

Generate data

Generate the default independent 10,000-member, two-year cohort for all three profiles:

scripts/generate

Generate a smaller single-payer fixture:

scripts/generate --payer aetna --members 1000

Generate a clean HCCI control without deliberate source-data issues:

scripts/generate --payer hcci --members 1000 --issue-profile none

List all options:

scripts/generate --help

The main options are:

Option	Purpose
--payer	Generate all, aetna, priority_health, or hcci
--members	Set the number of members per selected payer
--start-date, --end-date	Set the longitudinal observation period
--seed	Select a different deterministic population
--issue-profile	Use connector_eval or the clean none control
--output-dir	Change the payer-shaped CSV destination
--reports-dir	Change the private validation-artifact destination

Deterministic output

The default seed is 20260829, the default period is January 1, 2024 through December 31, 2025, and the default profile is connector_eval. The same Git revision and arguments reproduce the same members, identifiers, claims, amounts, issue locations, reports, and gzip bytes—even when Python hash randomization changes.

Use a different seed to create a different but still reproducible corpus:

scripts/generate --seed 314159

Changing the seed, member count, date range, issue profile, or generator revision intentionally changes the output.

Output files

The generator writes ordinary gzip-compressed UTF-8 CSV files with header rows:

data/generated/<payer>/*.csv.gz

The matching files under schemas/<payer>/ define exact column order, source type and length metadata, source positions where published, and Tuva-authored mapping decisions. Follow the source links in the repository for the publishers' field descriptions.

Validation artifacts are written separately and are ignored by Git:

reports/generated/<payer>/validation_report.json
reports/generated/<payer>/issue_manifest.csv
reports/generated/run_report.json

The issue manifest is evaluator-only answer material. Do not put it in the raw source location, a connector repository, or an evaluated agent's context.

Data-quality evaluation profiles

connector_eval, the default, introduces sparse and patterned problems rather than uniformly corrupting columns. Broad scenarios include clustered missing or plausible-looking invalid codes, multiple valid-looking DRGs on one claim, claim-level inconsistencies, subtle temporal and one-cent financial anomalies, enrollment conflicts, demographic drift, and transaction-history challenges.

Aetna and Priority Health include medical replacement and void histories where their public layouts expose defensible lineage. HCCI instead includes a source row replay challenge because its public layout does not expose equivalent adjustment status and lineage. The generator does not force the same issue into a source layout that cannot preserve its meaning.

Because the generator code is public, blind evaluation is a procedural control: isolate the connector-building agent from the generator repository and private reports, and keep the selected seed undisclosed. A new seed hides affected row keys but does not conceal the published issue classes.

Use --issue-profile none as a clean diagnostic control when distinguishing a connector defect from an intentional source-data problem.

Connector-development workflow

1. Generate one payer profile and retain its report and issue manifest outside the connector workspace.
2. Give the connector builder only the payer-shaped files or raw relations, public source dictionaries, and Tuva Input Layer contracts.
3. Profile relations, infer keys and grains, document a mapping plan, and build the connector.
4. Run Structural Data Quality before interpreting Logical Data Quality results.
5. After the run, compare final-action rows and Data Quality findings with the private issue manifest.

Generated NPIs are checksum-valid synthetic identifiers and deliberately do not identify NPPES providers. Exclude NPPES-derived provider-reference results from connector scoring; they are a synthetic-identity boundary, not an injected connector defect.

Warehouse-loading boundary

The public repository only generates files. It contains no warehouse client, loader, credentials, account identifiers, role names, or environment-specific ingestion instructions. If database relations are useful for a connector exercise, load only the payer-shaped CSV files through your organization's normal governed ingestion process and keep the private reports separate.

Limitations

• The datasets are for software testing, not clinical care, actuarial pricing, payment policy, network analysis, or payer benchmarking.
• Synthetic distributions are plausible test distributions, not estimates of payer performance or production populations.
• Public dictionaries omit proprietary value sets, adjudication rules, and masking logic; the generator does not attempt to recreate them.
• Passing these fixtures is useful connector evidence, not proof that a connector is ready for every production feed.

For implementation details and source provenance, see the repository's README, methodology, and source references.