Verification companion · private repository

The code behind Vantage, open for reading.

Vantage is an internal platform, so its repository can't be shared publicly. This page stands in for it — the architecture, and six verbatim excerpts from the codebase, so the engineering can be judged on its merits rather than a résumé line. Every block is copied unchanged from the file named above it, trimmed only where marked # ….

120k
lines of Python
206 modules
~20
tools behind
one auth layer
~5,700
automated
tests
7
-check CI gate
on every change
01 · What Vantage is

Vantage is the operations hub for a health-insurance data-operations team at Ideon. It consolidates roughly twenty tools — plan-benefit validation, carrier-rate reconciliation against live carrier APIs, CMS template validation, and operational reporting — behind one AWS-Cognito-authenticated Plotly Dash application.

The stack is Python, Plotly Dash 3.4 on Flask, and Gunicorn, over AWS — Cognito, Athena, S3, Elastic Beanstalk, and Cost Explorer — with Salesforce and Jira integrations. It runs on a single 2 GB Elastic Beanstalk instance, and much of the engineering that follows exists to stay correct and fast inside that constraint.

Vantage runs no AI at runtime — the validation is deterministic SQL and regex, which is faster, cheaper, and fully auditable. The AI is in how it's built: I develop this ~120k-line codebase with heavy AI-assisted engineering (agentic coding tools, spec-driven refactors), while every change still clears the seven-check CI gate and the ~5,700-test suite.

02 · Architecture at a glance

Every concern lives in exactly one place, and the boundaries are enforced by both convention and tests.

app-code/app/ — layered separation
app-code/app/
├── layouts/     UI structure only — 38 pages
├── callbacks/   read inputs → call ONE service/util → render — 31 modules
├── services/    multi-step orchestration across S3 + acks + metrics — 19
├── store/       persistence boundaries — all ack/flag I/O funnels here
├── utils/       pure, I/O-free logic + typed boundary wrappers — 53
├── queries/     100% of the Athena SQL, batch_1..batch_7 + named — 17
├── auth/        Cognito OAuth 2.0 + PKCE, JWT/JWKS, HMAC CSRF state
├── site_config.py   single source of truth for every value that changes
└── app.py       Flask server, auth middleware, rate limiting, routing

Three-layer access control

A tool is only “added” when the sidebar link, the route gate, and the ⌘K command palette all gate on the same Cognito group — verified by guard tests so the three layers can't silently disagree.

Single-source config

Plan year, table names, thresholds, and quarter labels live only in site_config.py. A new plan year is a one-file edit that ripples through every query, label, and cache key.

Memory-first deployment

One Gunicorn worker with 8 threads — one process to stay within 2 GB, threads so an in-process heavy run can't starve the status-poll requests into 504s.

03 · Code you can read

Six excerpts, each chosen to show a real engineering decision rather than boilerplate.

3.1

Stateless, self-verifying CSRF state for OAuth

The OAuth state parameter has to survive a round-trip to Cognito with no server-side session to check it against — so it's an HMAC-signed token with the timestamp inside the signed payload. A stale or forged token is rejected on the callback with no shared state.

auth/cognito_utils.py
def make_state(self, secret_key: str) -> str:
    """Format: <nonce>~<unix_ts>.<HMAC-SHA256("<nonce>~<ts>", secret_key)>

    The timestamp is part of the signed payload, so it cannot be forged
    independently — verify_state() can therefore reject stale tokens."""
    nonce = secrets.token_urlsafe(24)
    payload = f"{nonce}~{int(time.time())}"
    mac = hmac.new(secret_key.encode(), payload.encode(), hashlib.sha256).hexdigest()
    return f"{payload}.{mac}"

def verify_state(self, state: str, secret_key: str, max_age_seconds: int | None = None) -> bool:
    try:
        payload, mac = state.rsplit(".", 1)
        expected = hmac.new(secret_key.encode(), payload.encode(), hashlib.sha256).hexdigest()
        if not hmac.compare_digest(expected, mac):          # constant-time compare
            return False
        if max_age_seconds is not None:
            if "~" not in payload:                          # reject legacy tokens w/o a timestamp
                return False
            _nonce, ts_str = payload.rsplit("~", 1)
            if time.time() - int(ts_str) > max_age_seconds:  # reject stale/replayed tokens
                return False
        return True
    except Exception:
        return False

Why it's notable Constant-time comparison (hmac.compare_digest), an expiry baked into the signature so it can't be tampered independently, and backward-compatibility for pre-fix tokens. Paired with PKCE (S256) and per-request JWT/JWKS validation elsewhere in the file.

3.2

Race-free, self-healing concurrency guard

Heavy validation runs execute in unbounded background subprocesses; on a 2 GB box, two at once OOM-killed the worker. The fix is a cross-process test-and-set that rejects a second run rather than queuing it — and reclaims the slot if the holder died without releasing it.

utils/run_guard.py
mine = {"pid": os.getpid(), "name": func.__name__}
acquired = bool(cache.add(_LOCK_KEY, mine, expire=_LOCK_TTL_S))   # atomic: writes only if absent
if not acquired:
    # Slot busy — but is the holder still alive? A holder that died without
    # releasing (OOM/SIGKILL) must not wedge the slot; reclaim it.
    holder = cache.get(_LOCK_KEY) or {}
    hpid = holder.get("pid") if isinstance(holder, dict) else None
    if hpid is not None and not _pid_alive(hpid):
        cache.set(_LOCK_KEY, mine, expire=_LOCK_TTL_S)
        acquired = True

Why it's notable diskcache.Cache.add is an atomic compare-and-set across processes, not a racy get-then-set. PID-liveness reclaim is the primary recovery; the TTL is only a last-resort backstop. And the whole guard fails open — if the lock store itself errors, the run proceeds, because a broken lock must never block real work.

3.3

Push the aggregation into SQL

The raw plan × service-area × ZIP join is ~37 million rows nationally. Paging that to the app OOM-killed the worker. Athena collapses it to a few thousand rows with array_agg before the data ever leaves the warehouse — and packs the county's display name alongside its FIPS code so no second lookup is needed.

queries/service_area_queries.py
SELECT
    vpp.issuer_id                                                 AS issuer_id,
    vpp.service_area_external_id                                  AS service_area_external_id,
    mbs.audience                                                  AS audience,
    array_agg(DISTINCT concat(zc.fips_code, '|', zc.county_name)) AS counties,
    array_agg(DISTINCT zc.zip_code)                              AS zips,
    array_agg(DISTINCT zc.state_code)                            AS states,
    array_agg(DISTINCT mbs.hios_id)                             AS hios_ids
-- … joins …
GROUP BY vpp.issuer_id, vpp.service_area_external_id, mbs.audience

Why it's notable A four-order-of-magnitude data-transfer reduction, achieved by moving the work to the engine built for it — plus a small schema trick (fips|county_name) that avoids a round-trip.

3.4

A pure, testable aggregation core

Business logic is kept pure and I/O-free so it's exhaustively unit-testable without mocking AWS. This folds one change event into a running aggregate behind WREN's change analytics — including a domain rule that excludes “seeded” values (a change that merely fills a previously-empty field) from every metric.

utils/wren_analytics.py
def apply_event(agg: dict, ev: dict) -> None:
    """Fold one approval event into ``agg`` in place. Blank-fills are tallied but not counted."""
    if is_blank_fill(ev):
        agg["excluded_blank_fills"] = int(agg.get("excluded_blank_fills", 0)) + 1
        return

    agg["counted"] = int(agg.get("counted", 0)) + 1
    hios = str(ev.get("hios_id") or "").strip()
    if hios:
        agg["plans"][hios] = agg["plans"].get(hios, 0) + 1
        if iss := issuer_prefix(hios):
            agg["by_carrier"][iss] = agg["by_carrier"].get(iss, 0) + 1
    if fld := str(ev.get("field") or "").strip():
        agg["by_field"][fld] = agg["by_field"].get(fld, 0) + 1
    # … by_approver, by_day, per-run rollups …

Why it's notable The same pure core serves both the live incremental path and a full rebuild, so “computed live” and “recomputed from history” provably agree — a property asserted directly in the test suite — backed by a durable, day-partitioned S3 archive so the full record is retained even though the in-app view is capped.

3.5

One config edit rolls the platform forward

Seasonal values live in one file. This helper returns a carrier API's request date and encodes a real regulatory constraint — a past quarter can't be quoted with a past effective date — while keeping the true quarter start for scoping. Pure, with an injectable clock, so it's fully tested.

site_config.py
def uhc_post_effective_date(quarter, year=None, today=None) -> str:
    """The requestEffectiveDate to send UHC — a date the API will accept.

    UHC rejects a past effective date, so a past quarter is bumped to the earliest 1st/15th of a
    month still inside the quarter that hasn't passed yet; a current/future quarter is unchanged."""
    eff = uhc_quarter_effective_date(quarter, year)
    if not eff:
        return ""
    today_iso = (today or date.today()).isoformat()
    if eff >= today_iso:                      # current/future quarter -> unchanged
        return eff
    y = str(year) if year is not None else PLAN_YEAR
    candidates = [f"{y}-{mm}-{dd}" for mm in _UHC_QUARTER_MONTHS.get(quarter, ()) for dd in ("01", "15")]
    for cand in candidates:                   # earliest in-quarter 1st/15th that hasn't passed
        if cand >= today_iso:
            return cand
    return eff                                # quarter fully past -> best-effort true start

Why it's notable Domain rules that would otherwise scatter as magic strings are named, centralized, pure, and unit-tested. A clock is injected rather than called, so time-dependent logic is deterministic in tests.

3.6

A query wrapper that governs cost and won't hang the worker

Every Athena call goes through one wrapper that tags queries for cost attribution, sets generous client timeouts with retries, and accepts a max_rows abort guard and a progress_callback so a long query keeps the status-poll requests alive instead of triggering a 504.

utils/athena_utils.py
def execute_athena_query(
    query: str,
    timeout_minutes: int | None = 5,
    progress_callback: Callable[[], None] | None = None,
    max_rows: int | None = None,          # abort the fetch if the result exceeds this many rows
) -> pd.DataFrame:
    tagged_query = f"{_BW_TAG}\n{query}"  # /* source=vantage-bw */ — CloudWatch cost attribution
    athena_client = boto3.client(
        "athena",
        config=BotocoreConfig(connect_timeout=900, read_timeout=900, retries={"max_attempts": 3}),
    )
    # … start_query_execution(WorkGroup=…), poll to completion honoring timeout_seconds …

Why it's notable A single choke point makes cost attribution, timeout policy, retry policy, and the OOM abort guard uniform across every tool that touches the warehouse. No tool re-implements query plumbing.

04 · Quantifiable outcomes, per tool

Two honest categories. Code-grounded metrics are facts about what the system demonstrably does — verifiable in the repo. Operational impact depends on the team's real production data, so those cells name a metric to measure rather than an invented figure.

■ verifiable in-repo■ to quantify from ops data
ToolCode-grounded metricImpact to quantify
BenefitWatch130+ validation checks across 7 SQL batches, retargetable to any plan year with no per-query editsplans QA'd per run; issues caught pre-publish vs. manual review
QHP Validation~90 checks across 4 CMS template families; a column-reorder-resilient county cross-checkSERFF/HIOS rejections avoided per filing season
UHC / OxfordAge-banded rate-vector reconciliation per rating area; SAM parity over a ~144 MB uploadquotable-but-missing plans surfaced per quarter
Rate ValidationMarket-wide checks reusing the BenefitWatch engine with zero duplicated callbacksmispriced / rolled-over rating areas flagged per run
Service Area Validation~37M-row join reduced to thousands via SQL array_agg; Jaccard YoY matchingcarrier service-area shrinkage caught before it ships
WRENAthena plan-diff over 600+ carriers with anomaly detection + flag categorization, so changes are filterable by type / severity / carrier; durable audit archivechanges triaged per run; review time per changed plan
WREN AnalyticsAll-time aggregate that never truncates; exact “avg changes per plan / carrier / run”trend in change volume / rework over the season
Publishing dashboardQ1 view fans out a 15-query set via a thread pool; in-SQL market-exit exclusioncompletion-rate accuracy vs. the prior spreadsheet
Build Season ReportPer-ticket 0–100 credit score from Jira changelogs; per-type correction benchmarksthroughput / correction-rate trend across the season
Processor Report Card4-component weighted score + radar, scored against like-for-like team baselinesper-processor quality trend; coaching targets
ARIADeterministic (no-LLM) SBC parsing; re-parses only when a PDF's LastModified changesSBC-vs-database mismatches caught per scan
Acquisition Sheet3-layer state model + stable-ID Confluence deep-linkstracking hours saved vs. the shared spreadsheet
Platform~5,700 tests; 7-check CI; one-file plan-year rollover; single-worker survival on 2 GBprod regressions avoided per release
05 · How to verify further