Predictive Performance

Result · seed 42 · n = 200

200 of 200 posterior fits succeeded. Predictions evaluated at a held-out timepoint — trough 0.5 h before the next dose — not at the levels used to fit the posterior.

Comparison vs. Bai 2025 a posteriori

Side-by-side with the four-program × real-ICU-patients table from Bai et al. Direct numerical comparison is not valid — we ran synthetic patients with one sampling protocol; Bai ran real patients with mixed sampling. The point of the table is order-of-magnitude reasonability: Vancomyzer’s synthetic stress-test rBias and rRMSE land within the same range as commercial programs running on real ICU data.

Bai 2025 reference: rBias range -16.03% to 0.10%, rRMSE range 27.69% to 37.64%. Source: Bai et al. Ther Drug Monit. 2025;47:594–602, Table 3 (a posteriori).

Methodology

1. Generate n = 200 synthetic ICU patients with demographics matched to Bai 2025 Table 2 (age 61.71 ± 14.78, 63.1% male, weight ~65 kg, SCr ~0.66 mg/dL). The patient set is fixed and reproducible, so the cohort is identical every time the analysis runs.
2. For each patient, sample “true” PK parameters from the Goti 2018 model (CL = 4.5 × (CrCl/120)^0.8 × (WT/70)^0.75 L/h, V₁ = 58.4 × WT/70 L, Q = 6.5 × (WT/70)^0.75 L/h, V₂ = 38.4 × WT/70 L) plus log-normal BSV (ω_CL = 0.4, ω_V1 = 0.3, ω_Q = 0.5, ω_V2 = 0.4).
3. Apply a fixed regimen (15 mg/kg q12h, 1.5 h infusion, rounded to nearest 250 mg, capped 500–3000 mg) and simulate steady-state concentrations from the patient’s Goti truth.
4. Sample a peak (~1.5 h post-infusion-end after dose 4) and a trough (~0.5 h before dose 5). Add combined residual error: proportional 20% CV + additive 1.0 mg/L SD.
5. Feed the two noisy levels into Vancomyzer’s a posteriori Bayesian engine (which uses the Colin 2019 prior — NOT Goti). The engine recovers a posterior {CL, V₁, Q, V₂}.
6. Predict the concentration at a held-out timepoint (trough 0.5 h before the next dose) from Vancomyzer’s posterior. Compare to the patient’s Goti truth at that same timepoint.
7. Aggregate predicted–vs–observed pairs across the cohort. Compute the Sheiner–Beal relative bias (accuracy) and relative root-mean-squared error (precision).

The analysis is fully reproducible: the same fixed patient set produces the same results every time.

Limitations

• Synthetic ≠ real

  Goti+BSV truth captures population-level PK variability but not unmodeled ICU physiology — fluid shifts in sepsis, drug-drug interactions, third spacing, hypoalbuminemia, or assay-batch noise. Real-world rRMSE will be higher; the Bai 2025 27.69–37.64% range is the realistic target.

• Best-case sampling protocol

  We give the engine BOTH a peak and a trough at steady state — the highest-information protocol per ASHP/IDSA 2020. Real practice often has trough-only or a pre-steady-state level, which would degrade both rBias and rRMSE. The harness should be re-run with trough-only and pre-SS variants before any claim about robustness.

• Goti BSV values are literature-typical, not transcribed

  The raw OMEGA matrix from Goti 2018 isn't openly published. We used ω = 0.40 / 0.30 / 0.50 / 0.40 — within the range of published adult vancomycin popPK models. Tightening or widening BSV will move both metrics; the sensitivity has not been formally quantified.

• HD and ARC patients excluded

  Matches Bai's exclusion criteria and Vancomyzer's published scope. Continuous renal replacement, ECMO, and augmented renal clearance subpopulations are NOT covered by this validation. Performance there is unknown.

• Held-out point is one steady-state trough

  Predicting a peak from a peak+trough fit is mechanically easier than predicting a far-future concentration with covariate drift (changing SCr, weight). Multi-point and longitudinal validation is a future-work item.

Validation roadmap