Sample deliverable
Model & Backtest Audit — Findings Memo
This is an illustrative sample, not a real client engagement. Every number, name, and result below is fabricated on synthetic data to show the format and standard of a StatGazer findings memo. Real engagement memos are confidential and shared only with the client.
1. Engagement summary
A long-only systematic equity strategy was presented for allocation with a strong reported backtest. We were asked to reproduce the result from raw data and assess whether it would hold out of sample before capital was committed. We reproduced the strategy end to end and found that a material part of the reported performance does not survive point-in-time reconstruction. We recommend remediation before sizing.
2. Scope
- Reproduce the reported backtest from raw, point-in-time data.
- Test for look-ahead, survivorship, and data-alignment errors.
- Assess out-of-sample and regime sensitivity.
- Review the validation methodology and documentation against governance needs.
3. Headline impact
| Metric | Reported | Reproduced |
|---|---|---|
| Sharpe (annualized) | 1.8 | 0.7 |
| Annual return | 14% | 6% |
| Max drawdown | −9% | −21% |
| Hit rate | 58% | 52% |
Most of the gap is attributable to two correctable errors (look-ahead and survivorship); the remainder reflects ordinary out-of-sample decay once those are fixed.
4. Findings
A fundamental feature was joined on report date rather than the as-available date, leaking roughly one quarter of forward information into the signal. Correcting the alignment removes the largest single contribution to reported performance.
The investable universe was built from the current index membership, excluding delisted and acquired names. Reconstructing point-in-time membership lowers returns and materially deepens drawdowns.
After corrections, performance concentrates in low-volatility regimes and degrades in the high-volatility and rates-shock windows tested. The strategy is viable but should be sized and hedged with this dependence understood.
Validation was in-sample only, with no walk-forward evaluation and no written record of assumptions or limitations. Not currently defensible to a risk committee or an auditor.
Transaction-cost and slippage assumptions were reasonable and reproduced cleanly; no adjustment required.
5. Remediation, prioritized
- Fix data alignment — rebuild features on as-available dates; re-run the full backtest.
- Point-in-time universe — reconstruct historical membership including delisted names.
- Walk-forward validation — adopt an out-of-sample protocol and report calibrated uncertainty.
- Governance memo — document methodology, assumptions, and limitations for review.
6. Limitations
This review covers the strategy and data as provided. It is not investment advice and not a guarantee of future results. Conclusions depend on the corrected data reconstruction described above.
7. Deliverables provided with a real engagement
- This findings memo.
- A reproducible notebook regenerating every figure from raw data.
- A validation / model-risk report.
- A prioritized remediation list and, on a build, the corrected model with handoff documentation.
Illustrative sample on synthetic data. © 2026 StatGazer LLC.