foretop telltale — OpenAPI telemetry coverage
See which API failures have no telemetry.
Telltale compares an OpenAPI contract with the Prometheus alerts, OpenTelemetry Collector configuration, and Grafana dashboards a service actually ships.
Available now · Apache-2.0
What it checks
A bounded scanner with explicit evidence and an honest unknown state.
Map the documented surface
Autodetect an OpenAPI or Swagger document and enumerate every route plus its documented 4xx and 5xx responses.
Read the signals in the repository
Inspect Prometheus alert rules, OTel Collector pipelines and Grafana queries, and detect OpenTelemetry HTTP auto-instrumentation registered in the code itself — all without contacting a running service.
Keep uncertainty visible
Report route- and failure-mode coverage with source evidence; ambiguous matches remain ambiguous rather than becoming clear.
Use it locally or as a CI gate
Start report-only
The default check is report-only. Add --base origin/main to fail only when coverage that existed at the merge base has regressed—never on an absolute coverage target.
One result, several renderers
Choose table, Markdown, JSON, SARIF, annotations, and triage JSON without changing what the scan concluded.
Annotate the pull request
The public repository includes a composite GitHub Action that emits annotations, maintains one summary comment, and preserves the CLI's exit code.
Trust boundary
What leaves the machine?
Contract and telemetry analysis are local. Normal checks do not upload repository content or require an account; hosted reporting is explicit and sends finding metadata, never source bodies.
What is outside today's boundary?
- Committed configuration is evidence of intended coverage, not proof that telemetry arrived at runtime.
- A repository that registers HTTP auto-instrumentation in code has its trace coverage reported as unknown rather than absent — the routes are traced, but nothing here proves the instrumented application is the one the contract describes. Alert, metric and log absence is still reported as absence.
- OpenAPI references are resolved within the local document; remote references are not fetched.
- Conservative matching leaves uncertain evidence ambiguous.
Run it on yours
Install the published package, scan the current directory, and inspect every result before deciding whether to gate CI.