Most ed-tech AI only shows off the wins. We publish the places our AI falls short too — SLP rejection rates, teacher override rates, cases where our own code said "not enough data yet." If you are a skeptic, this is the page we wrote for you.
A non-zero failure rate is usually good news. It means a human is actually in the loop and something is being caught.
Percentage of AI-suggested IEP goal updates that a clinician rejected or edited before applying. A healthy number is non-zero — it means clinicians are reviewing, not rubber-stamping.
Source: clinical_signal_reviews table (Phase 3A — backend job in progress)
How often teachers override or edit AI-proposed session plans, conversation starters, or lesson suggestions. A non-zero number means the AI is proposing, not deciding.
Source: decision_effectiveness_scored event stream
Share of fluency profile reads that returned the explicit "insufficient_data" state. High = conservative (good). Zero would mean we are over-confidently labeling sparse data.
Source: fluency_profile_service.py ProfileStatus counts
Share of pronunciation attempts where transcription confidence was below the threshold or the audio could not be analyzed. A non-zero number is honest — speech is hard.
Source: pronunciation_assessment_service.py telemetry
Percentage of AI-drafted parent digest narratives that the parent or teacher edited before approving. Again, higher is not worse — it means the human in the loop is real.
Source: parent_digest_narratives.narrative_source_edit_history
Share of sessions where an LLM call timed out or returned an error. The product has rule-based fallbacks for pronunciation, fluency, mastery, and standards, so core learning survives.
Source: middleware/outbound_content_filter.py circuit breaker counters
Several metrics show an em-dash rather than a value. That is because the aggregation job that publishes them is still being wired up (Phase 4E of our AI Skeptic roadmap). Rather than estimate, we would rather leave the field blank until we can publish a real number. Check back — we will not backfill with made-up figures.
If you are a researcher, clinician, or ed-tech skeptic and you want to see the raw data behind any of these metrics before they are published, email Corey@cocovox.ai.
The patient tutor every kid deserves.