AI Evaluation & Quality
I design rubrics, test personas and scoring methods that show whether a conversational AI agent is accurate, safe and ready for real callers, and where it breaks when it is not.
Seven weighted dimensions, five critical-failure gates and eight test personas for judging whether a voice agent is ready for real callers. Includes a fully scored example call.
Read the project →About 60 prompt-injection attempts against an AI coach built to protect its instructions. Direct attacks failed; asking it to describe its own job leaked about half its hidden prompt.
Read the project →Thirty prompts run through Claude, ChatGPT and Gemini, scored blind against a weighted rubric, then re-scored by AI judges from two model families. Each judge scored its own family higher. Gemini placed last under every scorer, mostly for inventing contact details and facts.
Read the project →I am Director of Technology Operations at Entratus AI, where I help take voice agents from demo to production. That work is mostly evaluation: writing the definition of done, building caller personas, running test calls and deciding what has to be fixed before an agent goes live.
Before AI, I spent 35 years in broadcast audio, so I listen to agents the way an audio engineer does: latency, interruptions, mispronounced names and the half-second pauses that make callers hang up. Earlier, as Director of Marketing and Technology at Insurance Care Direct, I co-designed a dialer platform and built AI compliance and appointment-setting tools.
AI certifications include OpenAI Technical Practitioner, Foundational Knowledge and Consultative Solutions Practitioner. See all. Projects on this site use public tools and fictional scenarios only, never employer or client data.
Open to AI evaluation, QA and model-behavior roles and projects.