Essay
Are your eval scores measuring the model or the raters?
A psychometric teardown of an influential judging pattern in AI.
· 17 min read
Read the essayI'm Shawn M. Fraine, MA, a behavioral scientist and founder based in Bangkok. I help teams evaluate AI, build DramaLlama, and offer talks and advisory conversations about psychology, mental health, and human connection.

MA Clinical Psychology600+ research citationsAPA Division 44 award, 2018
A fixed-scope reliability diagnostic for one LLM judge, one rubric and one decision. You receive a clarified rubric, an annotated workbook and a decision memo within 15 business days of an agreed start.
Talks, panels, interviews and one-to-one advisory sessions on psychology, mental health, the brain, human connection and AI evaluation.
See speakingEssay
A psychometric teardown of an influential judging pattern in AI.
· 17 min read
Read the essayProduct
A community app for fans of LGBTQ+ Asian dramas, with 6,000+ users.
Fans of LGBTQ+ Asian dramas follow what is airing, find their next series and talk about it with each other.

I hold an MA in Clinical Psychology, coauthored a published meta-analysis and ran research operations for a psychology lab at Washington University in St. Louis. Since 2023 I have been building Psynergic Digital and DramaLlama from Bangkok. I also write romance fiction.
Tell me what you are evaluating, planning or thinking through. I read every inquiry myself.