Eugene Yan

Product evaluations for LLM systems require three steps: labeling a small dataset with binary pass/fail labels, aligning LLM evaluators, and running experiments—with at least 50-100 failure cases needed for effective calibration.

“Author of "How to Run Product Evals"”

Semantic IDs enable training unified LLM-recommender hybrid models that can understand both natural language and catalog items natively, allowing steerable recommendations with reasoning and explanations.

“Author of "Semantic IDs for Recommender LLMs"”

“Speaker at AIE Security track on AI security topics.”