Category: Uncategorized
-
Let’s evaling
Last Friday I had a shower thought about evals. An eval is a repeatable experiment on an LLM system: fixed inputs, one or more variants under test, and a scoring rule that turns outputs into numbers you can compare. That’s what Claude tells me anyway. evaling can help you if you’ve ever changed a prompt…
-
Hey Amro, what’s on your desk?
I shared a Tweet with a photo of my current work setup and a few folks asked what I use so I thought I’d put together a blog post with all the hardware on my desk. I have conflicting goals in choosing the gear I use to do my work. For instance, I want to…
Amro's Blog 