Candidate Availability
Required and preferred rules are kept separate and reflect the wording in the original posting.
About the Role
You will build and operate the core infrastructure behind online AI evaluation systems. You will design reliable, low-latency APIs, solve streaming and provider-failure challenges, add enterprise infrastructure capabilities, and create observability tools. You will also contribute across backend platforms and integrate evaluation data and customer benchmarks into products.
Requirements
- Approximately 5+ years of backend engineering experience with meaningful distributed systems, infrastructure, or developer-facing platform experience
- Strong proficiency in Go
- Experience with LLM provider APIs and streaming, token management, rate limits, and model-specific quirks
- Product-oriented API development mindset
- Comfort working with ambiguity in a startup environment
Responsibilities
- Design and implement low-latency, high-reliability APIs for leaderboards, models, and arenas
- Handle SSE and streaming responses across heterogeneous providers, including failure recovery, fallback, and response normalization
- Build rate limiting, authentication, usage metering, cost attribution, audit logging, and SOC 2 compliance capabilities
- Instrument infrastructure with distributed tracing, latency breakdowns, token-level usage tracking, and real-time dashboards
- Integrate with the evaluation platform, Arena data, and customer-specific benchmarks
- Contribute to the backend of Leaderboards and Evals platforms and help unify public and private data architectures
Benefits
- Equity
- Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs