Alex Lewis
Senior Software Engineer · Zapier
Summary
Senior software engineer with 10+ years shipping production software across frontend, backend, platform, and automation. Current focus: applied AI - agent infrastructure, LLM evaluation, retrieval, tool use, and developer tooling - building the systems that make LLM-powered products dependable.
Skills
- Applied AI
- AI agents & orchestration · LLM evaluation (LLM-as-judge, ground-truth measurement) · Retrieval-Augmented Generation · Tool calling & MCP · Prompt architecture · AI observability
- Languages
- TypeScript · Python · JavaScript · SQL
- Frameworks & Platforms
- React · Next.js · Node.js · PostgreSQL · AWS · Vercel
- Practices
- CI/CD · API design · Software architecture · Production observability · Serverless
Experience
Senior Software Engineer Zapier Aug 2025 – Present
- Designed and built an internal agentic end-to-end testing platform from zero: agent runtime, CI integration, and a plugin architecture that decoupled the platform from any single application. Benchmarked at ~90% first-run pass rate across 50 runs; packaged for one-command adoption (npx via a semantic-release pipeline) and running in 4 repos.
- Own the evaluation strategy for an LLM-as-judge system: deterministic verdict guards for everything machine-checkable, plus a ground-truth layer that cross-tabs judge verdicts against real merge outcomes - the measurement that exposed systematic judge leniency and drives recalibration.
- Architected an agent-orchestration harness (live permission prompts, egress sandbox, self-supervision, cost metering) and productionized it into a shared platform other engineers contribute to; made agent session transcripts first-class proof artifacts for observability.
- Shipped inline app authentication to all non-enterprise users of an AI assistant, removing the auth handoff from the conversation flow.
- Force multiplier beyond my team: built the org's Claude Code plugin catalog and drove adoption across engineering; diagnosed and fixed platform-level failures in the shared deploy pipeline (Sentry integration for every Next.js service, a cross-service auth failure spanning Vercel Secure Compute and PrivateLink).
Senior Software Engineer Your Money Line Oct 2023 – Aug 2025
- Designed and shipped the production RAG system behind a personal-finance guidance product end-to-end - chunking strategy, prompt architecture, and the evaluation loop that decided which prompts ran in production - in a domain where wrong answers carry real financial consequences.
- Took the company's AI surface beyond chat: an AWS Textract document-AI pipeline for loan-payment processing, a text-to-speech pipeline, and a shipped generative-AI product feature from a serverless LLM-assistant prototype.
- Architected a serverless HRIS data-sync integration (Lambda, API Gateway, Step Functions, SQS FIFO with idempotent dedupe, Aurora) handling both bulk import and near-real-time webhook delta sync.
Senior Software Engineer Outreach Aug 2022 – Aug 2023
- Led the Deal Overview engagement timeline from PRD to production: decomposed the epic, made the build-vs-library call (evaluated Recharts and D3, built custom - the libraries fought the use case), owned data contracts with the backend team, and shipped the visualization with data paging.
- Became the go-to owner of the Activity Feed, a high-traffic surface spanning two API generations: drove cross-team root-cause investigations with backend teams and landed 15+ fixes and features, including GraphQL partial-result support.
- Core contributor to the company-wide login/SSO migration: built the SSO callback flow and led a route-based rendering refactor that unblocked downstream auth work.
Sr. Consultant & DevOps Engineer CleanSlate Technology Group Aug 2018 – Aug 2022
Full role history is on LinkedIn.
Get in touch
Happy to talk about applied-AI and developer-tooling work. Contact me or email [email protected].