Building at Galileo — AI Evaluation

Pratik Bhavsar

Full Stack AI Engineer · Author of the Mastering GenAI Series

I've done it all across the AI stack: trained models, built evaluation datasets, designed benchmarks the community ships to, and written 100+ articles before distilling it all into 5 books on production GenAI.

25K+

5K+

Substack

100+

Blogs

Books

Models

Benchmarks

Read My Books → Get in touch →

Top AI Developer to Watch · 2023

Agent Leaderboard · 445+ Likes

5 Books Published

Featured in Andrew Ng's Newsletter

Top AI Developer 2023

25K+ LinkedIn Followers

IIT Bombay M.Tech

Latent Space Podcast Guest

100+ Technical Articles

BRAG SoTA Models

Hallucination Index

Maxpool Community Founder

5K+ Substack Subscribers

Agent Leaderboard · 445+ Likes

5 Books Published

Featured in Andrew Ng's Newsletter

Top AI Developer 2023

25K+ LinkedIn Followers

IIT Bombay M.Tech

Latent Space Podcast Guest

100+ Technical Articles

BRAG SoTA Models

Hallucination Index

Maxpool Community Founder

5K+ Substack Subscribers

About

My AI adventures
so far

I'm a full stack AI engineer at Galileo (employee #26, Series A), leading open-source evaluations and developer relations. I built the Agent Leaderboard, an open benchmark for AI agents in real-world tasks, along with the Hallucination Index and the BRAG model series for RAG evaluation.

Before Galileo, I was founding engineer at Enterpret (employee #6, pre-seed) building NLP systems for customer feedback at scale, and a principal data scientist at TaskHuman designing semantic search using transformers. Earlier, I launched end-to-end AI initiatives at Morningstar as their first quantitative research hire.

My RAG evaluation research was featured in Andrew Ng's newsletter, I was a guest on the Latent Space Podcast talking agent evaluation, and I've been named one of the Top AI Developers to Watch in 2023.

              IIT Bombay M.Tech · NIT Surat B.Tech
              Computer Science & Engineering
            

Benchmarks & Open Source

Measuring what matters
in production AI

Benchmarks, datasets, and models built in the open. Because evaluation should never be locked away behind a paywall.

FLAGSHIP LIVE

Agent Leaderboard v2

The enterprise-grade benchmark for AI agents. Tests 30+ LLMs across 500 real-world multi-turn support scenarios spanning banking, healthcare, investment, telecom, and insurance.

Leaderboard → Dataset Code

Live Leaderboard → Dataset on HuggingFace Code on GitHub v1 Blog

Research Index

Hallucination Index

A living index tracking factual reliability and hallucination rates across foundation models. Gives engineering teams the data they need to pick the right model for accuracy-critical workloads.

View on GitHub →

SoTA Models

BRAG: SoTA RAG Models

A series of state-of-the-art open-source LLMs fine-tuned for retrieval-augmented generation. BRAG models outperform much larger models on RAG evaluation benchmarks while staying production-friendly.

Read the Research → GitHub

Experience

From pre-seed to Enterprise,
applying AI at the edge

Galileo

AI Engineer

June 2023 – Present · Employee #26

Current

Leading open-source evaluations and developer relations. Built the Agent Leaderboard (open benchmark for AI agents), the Hallucination Index, and the BRAG model series. Authored 5 books on GenAI published under the Mastering GenAI Series.

Agent EvaluationRAGOpen SourceDeveloper RelationsSeries A

Maxpool

Founder

January 2019 – Present

Founded and grew Maxpool, an open community of AI professionals building production GenAI systems. Built Max, an AI research assistant for the Discord community.

Community BuildingGenAIDiscord

Enterpret

Founding Engineer, NLP Scientist

Feb 2021 – May 2023 · Employee #6 · Pre-seed

Built semantic search, reranking, text generation, MLOps, and NLP pipelines to categorize 50K+ topics of customer feedback.

Semantic SearchFine-tuningMLOpsNLP Pipelines

Jina AI

AI Engineer

Sep 2020 – Dec 2020 · Employee #6 · Seed

Contributed to Jina, the open-source neural search framework powering multi-modal search at scale.

Neural SearchOpen Source

TaskHuman

Principal Data Scientist

Oct 2019 – Sep 2020 · Employee #15 · Seed

Led development of core semantic search using transformers for autocompletion and search. Developed and deployed unsupervised deep learning recommendation engine.

TransformersElasticsearchFAISSAWS

Morningstar

Senior Data Scientist

Sep 2017 – Oct 2019 · 1st hire in quant research

First hire in the new quantitative research team. Led full-stack NLP with deep learning: financial document extraction, sentiment analysis, fund rating with ML, and search/recommendation.

BERTPyTorchElasticsearchFinance NLP

Community & Writing

Building in public,
learning in public

Maxpool Community

Founded in 2019, Maxpool is a thriving community of AI engineers building production systems. Join practitioners from around the world sharing research, tools, and real-world lessons.

maxpool.dev · Join the community →

Writing & Newsletter

I write about AI engineering, evaluation, and the business of AI. My Substack newsletter covers tech and business insights. Deep technical dives live on galileo.ai/blog.

pakodas.substack.com · Newsletter → galileo.ai/blog · Technical deep dives → medium.com/@pratikbhavsar · NLP archive →

Pratik Bhavsar

Guides on Building
Production AI

Eval Engineering

My AI adventures
so far