Sugeerth Murugesan

Featured Work

The keystone build runs live in your browser right now — plus a full post-training lab and a self-improving training loop with evaluation built in.

Keystone project · live now

Prompt Studio — zero-cost prompt agents in your browser

Built for people who won't write long prompts: tap once and a complete expert prompt is on your clipboard, ready for ChatGPT, Claude, or Gemini. Type a few words instead and a 7-dimension rubric scores, grades, and rewrites your prompt in real time — then hand it to one of three client-side agents (classify → critique → rewrite → verify) that render live step traces. Everything runs in the visitor's browser: a pure-JS heuristic engine on any device, Hermes-3 / Qwen2.5 on WebGPU (via WebLLM) when you opt in. No server, no API keys, $0 infrastructure, forever — the same thesis as my agent-evaluation work: useful agents need good scaffolding and verification, not expensive infrastructure.

one-click copy $0 infra 7-dimension rubric 3 agents · live traces WebGPU opt-in nothing leaves your machine

Launch the live app → one tap = expert prompt copied · nothing to install · works on phones too

Training · Post-Training · Eval

GPU Training Lab ~25 notebooks · live

~25 runnable notebooks covering the modern post-training stack end to end — DPO, GRPO reasoning, RLHF, LoRA / QLoRA, full fine-tuning, and distributed DeepSpeed training — plus LLM-as-Judge and multi-trace agent evaluation, and multimodal (Qwen2.5-VL) runs. Everything runs on free Colab / Kaggle T4 GPUs.

Live Code

Agentic Training · Evaluation

Self-Learning LLM Training Trainer → Judge → MetaJudge

A self-improving training loop where a hierarchy of agents — Trainer → Evaluator → Judge → MetaJudge — drives, evaluates, and critiques its own runs. The MetaJudge audits the Judge for bias and escalates close calls to a human queue; Hyperband allocates compute across an architecture search while a live dashboard streams round history, verdicts, and traces. nanoGPT training with GRPO and speculative-decoding experiments.

Live Code

Graph Intelligence · Agents

Bay Area Traffic — Live Network Intelligence 56 intersections · live

The Bay Area road network as a live, explorable graph — 56 intersections & the five bridges as nodes, 90 highway segments as edges, with a time-of-day congestion model updating in real time. Computes betweenness centrality to surface the critical interchanges, clusters regions, narrates an automated traffic story, and answers questions (“what if the Bay Bridge closes?”) — a free, in-browser graph-intelligence engine, no API keys.

Live

AI Systems Projects

Production-shaped LLM systems — retrieval, routing, agents, observability, and inference.

RAG Production RAG

Production-grade RAG over private documents: hybrid BM25 + dense retrieval fused via Reciprocal Rank Fusion, cross-encoder reranking, and answers with inline citations traceable to source chunks. Ships prompt-injection detection, confidence-based abstention, and a built-in evaluation suite + monitoring dashboard. FastAPI · ChromaDB · Sentence-Transformers.

Code Overview

Routing LLM Router

Intelligent model routing that scores every model across task match, complexity fit, cost, speed, and context to pick the optimal LLM per query — with Balanced / Max-Quality / Max-Speed / Min-Cost modes across OpenAI and Anthropic models. Runs entirely in the browser, no backend.

Live Code

Agent Observability OMNISCOPE

A multi-agent observability platform that captures the causal DAG of every inter-agent decision (not flat logs), scores quality with a 6-judge LLM panel, predicts failures on the live execution graph, and supports time-travel debugging. Drop-in adapters for LangChain, CrewAI, AutoGen, OpenAI Agents, and Anthropic; judges and embeddings run locally on Ollama.

Live Code

Inference Perf Explorer

An LLM-inference performance explorer: upload model perf sweeps and get a customer go / no-go answer, an engineer sanity-check, and side-by-side comparison — parsed and rendered live in the browser, no backend. React + TypeScript.

Live Code

Multi-Agent Agentic Post-Training Framework

A modular framework where specialized agents coordinate an LLM post-training pipeline — technique selection → training → optimization → evaluation — communicating over a structured message bus so the whole alignment process is observable and debuggable. Apache-2.0, with a Colab + Pages demo.

Live Code

Inference SGLang vs vLLM Playground

A side-by-side LLM-serving benchmark: send the same prompt to SGLang and vLLM, compare outputs, and measure time-to-first-token, total latency, and throughput (tokens/sec) with batch benchmarks and interactive charts. FastAPI backend.

Code

Multi-Agent VoyageAI — Multi-Agent Planner

A supervisor-worker multi-agent system built on LangGraph, LangChain, and Langfuse, where specialized agents (flights, hotels, activities, weather, budget, itinerary) collaborate to produce a plan — with full tracing/observability of the agent graph.

Live Code

Research & Experiments

Hypothesis → experiment → measurement. Each project asks a concrete question and reports what the data actually says.

Eval Suite ARES — Agentic RAG Evaluation Suite

F1 0.378 · +28.7% 55 configs swept

Searches the design space of a retrieval-augmented agent — retrieval depth, BM25 / dense hybrid weighting, prompt style, query decomposition, self-reflection — scoring each config on multi-hop QA (HotpotQA) and reporting the quality-vs-cost Pareto frontier. Across 55 configurations, the winner reached F1 0.378 (+28.7% over baseline) at 0.12 s/question. Headline finding: chain-of-thought prompting drove the entire gain — retrieval tuning barely moved F1, and retrieval recall and answer-F1 were decoupled. Runs fully local on Apple Silicon (MPS).

Code Findings

Alignment DPO from Scratch

Direct Preference Optimization on Qwen2.5-0.5B + Anthropic HH-RLHF (8k preference pairs), β=0.1, LoRA — ~35 min on a free T4 — built to show what the chosen / rejected log-probs actually do during training and what β trades off. Includes a write-up: “I ran DPO. Here's what I saw.”

Live Code

SWE-Bench LocalSWEAgent

6/7 patches $0 API cost

A lightweight SWE-bench agent asking whether a 3B model on a laptop can fix real GitHub issues. Honest result: a Qwen2.5-3B agent produced valid patches for 6/7 reachable issues at $0 API cost — but resolved 0%, consistently targeting the wrong root cause. Establishes a zero-cost baseline and pinpoints exactly where small models break down.

Code

GNNs Graph Neural Networks

GAT 81.8% NMI 4.5×

Trains GCN, GAT, and GraphSAGE on the Cora citation network and shows that GNN-learned embeddings capture community structure 4.5× better than raw features (cluster NMI 0.13 → 0.59); GAT reaches 81.8% test accuracy. Interactive attention visualizations + Colab.

Live Code

Training Orchestration Multimodal Training Sweep

A parallel multimodal-LLM training-sweep orchestrator with watchdog crash / stall recovery (halves the batch size and relaunches), SQLite-backed run + metric tracking, an auto-generated comparison dashboard, and a Claude-written verdict on the best variant — auto-published to GitHub Pages.

Live Code

Distributed Training Multi-GPU Training Viz

Trains the same nanoGPT four ways — single-GPU, DDP, FSDP, and DeepSpeed ZeRO-2 — streaming per-step loss, throughput, memory, and grad-norm to JSONL for side-by-side comparison, with a decision tree and documented multi-GPU gotchas. Reproducible on free Colab / Kaggle T4s.

Code

Agent Infra · Interactive We Gave Agents Memory but No Way to Forget

Most “agent memory” is just append-everything retrieval — and it makes agents worse over time. Why real memory needs forgetting, with a live in-browser simulation and a one-click Colab notebook (20,000-trial Monte-Carlo).

Read & run it Colab

Agent Infra · Interactive We Gave Agents Tools but No Undo Button

A long agent with no rollback doesn’t just fail — it finishes and quietly corrupts your state. The database wisdom (checkpoints, idempotency, compensation) that fixes it — with a live simulation and a Colab notebook (40,000-trial Monte-Carlo).

Read & run it Colab

Eval Study You’re Grading Your AI on the Wrong Test

Why a benchmark win ≠ a model that works — variance, contamination, biased LLM judges, and reliability@k — backed by a 20,000-trial simulation showing how more data on a contaminated test makes you more confidently wrong.

Read

Agent Study Your AI Agent Is a Marathoner, Not a Sprinter

A field-grounded playbook for long-horizon agents — compounding error, context rot, pacing, checkpoints, context resets, and sub-agent relays — backed by a 20,000-trial simulation of how agents fail (and recover) over long runs.

Read

More experiments & live training dashboards on GitHub →

Open Source

Tools others can install and run — plus earlier research software licensed by UC / LBNL.

agent-trace-viz Developer Tool · v0.7.0

A pip-installable, drop-in observability tracer for AI agents: one import records the full run as a tree of spans — plans, decisions, tool calls, sub-agent invocations, memory ops, retries, and the underlying Anthropic / OpenAI calls. Structural capture, not just a timeline.

Code

rag-optimized AI Tooling

An observable, optimized RAG pipeline with a web UI, real span tracing and guardrails, a one-click Render deploy, and a no-API-key demo mode. Includes a LoRA Lab that fine-tunes a tiny adapter in pure NumPy (hand-written gradients) live on free-tier hardware.

Code

Brain Modulyzer Research Software · UC / LBNL licensed

A hierarchical visual exploration technique for fMRI data — the system behind an IEEE TCBB '16 paper. Licensed by the University of California through Lawrence Berkeley National Laboratory.

Code IEEE TCBB '16 Video

ECoG Cluster Flow Research Software · Best Paper ACM-BCB '16

Interactive spatio-temporal cluster analysis of electrocorticography data — Best Paper at ACM-BCB '16, extended in BMC Bioinformatics '17. Licensed by the University of California through Lawrence Berkeley National Laboratory.

Code ACM-BCB '16 BMC '17

Technical Writing

Explaining how modern AI systems behave and how ML algorithms work — the long-form work lives on Medium.

Agent Autonomy · Interactive · D3 Autonomy Is a Dial, Not a Switch. · New & interactive

When should an agent ask, tell, or just act? An expected-loss framework (confidence × stakes × reversibility), a per-task-class trust ratchet, escalation mechanics that prevent rubber-stamping — and a live policy observatory: 400 streaming decisions, draggable thresholds, and a Pareto frontier showing exactly where your policy sits between interruptions and catastrophes.

Visualization × Agents · Interactive Agents Became Systems. Nobody Drew Them. · New & interactive

We debug distributed systems by scrolling JSONL. A five-layer framework for visualization across the agent lifecycle — trace waterfalls, behavior fingerprints, workflow flows, live architecture maps, deployment dashboards — grounded in Shneiderman / Tufte / Munzner, with a live D3 control room: plug the Planner, Memory, and Verifier in and out of a running agent system and watch the consequences flow.

Agent Infra · Interactive Your Agent Isn’t Dumb. Its Tools Are. · New & interactive

Most “agent failures” are interface failures: a tool definition is a prompt the agent re-reads at every decision point. The seven sins of tool design (ambiguous names, kitchen-sink params, unbounded responses, dead-end errors…), a per-tool eval loop — and a live grader that scores any MCP / function-calling JSON on 8 dimensions as you type.

Agent Evaluation · Interactive · D3 Six Agents Finish the Same Task. The Trace Knows Who’s Better. · New & interactive

Six differently-scaffolded agents, one task, same harness — an answer-only judge calls it a six-way tie while the traces blow them apart. A trace-metric framework (recall@K, progress-AUC, loop rate, verification, pass^k), behavioral clustering with kNN drift alarms, and a live D3 fingerprint lab that runs the whole experiment in your browser.

Agent Infra · Interactive Agents Can Sprint. Memory Lets Them Run Marathons. · New & interactive

The sequel to the marathon post: the baton itself. Models sustain 30+ hour tasks but every context window still gets wiped — why agent memory and its deployment is the whole ballgame, via Anthropic’s long-running harness and Nous Research’s self-correcting Hermes — with a live "memory relay" simulation you can steer (failure rate, context size) and the design-review checklist.

Self-Improving Systems · Essay Catching Goodhart in the Act · New & interactive

I let a model improve itself against its own grader and watched the score climb while held-out truth quietly fell. Divergence-onset monitoring caught the gaming within 2–11 generations in 100% of gamed runs — with a live simulation and the "verifier half-life" playbook.

AI Agents · Report Your AI Agent Aced the Test. It Still Can’t Do the Job. · New

Capability soared while 95% of enterprise pilots returned nothing. Why reliability — not intelligence — is the wall for AI agents in 2026, and the operator’s playbook for crossing it.

Multi-Agent · Report More Agents Isn’t Better: The Coordination Tax · New

Two frontier labs reached opposite conclusions on multi-agent systems. The reconciliation — single strong agent by default; multi-agent only for verifiable, parallel breadth — plus the ~15× token tax and the MAST failure taxonomy.

Agentic AI · Report Fifteen Domains, One Wall: The Verification Gap · New

Read agentic AI across fifteen domains at once and the same wall appears in every one. Why the bottleneck was never generation — it’s verification — and where value migrates next: the “Trust Stack.”

Explainability Why Did You Rank That? Explainability for LLM-Powered Retrieve-and-Rank · New

A method-by-method field guide to explaining the four-stage ranking funnel — SPLADE/ColBERT structural readouts, RankingSHAP, activation patching, faithful logit signals, counterfactual stage ablation — and the one rule underneath them all: plausible is not faithful. Fact-checked multi-agent research pass.

Retrieval / RecSys Two Towers, One Index: How Retrieval Models & ANN Search Are Co-Designed · New · interactive

A source-grounded deep dive with a drag-to-explore demo: the two-tower model and approximate nearest neighbor search are one co-designed system, not two. From DSSM and the logQ correction to HNSW, IVF-PQ and ColBERT — and how the exact same machine powers recommendation feeds. Assembled by a fact-checked multi-agent research pass.

Agent Infra We Gave Agents Memory but No Way to Forget · New & interactive

“Agent memory” is usually append-everything retrieval — and it makes agents worse over time. Why real memory needs forgetting, with a live in-browser simulation and a Colab notebook.

Agent Infra We Gave Agents Tools but No Undo Button · New & interactive

Without rollback, long agents don’t just fail — they finish and corrupt state. The database wisdom that fixes it — checkpoints, idempotency, compensation — with a live simulation and a Colab notebook.

Evaluation You’re Grading Your AI on the Wrong Test · New

A benchmark win isn’t a model that works — a playbook on eval variance, contamination, biased LLM judges, and reliability@k, backed by a 20,000-trial simulation of how grading method beats raw model quality.

Agents Your AI Agent Is a Marathoner, Not a Sprinter

Long-horizon agents fail like marathoners, not sprinters — a playbook on compounding error, context rot, pacing, checkpoints, context resets, and sub-agent relays, backed by a 20,000-trial simulation.

Earlier writing · Medium

Explainable AI for Deep Learning Interpretability

Designing deep learning models is hypothesis-driven; interpretability is key to trustworthy models — the Achilles' heel of deep neural networks.

Linear Regression from Scratch Foundations

An attempt to clearly explain the simplest ML algorithm in Python — predicting a linear relationship between inputs X and an output Y.

Neural Networks in Organizations Deep Learning

Neural networks are now ubiquitous — convolutional, recurrent and beyond — with manifold applications across medicine, transportation, and visualization.

Qualitative Methods to Evaluate ML Evaluation

The importance of visual analytics in machine learning, as ML methods play an increasingly influential role in the world.

Impact of Methods in Network Science Network Science

Transportation, brain, and social networks all rely on machinery from graph theory and data visualization.

AI Infrastructure & Experience

Where the models meet production — experience across industry research and the infrastructure that scales AI systems.

Experience

Staff ML Engineer / Scientist — Intel / Intuit

Building and shipping LLM-powered recommendation and agentic-AI systems in production. Also DeepInsight — a real-time ML analytics engine for network understanding (C++, Python, Kubernetes, Kafka, GoLang, TensorFlow, Apache Flink, ML anomaly detection).
Research Engineer — Samba TV

Co-creator of Sambavision — applied ML/CV research shipped into product.
Research Intern — Adobe Research

Interpretable deep learning — DeepCompare, for comparing and debugging deep models (patent + IEEE CG&A).
Graduate Fellow — Lawrence Berkeley National Lab

Computational Research Division — large-scale scientific data + ML.

Education

PhD in Computer Science — UC Davis

Advisors: Bernd Hamann, Gunther Weber, Kristofer Bouchard.
Research Associate — UC Berkeley

In conjunction with Lawrence Berkeley National Laboratory.

Infrastructure & Scale

The systems work behind the models. At Intel / Intuit I help build DeepInsight, a real-time ML analytics engine — streaming pipelines on Kafka and Apache Flink, services in C++ / GoLang / Python on Kubernetes, with ML anomaly detection. The same instinct runs through my open-source work: distributed training (DeepSpeed / FSDP / DDP), inference serving, and agent observability — infrastructure in service of shipping and scaling AI systems.