Manas Pathak

Experience

Machine Learning Researcher · HUMAIN Lab, UT Austin

Mar 2025 – present

LLM evaluation and reliability with Prof. Leqi Liu, leading to a first-author COLM 2026 paper. Built CUDA-parallel PyTorch and vLLM pipelines that cut evaluation runtime 70% across 100+ models, a distributed GPU job scheduler on AWS that saved ~$10k in cloud costs, Chain-of-Thought hallucination detection for Qwen/Gemma-7B, and React/FastAPI dashboards tracking 50+ distributed jobs.

Teaching Assistant, ECE 422C: Software Design & Implementation II · UT Austin ECE

Aug 2026 – present

Lead discussion sections and office hours for 100 students and grade projects and exams on Java, object-oriented design, and data structures with Prof. Edison Thomaz.

Software Developer Intern · IBM

May 2026 – Aug 2026 · New York, NY

Built a full-stack Python/Go platform, with a web UI and APIs, to evaluate and improve 1,000+ AI agent skills for 10,000+ employees. Designed a content-addressed cache with request coalescing that eliminated 4.8K redundant inference calls and cut latency 34%, cut infrastructure costs 40% by optimizing scheduling for 100+ Dockerized workers on Kubernetes, and wired an observability pipeline over 100K+ agent execution traces.

Undergraduate Department Tutor · ECE 312H, UT Austin

Jan 2026 – May 2026

Mentored 40+ students in C/C++ memory management, recursion, and graph traversal, and led GDB and Valgrind debugging sessions.

Software Engineering Intern · Graph Neural Networks Lab, UT Austin

Dec 2024 – Mar 2025

Built a distributed Go/TCP execution layer that load-balanced GPU workers with sub-5 ms task dispatch and 35% more throughput, and a Python profiler that turned timing logs into a call tree and cut computation time 22%. Also built pipeline orchestration and monitoring with TypeScript, React, and PostgreSQL that halved setup time.

Software Engineering Intern · ModHeader

Jun 2023 – Oct 2023

Full-stack work for a 250k-user browser extension: Svelte UI, Node.js APIs on DynamoDB, and an analytics dashboard in React and TypeScript.

Research

arXiv:2604.11996 published at COLM 2026 first author code ↗

Filtered Reasoning Score: Evaluating Reasoning Quality on a Model's Most-Confident Traces

Manas Pathak, Xingyao Chen, Shuozhe Li, Amy Zhang, Leqi Liu

Accuracy treats every answer the same, but deployed systems act on the outputs a model is most confident about. FRS evaluates reasoning quality on exactly those traces, conditioning the metric on the model's own confidence. The result is a view of reliability that accuracy alone cannot provide: two models with identical scores can behave very differently when you only trust what they are sure of, and FRS makes that difference measurable. Across 9 models and 6 reasoning benchmarks, FRS separates models tied at 63.6% accuracy by 16.5 points, agrees with human raters 78% of the time, and its confidence-conditioned top-K ranking lifts rank correlation from 0.45 to 0.73.

view PDF ↗ Deep Reinforcement Learning · UT Austin first author

Reward Hacking in PPO-Based Code Generation: When Passing Tests Does Not Mean Getting It Right

Manas Pathak

RL with execution feedback is increasingly used to improve code generation, but finite test suites are imperfect proxies for true correctness, which makes this a natural setting for reward hacking. In a controlled PPO study on HumanEval with DeepSeek-Coder-1.3B, I compare binary, partial, and heuristic reward structures across three seeds. Shaped rewards improve visible pass@1 but widen the robustness gap on HumanEval+ by up to 14 pp; pass@k alone overstates semantic competence by 10–18 pp. The work argues that robustness-gap reporting should accompany pass@k when evaluating RL-trained code models.

Projects

Live product

Atlas Research Console

A self-hosted research workspace I built for my lab. Members search papers and notes with BM25 full-text search over a replayable write-ahead log, register experiments, and measure treatment-versus-control lift with Impact Experiments. Live Docs let people edit together while agents propose tracked suggestions. Standalone engineering labs include a Raft key-value store and a Kafka event pipeline.

GoBM25Write-ahead logWebSocketRaftDocker

Open live console ↗ View source ↗

FairPlay Review

Chess fair-play review as selective prediction: streams a 150 MB Lichess slice (299K games), scores Stockfish move evidence with a calibrated boosted model, and fills a fixed-capacity reviewer queue. FastAPI serves risk at 76.8 ms p95; a React console replays positions with an evaluation bar.

PythonStockfishscikit-learnFastAPISQLiteReact

Ride Dispatch Simulator

Real-time ride dispatch with ETA-based matching, surge pricing, and live trip state machines, streamed to a browser dashboard over a WebSocket protocol written from scratch on a native HTTP server. Load-tested at 10,000 drivers and 10,000 requests: 770 req/sec, 0.12 ms p95 dispatch, 99.8% match rate.

Node.jsWebSocketGeospatialCanvas

CyberWise Seniors

Nonprofit I co-founded to protect 100+ seniors from phone, SMS, and email scams: text in a suspicious message and get back an immediate safe-or-not answer. A Spring Boot microservice wired to a BERT classifier returns risk scores in <200 ms, with Twilio webhooks and a React dashboard.

JavaSpring BootBERTTwilioReact

Android Weather Forecast

Java Android client that turns hourly Open-Meteo data into an interactive 7-day forecast with per-day temperature, humidity, wind, and rain charts across multiple cities. A per-city regression model trained on 120+ days of archive data predicts tomorrow's temperature and retrains itself after 7 days.

JavaAndroidRESTMPAndroidChartML

Order Smarter

A Spring Boot orchestrator that scrapes Uber Eats group-cart links, predicts calories with a Python NLP model, and returns structured nutritional insights through a REST API.

JavaSpring BootPythonNLPREST

WanderWise

Full-stack travel planner for building and sharing itineraries. JWT-secured CRUD over Django REST and PostgreSQL, OAuth2 APIs with rate limiting and Swagger docs, and a React + Tailwind front end on the Google Maps API.

ReactDjango RESTPostgreSQLGoogle Maps

Mavs Draft Tool

A scouting tool built for the Dallas Mavericks to filter and compare prospects for the 2025 NBA draft, turning raw player data into rankings the front office can act on.

TypeScriptReactData

Education

The University of Texas at Austin

Expected May 2028

B.S. Electrical & Computer Engineering · GPA 3.96 / 4.00
Technical Component: Software Engineering & Design

Relevant coursework

Distributed Systems Operating Systems Machine Learning Deep Reinforcement Learning Computer Architecture Data Structures & Algorithms Probability & Random Processes Software Design & Implementation I/II

About

I work on AI systems, with a focus on agents. As agents take on longer horizons and more autonomy, the hard problem shifts from making them capable to making them legible: tracing what they did, why they did it, and whether it worked.

That is the thread through my work. My research builds evaluation methods that condition on a model's own confidence, so a score reflects behavior you would actually act on. At IBM I built the evaluation and observability layer for agent workflows, instrumenting how agents select and compose skills across 100K+ execution traces. Capability claims should be verifiable. That is how we trust progress.

Degree: B.S. ECE, May 2028
GPA: 3.96 / 4.00
Honors: Undergraduate Research Fellowship ($1,000) · Tau Beta Pi · Eta Kappa Nu
Stack: Python, C/C++, Java, Rust, Go, TypeScript · PyTorch, CUDA, JAX · Kubernetes, AWS, Terraform