Arpan Sahu
Software Engineer II @ Microsoft — AI-powered developer tooling & large-scale data infrastructure.
I build intelligent systems that bridge AI models with production engineering workflows — MCP servers, Copilot agent skills and PR-automation over petabyte-scale data platforms — on strong backend fundamentals: Python/Django, distributed systems, Kubernetes and CI/CD.
About
Software Engineer with 5+ years across the spectrum — from fast-paced 0→1 product development to enterprise-scale, distributed, petabyte-scale systems. Currently a Software Engineer II at Microsoft building AI-powered developer platforms: MCP (Model Context Protocol) servers, Copilot agent-skills, and grounded, evaluated agents (PR automation, on-call assistants) over petabyte-scale data infrastructure.
I turn ambiguous requirements into shipped, reliable systems — spanning payments, security/XDR, search, observability, data-serving APIs and AI-agent tooling — owning ambiguous pieces independently when speed matters, and driving alignment across PM, infra, data and engineering when systems need scale.
Underneath the AI work is a strong systems-engineering foundation: distributed systems, Kubernetes and CI/CD. I love owning services end-to-end — schema design and APIs through queues, caching, deployment and observability. Most of my public projects are fully containerized and shipped through Jenkins / GitHub Actions and Kubernetes pipelines on real infrastructure (PostgreSQL, Redis, MinIO, RabbitMQ, Kafka).
Skills & Tech Stack
The tools and technologies I build with day to day.
AI & Developer Tooling
Languages
Backend & APIs
Frontend
Data & Messaging
Cloud & DevOps
Microsoft Data Platform
Observability
Experience
Software Engineer II · Microsoft
AI developer platforms & data-serving APIs over a distributed, petabyte-scale system (IDEAS).
- Built the Always-On MCP (Model Context Protocol) server — a standardized tool layer of 80+ tools (entity search, lineage traversal, Kusto querying, orchestration, incident workflows) — enabling Copilot and other agents to operate petabyte-scale data infrastructure in natural language, with least-privilege access and guarded writes.
- Authored Helix agent-skills (Data Skew Triage, Data Validation) that compose and govern the MCP tools into reusable, evaluated decision workflows — localizing issues, choosing safe remedies, validating correctness, and producing incident/PR-ready evidence.
- Built Pipeline Intelligence, a full-stack observability & governance platform — an interactive dashboard mapping ~108 entities across ~57 services and 8 tiers with live health, right-click blast-radius/impact analysis, ICM integration across 5 teams (triage 15 min → 5 sec), and a RAG Copilot over 80+ indexed docs for natural-language Q&A by non-technical users (replicable across the 105K+ dataset ecosystem).
- Built PIPA, a long-running agentic PR reviewer (hybrid neuro-symbolic: 107 rules + detectors, grounded in a 68K-node lineage graph, LLM for intent/blast-radius) with APAS, a deterministic approval control plane — screening 5,768 PRs across 88 repos in 30 days (~192/day) before approval.
- Built Oncall Pulse, a production dashboard + Teams bot (serverless, Azure Static Web Apps) — 6-sweep incident collection, 4-dimension shift scoring, SLA tracking, and GPT-4o handover summaries; cut on-call prep from 30–60 min to seconds.
- Designed and shipped UDM-compliant data contracts & APIs exposing petabyte-scale datasets to downstream products — schema design, API versioning, backward-compatible migrations, fiscal-year-rollover modeling, and reliability engineering that cut manual recovery 70%.
Software Engineer · Trellix (FireEye · McAfee)
XDR/SOAR SaaS platform — promoted from Associate Software Engineer.
- Built an XDR SaaS platform, migrating customers from per-tenant on-prem EC2 to a centralized Kubernetes architecture with a unified database — enabling scalable, pay-as-you-go services.
- Architected SOAR integrations (Tenable, Azure, VirusTotal, ServiceNow, Trellix/McAfee) handling 3M RPM and 500K concurrent playbook executions; built CI/CD that cut deployment times by 40%.
- Engineered a data-segregation engine over billions of records — moving from DB queries to Redis bit-strings for a ~99.88% memory reduction and 100× faster queries for real-time threat analysis.
- Built a search microservice consolidating three products behind one Elasticsearch-backed interface, improving retrieval speed by 80%.
- Implemented a dynamic, DB-driven rule engine replacing hardcoded detection logic, and built the Trellix IAM microservice for cross-platform authentication.
- Used Generative AI (LangChain, RAG, gRPC) to auto-generate SOAR marketplace apps and playbooks.
Software Engineer · GammaStack
Real-time betting platforms, payments and data pipelines.
- Designed a scalable betting platform for Betfaro supporting 50,000+ concurrent users and $1M+ daily transactions; the New Double Bet model drove a 60% profitability increase and 40% traffic surge.
- Integrated subscription- and order-based payments with crypto and fiat gateways (PayPal, Razorpay).
- Optimized data-processing workflows for KodeKloud by 20% using MongoDB, Flask and AngularJS.
- Reduced bug-resolution time by 15% and release cycles by 30% via automated testing and code-review practices.
Education
B.Tech, Computer Science & Engineering
Lakshmi Narain College of Technology & Excellence · CGPA 8.55
Projects
A selection of production-grade Django/Python/React projects — fully containerized and shipped through Docker, Kubernetes and CI/CD. Full source on GitHub.
Get in touch
I'm always open to interesting problems and conversations. Reach me directly: