Arpan Sahu

Arpan Sahu

Software Engineer II @ Microsoft — AI-powered developer tooling & large-scale data infrastructure.

I build intelligent systems that bridge AI models with production engineering workflows — MCP servers, Copilot agent skills and PR-automation over petabyte-scale data platforms — on strong backend fundamentals: Python/Django, distributed systems, Kubernetes and CI/CD.

About

Software Engineer with 5+ years across the spectrum — from fast-paced 0→1 product development to enterprise-scale, distributed, petabyte-scale systems. Currently a Software Engineer II at Microsoft building AI-powered developer platforms: MCP (Model Context Protocol) servers, Copilot agent-skills, and grounded, evaluated agents (PR automation, on-call assistants) over petabyte-scale data infrastructure.

I turn ambiguous requirements into shipped, reliable systems — spanning payments, security/XDR, search, observability, data-serving APIs and AI-agent tooling — owning ambiguous pieces independently when speed matters, and driving alignment across PM, infra, data and engineering when systems need scale.

Underneath the AI work is a strong systems-engineering foundation: distributed systems, Kubernetes and CI/CD. I love owning services end-to-end — schema design and APIs through queues, caching, deployment and observability. Most of my public projects are fully containerized and shipped through Jenkins / GitHub Actions and Kubernetes pipelines on real infrastructure (PostgreSQL, Redis, MinIO, RabbitMQ, Kafka).

Skills & Tech Stack

The tools and technologies I build with day to day.

AI & Developer Tooling

Model Context Protocol (MCP)AI Agent DevelopmentGitHub Copilot Extensions LangChainRAGPrompt EngineeringAI Evaluation Model Behavior Analysis

Languages

PythonTypeScriptJavaScriptJavaGo CC++SQL

Backend & APIs

DjangoDjango REST FrameworkFastAPIFlask Spring BootNode.jsGraphQLgRPC WebSocketsCelery

Frontend

ReactReduxMUIBootstrapHTML CSSjQuery

Data & Messaging

PostgreSQLMySQLMongoDBRedisApache Kafka RabbitMQElasticsearchPandasScikit-Learn

Cloud & DevOps

DockerKubernetesJenkinsGitHub ActionsAWS RancherHarborMinIONginxUbuntu

Microsoft Data Platform

Azure Data FactoryKusto (ADX)Microsoft FabricOneLake SCOPE / CosmosAirflowRedshift

Observability

SentryDatadogPrometheusGrafana OpenTelemetryJaeger

Experience

Mar 2025 — Present · Noida, India

Software Engineer II · Microsoft

AI developer platforms & data-serving APIs over a distributed, petabyte-scale system (IDEAS).

  • Built the Always-On MCP (Model Context Protocol) server — a standardized tool layer of 80+ tools (entity search, lineage traversal, Kusto querying, orchestration, incident workflows) — enabling Copilot and other agents to operate petabyte-scale data infrastructure in natural language, with least-privilege access and guarded writes.
  • Authored Helix agent-skills (Data Skew Triage, Data Validation) that compose and govern the MCP tools into reusable, evaluated decision workflows — localizing issues, choosing safe remedies, validating correctness, and producing incident/PR-ready evidence.
  • Built Pipeline Intelligence, a full-stack observability & governance platform — an interactive dashboard mapping ~108 entities across ~57 services and 8 tiers with live health, right-click blast-radius/impact analysis, ICM integration across 5 teams (triage 15 min → 5 sec), and a RAG Copilot over 80+ indexed docs for natural-language Q&A by non-technical users (replicable across the 105K+ dataset ecosystem).
  • Built PIPA, a long-running agentic PR reviewer (hybrid neuro-symbolic: 107 rules + detectors, grounded in a 68K-node lineage graph, LLM for intent/blast-radius) with APAS, a deterministic approval control plane — screening 5,768 PRs across 88 repos in 30 days (~192/day) before approval.
  • Built Oncall Pulse, a production dashboard + Teams bot (serverless, Azure Static Web Apps) — 6-sweep incident collection, 4-dimension shift scoring, SLA tracking, and GPT-4o handover summaries; cut on-call prep from 30–60 min to seconds.
  • Designed and shipped UDM-compliant data contracts & APIs exposing petabyte-scale datasets to downstream products — schema design, API versioning, backward-compatible migrations, fiscal-year-rollover modeling, and reliability engineering that cut manual recovery 70%.
MCPAI AgentsRAGAgent SkillsTypeScriptPython AzureKusto (ADX)APIsDistributed SystemsLLMs
Jul 2022 – Mar 2025 · Bengaluru, India

Software Engineer · Trellix (FireEye · McAfee)

XDR/SOAR SaaS platform — promoted from Associate Software Engineer.

  • Built an XDR SaaS platform, migrating customers from per-tenant on-prem EC2 to a centralized Kubernetes architecture with a unified database — enabling scalable, pay-as-you-go services.
  • Architected SOAR integrations (Tenable, Azure, VirusTotal, ServiceNow, Trellix/McAfee) handling 3M RPM and 500K concurrent playbook executions; built CI/CD that cut deployment times by 40%.
  • Engineered a data-segregation engine over billions of records — moving from DB queries to Redis bit-strings for a ~99.88% memory reduction and 100× faster queries for real-time threat analysis.
  • Built a search microservice consolidating three products behind one Elasticsearch-backed interface, improving retrieval speed by 80%.
  • Implemented a dynamic, DB-driven rule engine replacing hardcoded detection logic, and built the Trellix IAM microservice for cross-platform authentication.
  • Used Generative AI (LangChain, RAG, gRPC) to auto-generate SOAR marketplace apps and playbooks.
KubernetesKafkaElasticsearchRedisPython MicroservicesgRPCLangChainRAGCI/CD
Feb 2021 – Jun 2022 · Indore, India

Software Engineer · GammaStack

Real-time betting platforms, payments and data pipelines.

  • Designed a scalable betting platform for Betfaro supporting 50,000+ concurrent users and $1M+ daily transactions; the New Double Bet model drove a 60% profitability increase and 40% traffic surge.
  • Integrated subscription- and order-based payments with crypto and fiat gateways (PayPal, Razorpay).
  • Optimized data-processing workflows for KodeKloud by 20% using MongoDB, Flask and AngularJS.
  • Reduced bug-resolution time by 15% and release cycles by 30% via automated testing and code-review practices.
PythonDjangoFlaskMongoDBAngularJS PayPalRazorpayPayments

Education

Aug 2017 – Jul 2021 · Bhopal, India

B.Tech, Computer Science & Engineering

Lakshmi Narain College of Technology & Excellence · CGPA 8.55

Projects

A selection of production-grade Django/Python/React projects — fully containerized and shipped through Docker, Kubernetes and CI/CD. Full source on GitHub.

Loading projects…

Get in touch

I'm always open to interesting problems and conversations. Reach me directly: