Balaji Hariharan
I build production AI agent systems that enterprises trust
From multi-agent orchestration and LLM gateways to governance frameworks and MCP protocol servers — I design, build, and ship the complex AI infrastructure that moves agentic AI from prototype to production.
About
I'm an AI Solutions Architect and LLM Infrastructure Engineer who builds the systems that make agentic AI work in production — not demos, not wrappers, but the governance, security, cost controls, and orchestration layers that enterprises need before trusting autonomous agents with real workflows.
As the creator of ShackleAI, I designed and built an 11-microservice agent platform from scratch: an LLM gateway with multi-provider routing and PII scrubbing, a credential vault with AES-256 encryption and auto-rotation, RBAC+ABAC governance with kill-switches, and an orchestrator that decomposes natural language into executable multi-agent pipelines using CrewAI and OpenClaw.
My open-source contributions include an MCP memory server (1,853+ npm downloads) and a Python async agent framework on PyPI with 2,000+ tests at 97% coverage, per-call cost tracking across 100+ LLM providers, and 37 built-in tools.
I developed a rapid AI-accelerated development methodology — automated GitHub issue management, AI-driven sprint execution, and a 25-agent AI ecosystem that ships production code at lightning speed: 18 iterations in 3 days, 1,359+ commits across 7 repositories, 7 PyPI releases. This isn't just building AI — it's using AI to build AI, faster than any traditional team.
18+ Years Before AI Agents
From RPA to AI — I led the first wave of enterprise automation as Managing Director at Softomotive (acquired by Microsoft, became Power Automate). Now I'm building the second wave: autonomous AI agents with governance, security, and enterprise trust built in.
Enterprise Clients
Domain Expertise
Projects
Production systems I have built — from AI agent platforms to open-source developer tools.
ShackleAI Platform
The Operating System for AI Agents — 11 microservices, 142 API routes, 539+ issues closed, 241+ PRs merged, 1,971 CI/CD workflow runs. LLM Gateway, Credential Vault, Agent Governance, Orchestrator, and MCP-native tool infrastructure.
@shackleai/memory-mcp
Open-source persistent memory for AI coding tools. The first MCP-native memory server with semantic search, offline embeddings, and zero-config setup. Used by Claude Code, Cursor, and Windsurf developers worldwide.
ShackleAI FinServ Platform
Open-source reference platform for banking and financial services. Multi-service Java + TypeScript architecture with ML-powered credit scoring, fraud detection, and document AI. 253 commits, 53 PRs, 106 issues closed with 8 specialized AI agents.
ShackleAI Agent Framework (PyPI)
Enterprise-grade async-first Python agent orchestration on PyPI. 37 built-in tools, per-call cost tracking across 100+ LLM providers, PyArmor-protected releases. 2,000+ tests at 97% coverage across 117 test files.
WhatsApp AI Chat Platform
RAG-powered WhatsApp AI assistant for small businesses. Semantic search over business catalogs and FAQs, automated payment flows via Razorpay, action token parsing for escalation and order management. 10 development phases completed.
ShackleGuard
"Lighthouse for AI Agents" — AI agent security and red-team testing platform. Automated prompt injection detection, jailbreak testing, PII leakage scanning, hallucination measurement, and EU AI Act compliance reporting. Currently in design phase.
Skills & Expertise
Deep specialization in agentic AI systems — from architecture and LLM engineering to the operational layer that makes agents production-ready.
Agent Architecture
LLM Engineering
AI Operational Layer
AI-Accelerated Development
Platform Engineering
Languages & Frameworks
My 25-Agent AI Development Team
Custom-built AI agents I designed and operate — each specializes in a domain, collectively shipping production code 24/7
Latest Posts
Deep dives into AI agent architecture, MCP protocol, and building production AI systems.
AI Concepts Explainer: Labelbox, KV Cache, MoE Models & Data Residency
A research-based guide to four key concepts shaping modern AI infrastructure — Labelbox's data factory, KV cache mechanics in LLMs, Mixture of Experts architecture, and Anthropic's data residency compliance story.
25 AI Agents Shipping Code 24/7: My AI Development Ecosystem
How I built and operate a 25-agent development ecosystem that ships production code around the clock — architecture, patterns, and lessons learned.
How I Built an MCP Server from Scratch
A deep dive into building @shackleai/memory-mcp — a persistent memory server for AI coding tools with 1,784+ npm downloads.
Open Source
Tools I have built and released for the developer community.
@shackleai/orchestrator
Open SourceMulti-agent orchestrator for production workflows. MCP-native, PostgreSQL-backed, multi-package monorepo.
Journey
Key milestones in building AI infrastructure.
Founded ShackleAI
Started building the Operating System for AI Agents
Launched Memory MCP
Published @shackleai/memory-mcp on npm — now 1,784+ weekly downloads globally
Built Agent Framework
Python async-first orchestration on PyPI — 2,000+ tests, 97% coverage, 37 built-in tools
Shipped Chat Platform
WhatsApp AI assistant with RAG — 10 development phases completed
V2 Platform Pivot
Integrated OpenClaw + CrewAI for browser automation and multi-agent workflows
Launched Orchestrator
Open-source agent orchestrator on npm — 102 commits, 46 PRs merged, multi-package monorepo
Built FinServ Platform
Open-source banking reference platform — 253 commits, 53 PRs, 106 issues closed, 8 specialized AI agents
AI-Accelerated Development
Built 25-agent ecosystem — 675+ commits, 241+ PRs merged, 1,971 CI/CD runs, 539+ issues closed
Let's Talk
Hiring for an AI infrastructure role, need a specialist for agent projects, looking for a technical co-builder, or want hands-on AI workshops for your team — I'd love to hear from you.