ai-agent-profiler (aap) is a local-first observability and optimization tool for AI coding agents. It acts as a transparent HTTP proxy between your agent and the LLM provider, capturing every API call as NDJSON trace files and providing analysis, classification, and cost optimization.
Architecture
Zero-framework TypeScript. Dependencies: better-sqlite3, smol-toml, zod, @modelcontextprotocol/sdk, aws4. No Express, no ORM. Native node:http/node:https.
src/
capture/ Async NDJSON trace writer + secrets redaction
(authorization headers stripped before persistence)
proxy/ HTTP/S listener + router + transparent passthrough
Single-port routing: /<session_id>/<provider>/...
Bedrock support with SigV4 signing + binary stream
parse/ Off-hot-path trace parser → metrics
(tokens, latency, cost, context composition)
Provider-agnostic classifier: Claude Code + OpenCode
analyze/ Cache regeneration analysis, Claude transcript
parsing, idle-gap detection, search→read patterns
store/ SQLite metadata store (sessions, requests, tool calls)
search/ FTS5 full-text index over prompts, responses,
tool calls, file edits, shell commands, errors
recommend/ Per-session findings (amplification, duplication)
api/ HTTP read API (GET /sessions, /requests, /stats)
ui/ Static web dashboard
hook/ Shell-filter hooks (PATH interposition for git,
grep, ls, find, npm test, node --test)
cli/ aap.ts entrypoint + subcommands (serve, run, parse,
mcp, compare, export, search, tag, sessions, hook)
Design principles
Record-first, analyse-later. Raw traces are captured as append-only NDJSON files. Nothing blocks the hot path. All metrics, classification, and recommendations are derived offline via aap parse. Re-running parse is idempotent and updates the database when classification rules change.
Proxy is invisible. Never modifies request or response bodies during capture. Uses a tee pattern: request/response streams are piped through PassThrough capture sinks asynchronously with zero backpressure on the client. The agent cannot tell the proxy is there.
Secrets redaction. authorization and x-api-key headers are stripped before NDJSON writes. Credentials never touch disk.
Key features
Prompt-cache awareness. Captures provider cache-hit tokens and cache-creation tokens. Cost analysis reflects real spend, not theoretical.
Request classification. Tags every API call by kind: main (user turn), search (file search subagent), title (session title gen), recap (mid-session summary), compact (context compaction), etc. Classifier is provider-agnostic — works with both Anthropic format and OpenAI/DeepSeek format.
FTS5 full-text search. Indexes every prompt, response, tool call, file edit, shell command, and error. Also imports agent-native transcripts from Claude Code and OpenCode sessions. Find any past interaction by content.
MCP server. Exposes profiler data as MCP tools so coding agents can self-introspect: search_history, search_edits, search_errors, find_previous_fix, recall_session. Agents query their own cost profile mid-session.
Shell hooks. PATH interposition via aap run --hooks. Filters tool output (grep, ls, find, npm test) before it enters context. Does not modify API request bodies, preserving prompt cache.
Introspection. aap introspect runs deterministic SQL queries against the database and generates JSON reports. Reproducible analysis: same database → same result.
Optimize layer. Optional request shaping before the provider: prune stale tool results, truncate large outputs, deduplicate repeated reads, compact full context. Rules configured via config.toml.
Bedrock support
Speaks AWS Bedrock’s binary event-stream protocol via aws4 SigV4 signing. Separate proxy mode for signing-only passthrough.
Real usage data
After 160 sessions, 6,767 requests, 9,443 tool calls:
- Tool results consume 70% of all costs
- Optimization saved 21M tokens
- 88% of cache invalidations are kind-switches, not timeouts
- FTS5 index over 170+ content chunks
Run from source on GitHub (MIT). Node >= 20 required.