Technical Dossier // Production Research

Work & Research

Autonomous agent architectures, long-term memory evaluation, AI benchmark analysis, and production engineering notes.

[tmux:0 project/weft] pid: 9024

STATUS: RELEASED

Introducing Weft - Memory For Your Agents

Weft is a memory system designed to get your agents just the context they need, and nothing else.

#projects/weft#ai/development

$ weft up --profile local

[INFO] Starting Weft memory service

[OK] FastEmbed local embeddings ready

[OK] Docker container active on port 8080

[OK] MCP server registered

[tmux:0 research/longmemeval] pid: 4821

STATUS: COMPLETE

My Memory System Cheated To Beat LongMemEval Until I Fixed It

Last week I ran Weft, my homegrown memory layer, through LongMemEval and it scored 69.0% overall, 72.1% task-averaged. I was pleased. I shouldn't have been. The number was a lie, and the system that produced it was destroying the test data and calling it a win.

#coding/ai#memory

$ python eval_weft.py --dataset=longmemeval

[INFO] Initializing Oracle evaluator

[WARN] False 69.0% -> True 43.6%

[PERF] Memory rot at >32k tokens

[OK] Markdown fence bug resolved

[tmux:1 research/advisor-bench] pid: 6109

STATUS: BENCHMARK_DONE

I Benchmarked Anthropic's Advisor Strategy on Task Decomposition. The Expensive Model Was the Worst.

Anthropic's Advisor Strategy promises near-Opus intelligence at near-Sonnet cost. A server-side tool that pairs a cheap executor model with an expensive advisor. I benchmarked it in task decomposition.

#coding/ai#benchmarking

$ python run_advisor_bench.py

[INFO] Task decomposition: 42 graphs

[WARN] Opus advisor: 3.4x latency

[COST] +41% token spend over Sonnet

[OK] Sonnet standalone won benchmark