OpenClaw memory, benchmarked against ScallopBot
Both systems were run over the same LoCoMo long-conversation memory benchmark, on the same models, scored the same way. ScallopBot reaches F1 0.48 against OpenClaw’s 0.38 across 1,049 QA items — a 26% relative improvement. This page explains what the memory system does to earn that, and exactly what the number does and does not mean.
ScallopBot vs OpenClaw on LoCoMo
LoCoMo is a standardized benchmark of 1,049 QA items across 5 long conversations and 138 sessions. Both systems used Moonshot kimi-k2.5 for generation and Ollama nomic-embed-text (768-dim) for embeddings, and both were scored with token-level F1.
| Category | ScallopBot | OpenClaw | Relative |
|---|---|---|---|
| Overall F1 | 0.48 | 0.38 | +26% |
| Adversarial | 0.97 | 0.77 | +26% |
| Multi-hop | 0.42 | 0.32 | +31% |
| Temporal | 0.34 | 0.26 | +31% |
| Single-hop | 0.20 | 0.14 | +43% |
| Open-domain | 0.09 | 0.07 | +29% |
Bio-inspired memory lifecycle
Memory is not a write-once store with a similarity search bolted on. It runs a four-stage lifecycle modelled on how sleep processes memory in a brain — encode, consolidate, associate, prune.
Hybrid retrieval
Recall runs BM25 keyword matching and dense embedding search together, then optionally reranks the merged candidates with an LLM before anything reaches the context window. Keyword search catches the exact name or identifier an embedding blurs; the embedding catches the paraphrase BM25 misses. The rerank is the part that lifts the overall F1, and it is optional — turn it off and you trade a little accuracy for a cheaper, faster path.
Results are score-gated rather than top-k truncated. If nothing clears the bar, nothing is injected, which is most of why the adversarial category lands at 0.97: the system declines instead of confabulating from weak matches.
Temporal queries
Memories carry their dates into the embedding rather than sitting beside a timestamp column. A regex-based detector spots when a question is time-scoped — “last month”, “before the review”, “what changed since” — and routes it through time-aware retrieval instead of plain similarity. That is the 0.34-vs-0.26 temporal category.
MCP-native, in both directions
ScallopBot is an MCP client: a bundled skill lets it consume any MCP server you already run, so those tools are available to the assistant alongside its own.
It also exposes its memory over MCP. The bundled MCP server publishes three tools that any MCP client can call — Claude Code, or anything else that speaks the protocol:
It runs straight out of the build — node dist/mcp-server/index.js with SCALLOPBOT_DB pointed at your memory database. To register it with Claude Code:
claude mcp add scallopbot --env SCALLOPBOT_DB=/path/to/memories.db -- node /path/to/scallopbot/dist/mcp-server/index.jsThere is no npm package yet, so both paths point at your own clone of the repo.
Common questions
Does it work with OpenClaw skills?
Yes. Every capability in ScallopBot — bash, browser, file I/O, git, Docker, PDF, web search, memory — is a self-contained skill written in the OpenClaw SKILL.md format. Skills declare their own requirements (binaries, env vars, OS) and are gated at load time.
That means community skills built for OpenClaw install and run, including ones pulled from ClawHub with a single CLI command. The memory system is itself one of these skills, so it composes with the rest rather than sitting beside them.
How does it compare to Mem0 or Zep on LoCoMo?
It does not, and any table that puts those numbers in one column is misleading you. The ScallopBot-vs-OpenClaw result on this page is a controlled comparison: both systems ran the same LoCoMo items, the same generation model (Moonshot kimi-k2.5), the same embeddings (Ollama nomic-embed-text), and the same token-level F1 scoring.
Published LoCoMo figures from other memory vendors generally use different models, different retrieval budgets, and in several cases a different metric entirely — LLM-as-judge accuracy rather than token F1. Those scores are not on the same scale as these, so they are not reproduced here. The only honest way to compare would be to re-run every system under one harness.
Can Claude Code use it over MCP?
Yes. ScallopBot ships an MCP server that exposes its memory as three tools — memory_store, memory_recall and memory_temporal — so Claude Code, or any other MCP client, can write to and query the same memory the assistant uses.
Register it with: claude mcp add scallopbot --env SCALLOPBOT_DB=/path/to/memories.db -- node /path/to/scallopbot/dist/mcp-server/index.js. There is no npm package yet, so the paths point at your own clone.
It works in the other direction too: a bundled skill makes ScallopBot an MCP client, so any MCP server you already run is available to the assistant.
What actually drives the benchmark gain?
Four things, each measurable on its own. Score-gated context and anti-fabrication constraints produce the adversarial jump — 0.97 against 0.77 on questions designed to have no answer. Memory fusion and NREM consolidation drive multi-hop. Date-embedded memories plus temporal query detection drive the temporal category. Hybrid retrieval with LLM reranking lifts the overall number.
The full per-category breakdown, including the categories where the absolute scores are low for both systems, is on the homepage. Nothing is hidden by averaging.
→ What it costs to run · Full benchmark breakdown · Source on GitHub