Blog

Filesystem vs Database for Agent Memory

memory agents architecture

People often ask the wrong question:

Should agent memory live in the filesystem or in a database?

My background is in data systems, so I think about this the same way I think about analytics infrastructure.

A company does not run on analysts’ notebooks alone, and it does not ask analysts to do all of their work directly in warehouse tables. It needs both:

  • operational systems to run the business
  • a central warehouse to accumulate durable facts
  • analysts using SQL and Python to make sense of raw data
  • curated tables that business users can consume in BI tools

Agents need the same split.

  • the filesystem is where the agent does temporary work
  • the memory layer is where the organization keeps durable knowledge

That is the model behind Lobu’s Memory docs.

The warehouse analogy

In a normal data stack, information moves from operational systems into a warehouse, gets cleaned up by analysts, and becomes something the rest of the business can use.

Classic analytics stack
Step 1
OLTP systems
CRM, support, billing, product events
Step 2
CDC / sync
Changes flow out of operational systems
Step 3
Warehouse
One durable place to ask cross-company questions
Step 4
Analysts
Use SQL and Python to make sense of the data
Step 5
Curated tables
Cleaned models the rest of the business can trust
Step 6
BI users
Dashboards and self-serve reporting

That same pattern shows up in agent systems.

An agent pulls files and data from tools, APIs, or conversations, uses code and prompts to inspect them, and should write the durable result into an organizational memory layer that future agents and operators can reuse.

The metaphor is simple:

The filesystem is the analyst’s notebook. The memory layer is the warehouse.

Why you need both

The filesystem is excellent for ephemeral work:

  • downloaded PDFs
  • screenshots
  • temporary scripts
  • notebooks
  • CSV exports
  • intermediate analysis outputs
  • drafts and generated reports that may or may not matter later

But a filesystem is a bad place for organizational memory. It is local to one sandbox, hard to share across agents, hard to audit, and bad at answering questions like:

  • who owns this project?
  • what does this customer prefer?
  • what changed since last week?
  • what should another agent remember later?

The opposite mistake is to treat the memory layer like a junk drawer. Not every PDF, chart, script, or half-finished CSV deserves promotion into durable memory.

Good agents should be able to think messily and remember cleanly.

What should go where?

Keep in the filesystem
Step 1
Raw source files
Original PDFs, exports, and documents
Step 2
Temporary code
Scripts, notebooks, and one-off tooling
Step 3
Intermediate analysis outputs
Scratch results produced during the work
Step 4
Drafts
Half-finished writing and working notes
Step 5
Screenshots
Visual evidence and temporary captures
Step 6
One-off transformed data
Useful for the current task, not durable memory
Write to the memory layer
Step 1
Customer facts
Stable information about accounts and organizations
Step 2
User preferences
How people want to work and communicate
Step 3
Project state
Current phase, status, and major context
Step 4
Blockers and ownership
Who owns what and what is in the way
Step 5
Recurring operational knowledge
Patterns agents should reuse later
Step 6
Decisions worth recalling later
Approved outcomes and durable conclusions
Step 7
References to evidence
Links back to the source behind the fact

How this maps to Lobu

Lobu makes this split explicit.

  • Every agent has a local workspace.
  • In file-first projects, Lobu enables Owletto from [memory.owletto] in lobu.toml.
  • MEMORY_URL is still supported as an optional base-endpoint override for local or custom Owletto deployments.
  • If no Owletto config is resolved, Lobu uses @openclaw/native-memory, so memory stays local to that workspace.
  • If Owletto is configured, Lobu uses @lobu/owletto-openclaw when that plugin is installed, otherwise it falls back to native memory.

Owletto is the shared organizational memory layer:

  • shared across agents and sessions
  • structured as entities, relationships, and events
  • searchable by more than file paths and grep
  • inspectable and correctable by operators

So the practical model is:

  • filesystem = ephemeral working state
  • Owletto = durable organizational knowledge

The short version

If your agent is acting like an analyst, give it a notebook.

If your organization wants lasting knowledge, give it a warehouse.

That is why serious agent systems usually need both.