Perception-Driven Memory

niuma_code's memory system is built on memory-palace — it transcribes runtime events into persistent memory in real-time, rather than extracting memories only at conversation end.

Core Idea

Traditional approach: LLM conversation → extract memories after completion niuma_code approach: 10 perception events fire in real-time as work happens

File read      →  Eye event: "file X was examined"
Tool call      →  Body event: "tool Y was executed"  
Code written   →  Tongue event: "file X was modified"
Error occurred →  Nose event: "error in tool Y"
Task completed →  Outcome event: "goal Z achieved"

10 Perception Events

Event Chinese Trigger What It Records
eye_read 目 File read Which files were examined
eye_search 目 Code search What was searched for
body_tool 身 Tool call Tool name, input, success
body_bash 身 Bash command Command, output, duration
tongue_write 舌 File write What was written, file path
tongue_edit 舌 File edit What was changed, diff
nose_error 鼻 Error occurred Error type, context
nose_warning 鼻 Warning Warning context
outcome_task 果 Goal set Task description, scope
outcome_done 果 Goal achieved Completion score (0.0-1.0)

How Memory Is Stored

Fact Triples

Structured knowledge stored as entity-attribute-value:

{
  "entity": "UserService",
  "attribute": "file_path",
  "value": "app/services/user.py",
  "importance": 0.8,
  "session_id": "abc123"
}

Conversation Summaries

Full conversation context compressed into searchable summaries:

{
  "text": "User requested adding pagination to GET /users endpoint. Implemented with limit/offset parameters.",
  "importance": 0.6,
  "session_id": "abc123"
}

4-Layer Retrieval

When a new conversation starts, memory is retrieved through 4 layers:

Layer Purpose Example
1. Active task Current work context "Currently editing auth.py"
2. Recent session Last few conversations "Yesterday's bug fix in login"
3. Important facts High-importance knowledge "Project uses FastAPI + SQLAlchemy"
4. Semantic search Topic-relevant memories "Related to authentication patterns"

Retrieved memories are injected into the system prompt at stable positions.

Bayesian Decay

Memory importance decays over time following a Bayesian model:

score = base_importance × recency_factor × frequency_factor
Factor Behavior
base_importance Set when memory is created (0.0-1.0)
recency_factor Decreases with time since last access
frequency_factor Increases with how often memory is recalled

This ensures recently-used, frequently-relevant memories surface first.

Perception Events in Action

Example: Bug Fix Session

1. User: "Fix the KeyError in login"
   → outcome_task: "Fix KeyError in login function"

2. LLM reads auth.py
   → eye_read: "auth.py examined"

3. LLM finds error on line 42
   → body_tool: "locate_symbol used"

4. LLM edits auth.py
   → tongue_edit: "auth.py modified"

5. LLM runs tests
   → body_bash: "pytest tests/test_auth.py"

6. Tests pass
   → outcome_done: "score=0.95"

All 6 events are stored in real-time. Next session, when you start working on auth, these memories are automatically retrieved.

WAL (Write-Ahead Log)

Memory writes go through a WAL for crash safety:

1. Event fires → append to WAL file
2. WAL buffer fills → flush to database
3. Database write → WAL checkpoint

If niuma_code crashes mid-session, the WAL replays on next startup to recover any lost events.

Configuration

{
    "memory": {
        "enabled": true,
        "max_contexts": 5,
        "auto_recall": true,
        "wal_encrypt": true
    }
}
Setting Default Description
enabled true Enable/disable memory system
max_contexts 5 Maximum memory contexts
auto_recall true Auto-inject relevant memories into system prompt
wal_encrypt true Encrypt WAL file for privacy

Privacy