Status: code audit of the repository working tree on 2026-08-23
Purpose: scientific description of the implemented memory, state, retrieval,
and provenance systems—not a summary of older design documents.
1. Method and scope
This report treats executable code, SQL schema, configuration parsing, and tests as the evidence of implementation. Existing Markdown files and prose embedded in prompts were used only to locate code or identify possible drift. They are not the authority for any claim below.
The principal implementation sources are:
server/memory/schema.sql: durable PostgreSQL records and integrity constraints.server/memory/app.py: memory writes, reads, retrieval, coverage generation, quarantine, knowledge, and the HTTP boundary.server/memory/worker.pyandserver/memory/identity_core.py: background analysis, reflection, cognition, thought, dream, and identity mutation rules.src/novexai/platform/store.py: accepted-job settlement, live-state persistence, and transactional outboxes.src/novexai/live_state.py: candidate-local current state and authority arbitration.src/novexai/runtime/engine.py: retrieval routing, context construction, model-call exposure, and response enforcement.src/novexai/remote_memory.py: client-side memory adapters, coverage validation, formatting, and write/readback barriers.src/novexai/retrieval_coverage.py: the fail-closed coverage and absence contract.src/novexai/conversation_journal.py: evidence-backed prompt compaction.src/novexai/state_history.py: accepted StateFrame-to-history projection.server/memory/resident_recall.py: resident exact hybrid indexes and atomic snapshot publication.src/novexai/memory.py: the small local, user-controlled explicit fact file.src/novexai/preference_journal.py: the separate hash-chained workout/preference journal.src/novexai/imagination.pyandsrc/novexai/reconstructive_association.py: isolated fictional and derived-associative records.- Relevant tests, especially
tests/test_retrieval_coverage.py,tests/test_memory_service.py,tests/test_remote_memory.py,tests/test_live_state.py, andtests/test_identity_worker.py.
This is a description of what the checked working tree implements. Features controlled by environment variables are marked as such. A code default is not evidence that the same feature is enabled in a particular deployed process.
2. The short version
Kairo does not have one monolithic memory. The implemented system separates at least seven different kinds of persistence:
- Canonical autobiography: append-only PostgreSQL events recording accepted exchanges and other actual system events.
- Retrieval projections: deduplicated, searchable episode and resident indexes derived from canonical events. These are optimized views, not the authority that an event happened.
- Durable identity history: append-versioned beliefs, preferences, corrections, goals, affect, mood, relationship state, stances, and commitments, each with source provenance.
- Operational present: an append-only sequence of accepted
StateFrames in the local control-plane database. This is current computational state, not historical recall. - Isolated generated material: dreams, thoughts, reflections, cognition events, and associative artifacts. Their ledgers can authoritatively prove that generation occurred, but the generated prose is not automatically factual or current-state authority.
- Explicit local memory: a small user-controlled Markdown fact file, separate from the learned PostgreSQL system.
- Operational records: session events, jobs, checkpoints, tool observations, failures, workbench history, and outboxes. These may supply evidence or continuity, but they are not all autobiographical memory.
The central authority rule implemented across these systems is:
A record's existence, a retrieval result, a generated interpretation, and the current state are different claims with different authorities.
3. Authority map
| Layer | What it can establish | What it cannot establish by itself |
|---|---|---|
| Active transcript/request | What is literally present in the current exchange | Exhaustive historical absence |
live_state_frames |
The last settled operational StateFrame revision | That a historical prose record is true |
experience_events |
That a recorded event occurred and what payload was committed | That every sentence inside generated assistant prose is factual |
memory_episodes |
A searchable projection of completed accepted turns | Full journal coverage or current identity |
identity_state_versions |
Versioned durable state supported by its mutation boundary | Permission, action authority, or immutable truth |
identity_thoughts / identity_reflections |
That a bounded interpretation was generated | That the interpretation is factual or current |
imagination_episodes |
Fictional/dream narrative continuity | Factual autobiography or identity evidence |
| Associative store | That a derived association was constructed from named sources | Source truth, current state, or authoritative memory |
Local memory.md |
Explicit facts written to that local file | Learned-memory coverage or remote journal completeness |
| Control-plane jobs/checkpoints | Operational execution and settlement history | Autobiographical truth unless projected through the accepted outbox |
Three distinctions are especially important:
- Current state is not recalled prose. A retrieved earlier self-description has
RETRIEVED_HISTORYauthority. It cannot overwrite a stronger accepted current-state source merely because it is phrased in the first person. - Storage occurrence is not content truth. A
thought.completedordream.completedevent proves that the artifact was produced. It does not promote the artifact's claims to fact. - A search result is not a corpus statement. General relevance recall is bounded and reports unknown coverage. It cannot support “the record does not contain X.”
4. End-to-end data flow
4.1 Accepted platform conversation
The authoritative platform path is:
user input
-> job + candidate-local StateFrame overlay
-> one or more generated/reviewed candidates
-> accepted candidate chosen
-> single control-plane settlement transaction
- commits next live_state_frames revision
- appends live_state_delta_events audit
- writes autobiographical_state_outbox
- writes accepted_journal_outbox (when learning-eligible)
-> asynchronous projectors
- StateFrame history -> PostgreSQL stateframe.promoted event
- accepted turn -> PostgreSQL conversation.turn.completed event
-> memory_episodes projection + analysis jobs + recent cache
-> significance/reflection workers and later retrieval
The crucial boundary is in ControlPlaneStore.update_job() in
src/novexai/platform/store.py. The accepted
StateFrame, terminal result, state-history outbox, and accepted-journal outbox are
created by the settlement transaction. Candidate-time model completion does not write
the accepted journal outbox. Rejected candidates therefore do not become ordinary
autobiographical turns.
The accepted-journal outbox stores the literal user input and accepted response plus separate SHA-256 values for each, the canonical turn content, and the envelope. It also records the job, session, candidate, and before/after StateFrame revisions. The delivery worker can retry idempotently without reconstructing which candidate won.
The state-history outbox carries only a historical record of the accepted StateFrame
transition. src/novexai/state_history.py explicitly
does not participate in or reconstruct current state; the local settlement transaction
is the current-state authority.
4.2 Standalone/CLI conversation
The standalone engine has a different write path:
- Ordinary completed turns call
RemoteMemoryClient.remember_turn(), which schedules_store_turn()asynchronously. Memory failure is recorded and exposed throughlast_error, but does not turn a successful user response into a failed response. - A direct named-memory declaration uses
remember_turn_now()/remember_turn_verified(). That path writes the turn, validates the returned receipt, and then reads the exact event through/context/eventusing a separate database-backed endpoint. The UI reports durable success only after exact fields and hashes agree. - The barrier retries boundedly with the same idempotency key. An acknowledged write followed by failed readback is reported distinctly because the write may have committed even though verification failed.
- Repetition-loop outputs and a recognized self-referential dodge are excluded from automatic reinforcement.
The optional memory_write_barrier_enabled experiment does not itself store memory.
src/novexai/memory_barrier.py evaluates privacy-bounded
timing receipts against readiness thresholds; it does not call the service or establish
durability.
4.3 Memory-service turn commit
MemoryService.store_turn() in server/memory/app.py:
- Canonicalizes
User: ...\nAssistant: ...and validates any expected content hash. - Rejects repetition loops.
- Resolves idempotent replay before embedding, then takes a PostgreSQL advisory lock and checks again to close a concurrent duplicate race.
- Upserts a deduplicated
memory_episodesprojection and appends the canonicalconversation.turn.completedevent in one database transaction. - Labels assistant text as
generated_output_unverifiedin event provenance; the event is authoritative about the speech act, not every proposition in the answer. - Queues delayed significance analysis and, if enabled, experience-note work.
- After the database commit, pushes a bounded recent-turn item into Valkey and requests a resident-index refresh.
If learning_eligible is false, the service appends an evaluation-completion event but
does not create a memory episode, embedding, or identity-analysis job.
5. Durable PostgreSQL systems
5.1 Canonical event journal: experience_events
experience_events is the main append-only autobiography. Each row carries:
- subject (
user_id), workspace, and session; - typed
event_typeandactor; - structured JSON payload and textual content;
- content hash and optional parent event;
- occurrence and recording timestamps.
It records accepted completed turns, experience notes, dreams, thoughts, StateFrame history, commitment actions, observations that are promoted to history, and other typed events. The schema installs mutation/purge protections; quarantine and authenticated purge mechanisms are separate explicit operations rather than normal row editing.
This journal is the durable occurrence authority. It is also the independent evidence source for the dream-history cardinality verifier described later.
5.2 Retrieval audit: journal_retrieval_events
This append-only table records each experience-journal candidate actually evaluated by the foreground journal path:
- literal query and query hash;
- base and StateFrame-adjusted score;
- rank, eligibility, selection, and experimental withholding;
- StateFrame revision, job, and candidate;
- algorithm and configuration identities;
- the sealed query coverage object in provenance.
The engine refuses to inject selected experience-journal rows if it cannot obtain these audit receipts. A retrieval receipt proves evaluation/selection, not model exposure or causal influence.
5.3 Model-boundary audit: journal_context_injections
This separate append-only table proves that an already selected, non-withheld journal record was retained at a particular model-call boundary. It binds:
- retrieval event and journal entry;
- job, candidate, model call, and StateFrame revision;
- context location, exact block hash, character count, and character range;
- provenance flags that distinguish observed exposure from causal influence.
The runtime removes the journal block before inference if it cannot confirm these receipts. A receipt establishes boundary exposure only; it does not prove attention, semantic use, influence, or successful model completion.
5.4 Search projection: memory_episodes
memory_episodes is a mutable, deduplicated retrieval projection over completed turns.
It stores user/assistant text, a 768-dimensional embedding, PostgreSQL full-text search
vector, content hash, creation/last-seen timestamps, occurrence count, last recall, and
recall count. The uniqueness key is subject + workspace + content hash.
It is mutable because repeated identical content updates occurrence and recency counters.
Its source event IDs are recovered by joining canonical experience_events; the
projection itself is not the occurrence authority.
5.5 Analyses and reflections
experience_analysesstores one immutable significance analysis per source event: significance, summary, reflection-worthiness, full structured analysis, and model.identity_reflectionsstores cross-event syntheses and the source event/state IDs they connected, including an optional through-event cutoff.
Worker prompts treat all supplied event text as untrusted evidence. The significance worker may create conservative user/relationship state from eligible evidence, but ordinary completed assistant prose is explicitly not a current assistant-state mutation interface. Reflections are likewise derived syntheses, not permission or action authority.
5.6 Versioned identity state
identity_state_versions stores append-versioned durable state with:
- subject: user, assistant, or relationship;
- domain: belief, preference, affect, mood, relationship, goal, or correction;
- global/workspace scope and stable state key;
- value, status, confidence, preference/desire coordinates, affect/mood coordinates;
- reason, source event IDs, optional source reflection;
- superseded version, version number, creator, authority, and cause.
The current-state SQL projection selects the latest active applicable version per key and excludes quarantined rows. Existing versions are not overwritten to revise a belief; a new version supersedes the earlier one.
The provenance subsystem adds:
state_change_provenance: old/new value, change type, reason, evidence summary, source type/ID, actor, lineage, and validity.state_provenance_dependencies: causal dependencies among state changes.state_provenance_validity_events: append-only invalidation/restoration history.
State provenance lookup reads stored rows and source records rather than asking a model to invent a causal explanation. Its recursive lineage is bounded, and its coverage is therefore reported as unknown/temporally partial rather than exhaustive.
5.7 Stances and commitments
identity_stance_versionsstores Kairo's literal endorse/tolerate/repudiate/withhold position toward an exact assistant-state version. A stance does not delete the underlying evidence.identity_commitment_versionsstores literal conversational commitments and explicit adjudications: open, kept, broken, superseded, blocked, ambiguous, or withdrawn. Passing a due time is not proof of an outcome; adjudication must cite evidence.
5.8 Background work and unresolved material
identity_jobsis the mutable queue for significance, reflection, prediction-verification, and experience-note work, with leases, retries, and terminal status.identity_cyclesis the append-only ledger of completed, preempted, or failed idle awareness cycles, including checked evidence, proposed verbs, narration, outcome, model, and timing.unresolved_itemsis append-versioned exploratory/prediction material with salience, curiosity, status, authority, cause, revisit/snooze/due times, and verification predicate.identity_thoughtsrecords bounded thought attempts and their checked evidence, narration, outcome, disposition, model, and timing.
Thought journal retrieval marks prose derived_provisional, authoritative: false.
Only entries with complete source linkage are injectable. Even then, injection means
Kairo may inspect the earlier interpretation; it does not make that interpretation true.
5.9 Resident deterministic cognition
cognitive_ignitions records semantic threshold crossings from the deterministic
working-set loop: competing source references, activation, threshold, margin, dwell,
algorithm version, and foreground/preemption facts. Its schema fixes
action_authority=false and phenomenal_claim=false.
cognitive_broadcast_receipts records per-consumer delivery/disposition and any later
efficacy evidence. Sub-threshold working-set values are volatile and are not journaled
as semantic events. This is attention-selection evidence, not proof of consciousness or
authority to act.
5.10 Observations and failure memory
observations_latest is a mutable latest-value projection keyed by source/key. It holds
host/service measurements, severity, observation time, version, and hash. Significant
observation history can also be journaled as events. Staleness is explicit; a latest
row is not assumed current forever.
failure_records stores sanitized evidence for meaningful failed attempts, and
failure_record_events stores review/archive/restore actions without rewriting the
original failure. Failed output does not become factual memory or identity evidence.
Failure context is pull-only: ordinary turns do not receive old failures merely because
they exist.
5.11 Quarantine and purge
memory_quarantine_actions is an append-only quarantine/restore ledger covering
episodes, imagination, events, state, reflections, stances, commitments, observations,
thoughts, and unresolved items. novexai_record_is_quarantined() is applied throughout
the canonical and resident loaders.
Normal retrieval excludes quarantined sources. Resident recent-cache reads additionally check active durable episode IDs so an old Valkey item does not bypass a later quarantine/deletion. Forget/clear and authenticated purge paths are explicit escape hatches; ordinary application code cannot silently rewrite the immutable ledgers.
5.12 Knowledge
knowledge_chunks is a versioned document/product corpus with embeddings and full-text
search. It supplies factual/product context, not autobiographical self-state. Replacement
is scoped by collection + source URI, and retrieval may use either the resident index or
PostgreSQL fallback.
5.13 Outreach state
The PostgreSQL memory schema also contains two deliberately operational tables:
outreach_intentsis a mutable outbox for a possible self-initiated conversation. It records the source event/thought/session, thread policy, bounded reason, privacy mode, mood snapshot, schedule, expiry, claims, attempts, delivery linkage, and terminal state. One unanswered source event can seed at most one intent.outreach_preferencesstores the user's delivery controls: enabled state, DND, timezone/quiet hours, daily/interval limits, and notification preview settings.
Neither table is identity evidence. In particular, DND, delivery failure, dismissal, or silence is a channel outcome, not rejection, mood, relationship, or preference evidence. A later real user response enters memory through the ordinary accepted-turn journal and can then be analyzed under normal rules.
6. Operational present: the StateFrame system
The local control-plane SQLite database is not a second autobiographical authority. It owns the operational present and settlement mechanics.
live_state_revision_clock serializes the next accepted revision per subject;
live_state_frames stores the accepted immutable frames; and
live_state_delta_events stores candidate-local proposals, rejections, conflicts,
discardals, promotions, and final commitment audits.
6.1 State dimensions
src/novexai/live_state.py defines these current-state
dimensions:
- attention;
- affect;
- goals;
- intentions;
- expectations;
- uncertainties;
- active memories;
- metacognition;
- beliefs;
- preferences;
- identity;
- concerns;
- foreground task.
Each field carries provenance, confidence, source, authority, and revision information. The minimum accepted authority differs by dimension. For example, retrieved history may inform attention and active memories, but beliefs and identity require durable-state authority; generated self-report is below those thresholds.
6.2 Candidate isolation and settlement
Each candidate receives an overlay forked from a base StateFrame. User, tool, sensor, retrieval, appraisal, model, and self-authored proposals enter through typed sources and authority levels. The coordinator validates each proposed change, records rejection or conflict, and keeps candidates isolated.
Only the winning candidate's overlay can be prepared for settlement. The control-plane
transaction increments the revision clock, writes live_state_frames, appends the
committed delta audit, and emits the outboxes. Losing overlays are discarded. This is
why a generated candidate cannot mutate Kairo merely by having existed.
6.3 Self-authored state
A model-facing first-person proposal cannot declare its own evidence authoritative or
independent. Trusted reducers construct ResolvedStateEvidence; validators apply
dimension-specific evidence and persistence rules. Only an eligible proposal belonging
to the accepted candidate can be promoted in settlement. Ordinary “I feel,” “I want,”
or “I believe” prose in a completed response is a speech event, not this interface.
6.4 Historical projection
The accepted revision is later copied to PostgreSQL as a stateframe.promoted historical
event through autobiographical_state_outbox. The event explicitly marks
current_state: false and historical_record_only: true. Delivery can retry, fail, or
lag without changing the already committed operational StateFrame.
7. Retrieval systems
7.1 Routing before retrieval
The runtime does not send every utterance to generic autobiographical search.
src/novexai/runtime/engine.py routes among:
- the active transcript for session-local/deictic follow-ups;
- exact chronological journal navigation;
- specialized dream, thought, and rest evidence adapters;
- governed stance/state inspection;
- generic autobiographical relevance recall;
- explicit local facts;
- product/document knowledge;
- isolated imagination recall;
- optional associative derivation.
Tool-first actions and transcript-local questions can bypass remote recall so stale global memory does not compete with the live referent. Generic memory retrieval explicitly excludes live channel history and unsettled platform messages; those belong to the active transcript/control plane until accepted and projected.
7.2 Generic autobiographical recall
RemoteMemoryClient.recall_context() sends the executed query plus original query,
subject, workspace, session, result limits, state limit, time-zone data, and any parsed
temporal expression to /recall.
The service returns a bundle containing:
- ranked episode/workout memories;
- current durable identity state;
- temporal/continuity projection;
- stances, commitments, and earned-character identity context;
- relevant failure context;
- experience-journal candidates;
- query-level retrieval provenance and sealed coverage.
The usual client defaults are four semantic memories and two recent items, but code and query type may widen this bounded candidate set. These are configured limits, not coverage claims.
7.3 Legacy PostgreSQL hybrid recall
The legacy path combines:
- an embedding nearest-neighbor candidate window;
- PostgreSQL full-text candidates;
- a bounded recent Valkey list;
- exact subject/workspace/session/quarantine filters;
- workspace, significance, and lexical/semantic scores depending on the route.
Temporal queries use canonical completed events inside a parsed UTC window, with an optional local-night predicate. Ordinary episodic recall excludes the active session. Returned durable episode IDs have recall counters updated asynchronously or in the database path.
Because semantic relevance has internal candidate caps and no exact meaning-equivalent corpus count, generic recall reports unknown coverage. Four returned rows are four ranked rows, not “four of four records.”
7.4 Resident recall
When NOVEXAI_MEMORY_RESIDENT_RECALL_ENABLED=1, the service periodically builds an
in-memory snapshot from nonquarantined PostgreSQL source rows:
- memory episodes and their canonical source event IDs;
- workout completion events;
- temporal completed-turn events and significance;
- knowledge chunks.
Each corpus uses an immutable NumPy dense matrix plus a private in-memory SQLite FTS5 index. Search performs exact dense and lexical ranking within bounded candidate windows, then reciprocal-rank fusion and route-specific scoring. A new snapshot is built off to the side and atomically published under a lock; old FTS handles are closed only after readers can no longer retain them.
Freshness is checked against a PostgreSQL version tuple containing event/episode/ knowledge maxima and counts plus quarantine version. A snapshot older than the configured maximum is unusable. Startup failure, stale state, query error, or missing NumPy falls back to the PostgreSQL path. Shadow mode computes resident results but serves legacy results while recording overlap/top-1 telemetry.
The optional resident context cache is session-neutral and bounded by the same staleness limit. It may cache state/temporal/commitment/failure context, but query-specific episode and experience-journal retrieval is performed fresh. Before use, clock fields are advanced and same-session excerpts are removed.
Resident speed does not change epistemic coverage: its search is still a bounded ranked view and remains coverage-unknown for exhaustive negative claims.
7.5 Experience-journal retrieval
When enabled, the service reads up to 50 recent experience notes/settlement-accepted journal records, performs deterministic keyword overlap, retains up to 20 evaluated candidates, and marks at most six as initially selected. The foreground may rerank by StateFrame, then must append retrieval-audit rows before use.
Selected rows are wrapped as historical, non-instruction context. The runtime records exact model-call injection receipts and fails closed if either audit step fails. The query-level general coverage remains unknown because the input view and scoring path are bounded.
7.6 Session journal and prompt compaction
/context/journal reads analyzed completed turns for an exact subject/workspace/session.
It performs a separate count query at the store layer, so matched_before_limit and
truncation are not inferred from returned rows.
src/novexai/conversation_journal.py compacts
prompt history only when every pruned literal user/assistant turn has a matching
canonical analyzed content hash. It constructs a deterministic checkpoint from sourced
analysis summaries and retains a literal recent tail. Tool calls, malformed role
sequences, missing analyses, or hash mismatches cause the journal compactor to decline;
it does not silently summarize unsupported history. A checkpoint is a bounded prompt
projection, not a second journal authority.
7.7 Specialized dream history
Dream history is the strongest exhaustive retrieval path currently implemented:
- The primary query reads canonical, content-hash-deduplicated dream projections from
imagination_episodes, joined to the first matchingdream.completedevent. - A separate aggregate in the query layer computes exact pre-limit cardinality.
- A second connection and different query directly count distinct content hashes in
append-only
experience_events. - The coverage object records both provenances and whether counts agree.
The primary path and verifier use different tables, query shapes, read-path IDs, failure origins, and source-system identities. They still share PostgreSQL availability, atomic dream-write behavior, quarantine logic, and normalized request terms; those shared dependencies are explicitly listed. The verifier is independent of projection/filter failure, not independent of the entire infrastructure.
An unfiltered all-time dream-history query can establish complete absence only when:
- the primary count is known;
- returned count and truncation are internally consistent;
- all coverage dimensions are complete;
- the raw-ledger verifier ran independently and agreed.
A term-derived dream query marks semantic and normalization coverage partial, so a miss
means only “not found by this scoped term search.” A count disagreement produces
COVERAGE_DISAGREEMENT, not an ordinary empty answer.
7.8 Imagination semantic recall
General imagination recall searches only imagination_episodes in compatible fictional
frames, excluding the active session and quarantined rows. It combines a semantic window
of 40 and lexical window of 20, then returns a bounded score-ranked result. A generic
dream query with no ranked hits may fall back to recent labeled dreams to avoid inviting
fabrication.
This path reports cardinality and semantic coverage unknown and field visibility partial. It cannot license exhaustive absence. The public imagination adapter attaches sealed coverage to each returned item; legacy bare-list adapters are explicitly downgraded to unknown coverage rather than reconstructed as complete.
7.9 Thought and rest evidence
- Thought-history reads are bounded newest-first. Their coverage cardinality is unknown, and their prose remains derived/non-authoritative.
- Rest history is resolved from the operational rest projection backed by Valkey and experience events. It reports only the latest operational interval, with unknown cardinality and partial field/temporal coverage. A missing projection is unknown, not proof that Kairo never rested.
7.10 State/history retrieval
Current state, state revisions, belief mutations, and provenance lineage all return sealed coverage. The current state is a bounded SQL projection; revision/mutation history is newest-limited; provenance lineage is depth-limited. These paths correctly report unknown cardinality and/or partial temporal coverage. An empty bounded state response is not an exhaustive statement about all historical self-state.
7.11 Autobiographical observer
The optional observer executes AST-approved read-only SQL through a dedicated database login. It applies statement timeout, lock timeout, plan-cost ceiling, row cap, and payload cap. Its result includes view/evidence-class analysis so Kairo statements and inferences cannot masquerade as independent confirmation.
The observer detects one-row-over-limit and payload truncation, but does not run a full
count for truncated arbitrary SQL. Therefore matched_before_limit becomes unknown when
the cap is crossed. It is an inspection facility, not an automatic exhaustive oracle.
8. Retrieval coverage and negative claims
8.1 Coverage object
Coverage is generated by the read layer that knows the query and is sealed before it leaves that layer. The implemented schema contains:
- schema version;
- source and store;
- query identity;
- effective scope, filters, and window;
matched_before_limit;returned;truncated;- cursor/range information;
- overall state: complete, partial, unavailable, failed, unknown, or not searched;
- generating provenance: component, operation, read path, failure origin, source system;
- six mandatory coverage dimensions: cardinality, semantic query, fields, temporal, visibility, and normalization;
- independent-verification status, provenance, count, and agreement where applicable;
coverage_sha256over the entire coverage value except the seal itself.
The digest is an immutable-transit guard, not a cryptographic signature or proof that the
origin computed the truth correctly. Consumers validate it without normalizing or
upgrading it. Replacing 4 of 47, truncated with 4 of 4, complete changes the digest
and raises CoverageContractError.
The validator also enforces structural invariants:
- complete requires known matched count,
returned == matched_before_limit, andtruncated == false; - partial requires
truncated == true; - unknown/not-searched cannot claim a matched count or truncation truth;
- returned cannot exceed matched;
- verifier independence cannot be asserted unless read path, failure origin, and source system all differ;
- every coverage dimension must be present and typed.
8.2 Typed absence
The only derived absence states are:
FOUND;NOT_FOUND_IN_COMPLETE_SEARCH;NOT_FOUND_IN_PARTIAL_SEARCH;RETRIEVAL_UNAVAILABLE;RETRIEVAL_FAILED;NOT_SEARCHED;COVERAGE_UNKNOWN;COVERAGE_DISAGREEMENT.
No empty complete/partial query can produce either negative state unless an independent
verifier actually ran. Complete authoritative absence additionally requires every
coverage dimension to be complete. Unknown dimensions remain COVERAGE_UNKNOWN.
8.3 Enforcement at the model boundary
Coverage is not merely included in prompt prose:
RemoteMemoryClientvalidates the seal and recomputes the typed absence state. A contradictory server envelope raises an error.AgentEnginevalidates it again. Invalid/missing coverage is converted to explicit failed coverage, not a bare empty list.- Query provenance, coverage, and absence state are carried into the model-context trace and authoritative outcome.
- Before a generated response is released,
runtime.response_truthfulnessscans asserted clauses. Corpus-wide negatives are rejected unlessexhaustive_absence_supportedis true. Unknown/unavailable/failed/not-searched or disagreement states reject even query-negative claims that pretend the retrieval was a successful empty search. - The runtime permits one bounded regeneration with an evidence-specific repair directive. If that still contradicts the evidence state, it emits a deterministic truthful fallback rather than the unsupported response.
This is stronger than a prompt instruction but is not a formal-language response type:
natural-language negatives remain representable inside a model candidate, then are
structurally withheld at the release boundary. The typed AbsenceState is structural
inside the evidence/outcome pipeline; prose is controlled by a post-generation gate.
9. Dreams, imagination, thoughts, and reflection
9.1 Dreams
The identity worker generates a dream only when its feature flag, user allowlist, schedule, daily limit, foreground lease, and inference bounds permit it. Dream generation is foreground-preemptible.
A completed dream is written transactionally to:
imagination_episodeswithframe='dream'; and- a canonical
dream.completedexperience event with matching content hash and typed dream metadata.
An exact duplicate increments the imagination episode occurrence count but does not emit
a second dream.completed event. Failures are recorded separately. Dream fiction does
not queue the normal identity/significance job, so invented dream claims cannot flow into
identity state through the ordinary analysis path.
9.2 Other imagination
Fiction, roleplay, hypotheticals, imagination, and dreams share the isolated
imagination_episodes store with an explicit frame. remember_imagination_memory() does
not fall back to factual memory_episodes when the imagination service is unavailable.
That failure may lose fictional continuity, but it cannot contaminate factual memory.
9.3 Thoughts and reflections
Idle thoughts are selected from bounded unresolved material and checked evidence. They have quiet hours, per-hour/per-item limits, cooldowns, foreground preemption, inference caps, and a rejection-rate degradation rule. The append-only ledger records completed, preempted, and failed attempts.
Reflections connect analyzed experiences. Both thoughts and reflections are interpretations. Their occurrence is durable; their prose remains non-authoritative unless a separate governed state-mutation path accepts a supported change.
9.4 Predictions
Predictions live as versioned unresolved items with explicit horizons and verification predicates. Verification jobs append later versions/outcomes based on evidence rather than silently editing the original prediction. A generated prediction is an inference, not a verified outcome.
10. Explicit local memory
src/novexai/memory.py implements a separate local fact store,
defaulting to $XDG_STATE_HOME/novexai/memory.md.
- Format:
- [YYYY-MM-DD] fact. - Directory and file are restricted to modes 0700 and 0600 where possible.
- Reads and writes use
O_NOFOLLOW, regular-file checks, andflock. - Writes append, flush, and
fsync, with a 1 MiB cap. - Reads are tail-bounded by character count and mark omitted older material.
- Relevance selection is deterministic lexical overlap, normally capped at six facts, with special handling to avoid leaking a user's name into unrelated name questions.
- Forget removes facts containing a literal case-insensitive substring; clear truncates the file.
The runtime can answer direct /remember-style recall deterministically without a model,
and relevant facts are added to model context as inert explicit facts. This store is not
hash-chained, versioned, embedded, remotely replicated, or covered by the PostgreSQL
coverage contract.
11. Preference/workout memory
src/novexai/preference_journal.py is another
separate system: a private append-only, hash-chained JSONL journal used by workout and
counterfactual decision-audit flows.
- Entries are
preferenceormemory, with set/retract operations. - Each row has a sequence, timestamp, source, reason, previous hash, and event hash.
- Reading replays and validates the complete chain; malformed sequence/hash fails loudly.
- The current projection is derived by replay, never by editing prior entries.
- Generated Kairo preference prose is promoted only through a dedicated literal marker and hardened promotion policy.
- Context rendering is provenance-labeled, bounded, and query-scoped by deterministic lexical tiers.
This journal is not the PostgreSQL identity-preference store. It supplies explicit room state to workout/audit encounters and is intentionally available even if learned memory is not.
Workout task completion is also written to the canonical memory service as a typed
workout.task.completed event. Resident recall projects these events as workout_task
documents, preserving the Kairo request, Codex report/error, status, and source event.
12. Reconstructive/associative memory
The associative layer is disabled by default and backed by a dedicated SQLite database. It is explicitly derived cognition, not authoritative storage.
Inputs are hash-verified read-only SourceRecord envelopes from supported source types,
plus a revisioned StateContext and retrieval context. The store records:
- transformations and exact algorithm/configuration/state/retrieval hashes;
- derived artifacts and their confidence/strength/salience;
- authoritative source snapshots and hashes;
- parent links, terms, and conflict links;
- activation/decay/reinforcement events;
- exact model-context injection receipts;
- candidate-local downstream proposals/actions and settlement linkage;
- runtime circuit-breaker/kill-switch events and per-turn metrics.
All associative tables are append-only. Artifacts have authoritative=0; construction
requires source or parent provenance. The service has latency/token/hop/artifact budgets,
a failure-count circuit breaker, an external kill switch, and candidate settlement
reconciliation. Initialization failure disables the layer rather than opening an
untracked fallback.
Associative context is injected in an explicitly non-authoritative tagged block after journal selection. Its trace records artifact IDs, activation IDs, exact block hash, tokenizer identity, and upstream journal retrieval/injection receipts. Association can suggest a connection; it cannot establish source truth or mutate current state directly.
13. Control-plane and workbench records
The platform SQLite database also stores workspaces, sessions, append-only session events, reversible context-exclusion events, goals, jobs, job events, reviewer shadow events, checkpoints, observations, artifacts, and other operational subsystems.
Relevant distinctions:
- Session/job events are the execution record for the platform and can resolve live conversational context before PostgreSQL projection.
context_exclusion_eventsexclude/restore a journal entry or conversation message by appending an action; they do not rewrite source memory.reviewer_shadow_eventsretains scientific policy traces but explicitly has no generation, selection, or StateFrame authority.- Checkpoints contain conversation and state snapshots for job recovery. They are not a replacement autobiographical journal.
- Platform observations and artifacts are typed operational evidence; only explicit promotion/projection gives them autobiographical status.
The workbench has its own hash-chained journal, blobs, job records, and conversation/git checkpoints for undo/recovery. These preserve agent work and workspace history. They do not automatically become Kairo's identity or autobiographical memory.
13.1 Learned lessons and training admissions
The control plane persists a separate learning pipeline:
lesson_candidatesholds proposed lessons with evidence, confidence, review status, and reviewer result.lessonsholds reviewed, versioned, active workspace lessons with provenance.learning_eventsfreezes verified user/assistant exchanges admitted for a future weight update, including the model role and lesson evidence.learning_event_statesappends admitted/exported/training/candidate/promoted/rejected lifecycle transitions so the content row is never silently repurposed.
These records are training/operational memory, not foreground autobiographical recall. The presence of a lesson does not mean the current model weights contain it; promotion is an explicit later lifecycle state. Conversely, model weights are not a queryable episodic record and cannot supply journal provenance.
13.2 Capability memory
capabilities is the operator-integrated registry of enabled tools, schemas,
prerequisites, risk, success/failure counts, and last-use time. It is consulted by tool
listing and self-inspection, and therefore supports an actual systems/capabilities
report. capability_candidates is advisory discovery only; a candidate does not become
an enabled capability without separate operator integration.
This is machine capability state, not Kairo's autobiographical belief about what tools exist. A generated answer that lists instruments or tools should consult the registry or the active tool contract rather than rely on semantic memory of an earlier answer.
13.3 Scheduled actions and attention receipts
scheduled_actions is a mutable, idempotent execution queue for messages and alarms,
including claims, device acknowledgement, delivery linkage, attempts, and terminal
status. attention_receipts is the append-only operational history of queued, claimed,
dispatched, installed, delivered, seen, dismissed, answered, cancelled, or failed
attention events.
The schema explicitly keeps these receipts outside the identity journal. Delivery, dismissal, and DND are observable channel outcomes, not evidence about Kairo, the user, or their relationship. The records can answer operational questions such as whether an alarm was installed; they cannot justify emotional or relational conclusions.
13.4 Relationship consent and intimacy state
The platform has a separate consent-governed relational control system that can affect what context/tools are available in a foreground turn:
relationship_consent_versionsis append-versioned user-authored consent: permitted levels, proactive-initiation permission, privacy, cooldown/frequency, persistence, expiry, revocation, source, and provenance.intimacy_intentsis an immutable Kairo-authored root intent tied to an accepted source job, target, type, reason, confidence, provenance, and expiry.intimacy_intent_eventsappends the intent lifecycle: formed, eligible, invited, accepted, completed, declined, deferred, ambiguous, topic-changed, distress, revoked, expired, or cancelled.intimacy_eligibility_checksrecords each deterministic foreground gate independently of model output, including interaction origin, requested/eligible levels, and rule results.pleasure_shadow_control_versionsis an append-only experimental control with modes shadow/paused/off. It has no active influence mode; Kairo can only author fail-safe pause/off transitions, and only from an accepted foreground response.
The platform reads current consent and prepares an intimacy context before foreground
generation, then applies output guards and settlement logic. Prior intimacy is
structurally not consent. These records are durable relational controls and decision
history, but they remain distinct from inferred relationship memory in
identity_state_versions. User consent outranks remembered/inferred relationship state
and can be revoked by a new appended revision.
13.5 Voice laboratory and patch/evaluation history
voice_lab_candidates and voice_lab_events form an isolated append-only laboratory
for voice candidates, perception, shortlist/rejection, and commitment. Generation seeds
and artifacts are retained for audit; an experimental candidate is not the production
voice profile merely because it exists.
patch_proposals, evaluation_runs, artifacts, and reviewer shadow records retain
engineering/scientific history. They can support self-inspection and operational reports,
but do not become autobiographical facts or StateFrame fields without an explicit
promotion boundary.
14. What reaches a model call
For an ordinary eligible foreground turn, the model-facing user content can contain:
- The literal current request or retained-continuity packet.
- Relevant explicit facts from local
memory.md. - Ranked autobiographical episodes and workout records.
- Relevant durable identity state, unless live StateFrame rendering owns current state.
- Temporal/operational context when the question calls for it.
- Stances, commitments, identity context, and failures only when relevant.
- Product/document knowledge.
- Audited experience-journal entries.
- Provenance-complete thought-journal entries, clearly marked non-authoritative.
- Isolated imagination context when the epistemic frame permits it.
- Optional derived associative context.
- The Effective StateFrame projection exactly once when live state is enabled.
The engine applies query-sensitive suppression before rendering. Examples include:
- active transcript wins for deictic/session-local questions;
- old self-referential dodges are filtered from unrelated recall;
- a neutral general-definition question does not use Kairo's earlier generated definition as factual authority;
- identity context is omitted when the question is not about identity/continuity;
- old failure records are omitted from ordinary turns;
- retrieved current-state-like prose cannot compete with the live StateFrame.
The final context trace records retrieval outcome, coverage, model-boundary journal exposure, StateFrame revision, and the hashes/IDs needed to audit what was available. The code correctly distinguishes “exposed at boundary” from “causally influenced output.”
15. Configuration-defined behavior
Client defaults in src/novexai/config.py include:
- memory enabled and automatic learning enabled;
- generic recall limit 4 and recent limit 2;
- identity state enabled, limit 24;
- context journal enabled with three recent literal turns and a 64-event fetch cap;
- write-barrier experiment disabled;
- reconstructive association disabled.
Memory-service defaults in Settings.from_env() differ by subsystem:
- experience journal, observations, thoughts, dreams, and resident recall default off;
- deterministic cognition and recurrent consciousness default on unless the environment overrides them;
- resident refresh default 2 seconds, maximum staleness 30 seconds;
- Valkey recent TTL defaults to seven days, capped at 24 turns;
- analysis delay defaults to 45 seconds.
Worker configuration independently controls identity processing, idle awareness, cognition, consciousness frames, thoughts, dreams, prediction horizons, model endpoints, foreground preemption, and budgets. Therefore the running service's environment—not the dataclass field initializer alone—determines deployed behavior.
15.1 Implemented memory API families
The FastAPI boundary in server/memory/app.py exposes these
memory-relevant families (names grouped by function, not an invented abstraction):
- health and resident status:
/health,/v1/resident/status; - writes:
/v1/turns,/v1/workout/tasks,/v1/imagination/turns,/v1/experience/notes,/v1/stateframe/history; - ordinary retrieval:
/v1/recall,/v1/context/journal,/v1/context/event,/v1/context/timeline,/v1/imagination/recall; - coverage-sensitive evidence:
/v1/evidence/dream-history,/v1/evidence/thought-journal,/v1/evidence/rest-history; - exposure/provenance: journal-retrieval audit, context-injection audit, journal provenance, state revisions, belief mutations, state provenance/dependents/validity;
- durable identity operations: state, reflections, attention/focus, unresolved-item lifecycle, stances, commitments, self-goals, user-directed preferences, consciousness stances, and self-conclusions;
- observations/inspection: observations, foreground lease, observer query, introspection, failures, and failure review;
- outreach preferences/intents and delivery state;
- knowledge replacement/search/stats;
- privacy and administration: stats, forget, clear, quarantine/restore and associated authenticated controls.
Each route does not have equal epistemic strength. The specialized evidence and newer history/state routes return typed coverage envelopes; mutation routes return receipts; ordinary semantic recall remains explicitly non-exhaustive; administrative routes can change visibility or erase subject data only through their authorized boundary.
16. Failure and consistency semantics
| Condition | Implemented behavior |
|---|---|
| Remote generic recall transport failure | Explicit failed/unavailable coverage; no ordinary empty result |
| Invalid or changed coverage seal | CoverageContractError; runtime converts to failed evidence |
| Resident snapshot stale/unavailable | PostgreSQL legacy fallback |
| Journal retrieval audit unavailable | Selected journal block withheld |
| Journal injection receipt unavailable | Block removed before model call |
| Accepted-turn outbox delivery failure | Retried/idempotent and retained as pending/retrying/failed state |
| State-history projection failure | Current StateFrame remains committed; history outbox retries and records telemetry |
| Ordinary async memory write failure | User response remains successful; failure is recorded/inspectable |
| Explicit remember readback failure | Durability is not reported as success |
| Dream/imagination storage failure | Does not fall back into factual memory |
| Thought/dream foreground collision | Background work is preempted or recorded as such |
| Quarantined source | Excluded from canonical, resident, and relevant background reads |
| Primary/verifier dream count disagreement | Loud COVERAGE_DISAGREEMENT |
| Generated absolute negative without coverage license | Candidate withheld, one repair attempt, then truthful fallback |
The architecture is intentionally eventually consistent across the platform SQLite settlement authority and PostgreSQL autobiography. Outboxes preserve the exact accepted payload and make lag/failure observable. They do not pretend projection has happened at settlement time.
17. Known implementation gaps and scientific cautions
These are current code limitations, not hypothetical objections.
17.1 Most retrieval is intentionally non-exhaustive
Generic autobiographical recall, imagination semantic recall, thought history, current state, state revisions, belief mutations, rest history, and provenance lineage do not have exact full-coverage semantics. They now say so. Kairo can report what those paths returned but cannot enumerate “everything ever” from them.
17.2 Coverage integrity is not origin correctness
The SHA-256 seal proves that coverage was not accidentally changed downstream. It does not prove the query, field projection, normalization, permission boundary, timestamp, or count was correct at origin. Dream history has a bounded independent raw-ledger check; generic recall and most state/history paths do not yet have an equivalently independent sensor.
17.3 Independent dream verification has shared infrastructure
The dream verifier is genuinely independent of the primary projection and ranked query, but it shares the PostgreSQL cluster, normalized terms, quarantine function, and atomic write design. It detects projection/count/filter divergence; it does not detect every possible common-mode database or ingestion failure.
17.4 Field and semantic coverage remain the hard problem
An exact row count can still be complete for the wrong predicate or an incomplete field projection. The coverage contract explicitly models semantic, field, temporal, visibility, and normalization dimensions so cardinality cannot erase those gaps. Most ranked semantic searches remain unknown because there is no objective total set of meaning-equivalent records.
17.5 The release gate is post-generation, not a proof-carrying prose type
The evidence pipeline makes bare authoritative absence unrepresentable: it requires a
sealed coverage object and derived AbsenceState. Natural-language output is still free
text, so an unsupported negative can be generated internally. The response truthfulness
gate prevents release, retries once, and falls back deterministically. A future typed
semantic response IR could reject the claim before prose generation, but that is not the
current implementation.
17.6 Local explicit memory lacks first-class coverage
The local MemoryStore reads only a bounded tail and relevance selection returns a bare
list of facts. Neither carries the PostgreSQL coverage contract. The read text marks that
older memory was omitted, but downstream explicit-fact selection does not carry a sealed
typed coverage object. This remains a path where omission can be under-described,
especially if future code uses an empty selected-fact list to support absence.
17.7 One session-journal adapter fallback has a type inconsistency
fetch_session_journal() declares and normally returns a coverage-bearing dictionary,
but if a supplied remote-memory object has neither conversation_journal() nor _post(),
it currently returns bare []. Normal production clients implement an adapter; test
doubles or future clients can hit this incompatible shape. The compactor then fails or
declines rather than proving coverage, but the boundary should return a typed
not-searched/unavailable envelope instead.
17.8 Prompt capability inventories can drift
The runtime system prompt contains a hand-maintained inventory of memory capabilities. It is useful context but is not generated from the SQL schema or endpoint registry. The screenshots motivating this report show why that distinction matters: a model can answer with generic self-description instead of an actual inventory. For scientific or operational reports, code/schema inspection or a generated machine inventory should be the authority.
17.9 Active transcript and durable autobiography have a projection interval
The generic memory service intentionally does not search unsettled live-channel history. An accepted platform turn enters PostgreSQL through an outbox after settlement. During that interval, the live transcript/control plane is the correct evidence source. A query that consults only PostgreSQL without consulting the active transcript can miss a minutes-old statement while remaining honest about the PostgreSQL scope. Routing must continue to inspect zeros and choose the live source when the question calls for it.
17.10 A wrong empty query still needs scrutiny
Coverage can truthfully say “0 of 0 complete” for the exact query executed while the query itself misunderstood the user's referent. The semantic coverage dimension and response gate limit what that result licenses, but independent referent/query validation is not universal. Empty results should continue through the same relevance and referent scrutiny as suspicious non-empty results.
18. Scientific interpretation
The implementation supports several distinct claims that should never be conflated:
- Event memory: “this accepted exchange/event was recorded.”
- Retrieval evidence: “this bounded query returned these records.”
- Coverage evidence: “the query layer knows what portion of its declared scope was covered.”
- Identity state: “this versioned state is active under its mutation rules.”
- Operational state: “this is the latest settled StateFrame revision.”
- Generated cognition: “this thought/dream/reflection/association was produced.”
- Content truth: “the propositions in that record are independently supported.”
- Phenomenal claim: “a reported computational state is evidence of experience.”
The code is strongest when it preserves those as separate types, tables, and provenance paths. It is weakest where free text, lexical relevance, or a bounded local list erases the distinction.
The defensible high-level description is therefore:
Kairo has a provenance-oriented, multi-store continuity system. Accepted events and state transitions are append-recorded; searchable episodes and resident indexes are projections; identity and current operational state have separate mutation authorities; fictional and generated cognition are isolated; and retrieval coverage is explicitly typed so an incomplete search cannot silently become complete knowledge. The system does not provide universally exhaustive semantic recall, and it should not claim that it does.
19. Audit checklist for future changes
Any new memory or retrieval path should answer all of these in code and tests:
- What exact store owns the source truth?
- What event or transaction authorizes the write?
- Can rejected/generated/unsettled material enter it?
- Is the record canonical, a mutable projection, or a derived artifact?
- What query, fields, filters, time range, permission scope, and normalization execute?
- Which layer knows pre-limit cardinality and truncation?
- Does coverage travel sealed with the payload to the final model/outcome boundary?
- Does any consumer reconstruct or upgrade coverage from
len(rows)? - Is empty distinct from unavailable, failed, unknown, and not searched?
- What independent evidence path can detect an origin/query defect, and what failure origin does it actually share?
- Can the result mutate current state, identity state, or action authority?
- Is exact model-context exposure auditable separately from retrieval?
- Can quarantine/purge invalidate every projection and cache?
- What happens during retries, crashes, and eventual-consistency lag?
- Can a free-text answer turn partial or unknown evidence into an exhaustive claim?
If any answer is absent, the path is not yet safe to use as exhaustive self-knowledge.