🧠 Learner Bot RAG System

1. Overview

Every child on the Pickatale platform gets their own AI-driven Learner Bot. The bot tracks curriculum mastery, vocabulary gaps, and reading engagement — then generates nightly recommendations and teacher/parent reports.

The RAG system is the intelligence backbone: a shared infrastructure with strict per-child data isolation. Heavy processing runs on dedicated worker servers (no GPU on primary). Results are stored in a central knowledge graph queried by the Learner Bot.

Key Design Principles

• One shared RAG, per-child isolation internally — separate RAG per child would be unscalable
• Workers separate from primary server — primary server runs live projects; graph processing cannot share its resources
• Educational knowledge graph, not chat memory — the goal is curriculum mastery tracking, not conversation history
• Cloud embeddings, no local GPU — primary server has no GPU; OpenAI Embeddings API is reliable and cost-effective
• Data flows UP only — aggregated, anonymized. Never sideways between schools or down to children

2. Architecture

Primary Server (readingtester.com)
├── Telemetry DB (port 3110)      — raw reading events
├── Per-child markdown logs        — data/children/{child_id}/logs/
└── Existing projects              — untouched, no resource competition

        ↓ nightly batch (worker servers only)

Worker Servers (dedicated, separate machines)
├── Pull raw telemetry per child cohort
├── Call OpenAI Embeddings API (text-embedding-3-small)
├── Mine against predefined educational goals
├── Generate mastery scores + knowledge relationships
└── Write results → Central RAG Server

        ↓ merge results

Central RAG Server (dedicated, separate machine)
├── Neo4j         — knowledge graph (educational relationships)
├── PostgreSQL    — identity, access control, raw profiling
└── pgvector      — semantic similarity search

        ↓
Learner Bot (port 3120) queries Central RAG nightly
        ↓
Content recommendations + Teacher/Parent reports

Component Responsibilities

3. Data Pipeline (5 Stages)

Nightly Batch Flow (per child cohort)

1. Worker pulls raw telemetry for child cohort from Telemetry DB
2. Worker reads per-child markdown log
3. Worker calls OpenAI Embeddings API → generates vectors
4. Worker mines against predefined educational goals
5. Worker writes to Central RAG:
   a. Updates mastery scores on Neo4j nodes
   b. Updates pgvector embeddings
   c. Appends to child profile in PostgreSQL
6. Learner Bot queries Central RAG
7. Learner Bot generates recommendations + reports
8. Reports delivered to Teacher Dashboard + Parent view

Batch window: 00:00–06:00 local time per region.

Real-time Flow

Child reads → Telemetry event logged → Telemetry DB
Teacher requests report → Learner Bot queries Central RAG (cached results)
Content recommendation needed → pgvector similarity query → ranked book list

4. Graph Schema (Neo4j)

Nodes

Relationships

Example Query Patterns

// Find concepts a child is struggling with, that are prerequisites for their next objective
MATCH (c:Child {child_id: $cid})-[:STRUGGLING_WITH]->(concept:Concept)
      <-[:PREREQUISITE_OF]-(blocker:Concept)
RETURN blocker.title, blocker.difficulty ORDER BY blocker.difficulty

// Find books that teach concepts this child needs
MATCH (c:Child {child_id: $cid})-[:STRUGGLING_WITH]->(concept:Concept)
      <-[:TEACHES]-(book:Book)
WHERE book.fk_level BETWEEN $child_fk - 0.5 AND $child_fk + 0.5
RETURN book.title, book.id ORDER BY book.fk_level

5. Multi-Tenancy & Access Control

Tenant Hierarchy

Nation
  └── District
        └── School
              └── Class
                    └── Child

Data Isolation Rules

• Every node carries: child_id + school_id + district_id + nation_code
• All queries are scoped to the appropriate level — never cross-school or cross-district
• PostgreSQL RLS enforces tenant boundaries at the database level
• Neo4j RBAC enforces property-level access in the graph
• Data flows UP only (aggregated, anonymized) — never DOWN or SIDEWAYS

Access Control Model

6. Privacy & GDPR

• Per-child folder isolation: data/children/{child_id}/
• AES-256 encryption at rest on all raw markdown files
• Deletion: remove child = delete folder + purge all graph nodes with that child_id
• No PII in telemetry events — child_id only, never name/email
• Embeddings tagged with tenant metadata, never raw text stored in embeddings layer
• k-anonymity enforced at district level and above (minimum group size: configurable, default 10)
• GDPR export: ZIP per-child folder + export graph node properties on request

7. Decisions Log

8. Assumptions

• Worker servers will be provisioned by Jixian (cloud VPS or cloud functions)
• Central RAG server is a separate dedicated machine — not the current primary server
• Curriculum objectives are sourced from CM (cm.readingtester.com) via API only (no direct DB access)
• OpenAI text-embedding-3-small is the embedding provider
• Nightly batch window: 00:00–06:00 local time per region
• Learner Bot is the ONLY service that queries the Central RAG directly

9. Open Questions

To be resolved in Sig + Jixian design call before build begins.

10. Next Steps