Every conversational digital twin on this platform is built from the same stack of published techniques — not model tricks. This page names each technique, describes how we apply it, and cites the paper, standard, or documented source of the approach so you can verify the underlying method independently.
Model the expert as a persona layer over a corpus, not as a fine-tune. Keep persona and knowledge separate so both can be edited and audited.
Each expert has a `system_prompt` (persona / voice / rules) plus a private, chunked, embedded corpus. Grounding rules are appended by the app so the persona editor can't accidentally disable them.
Instead of relying on a model's parametric memory, fetch relevant passages from a trusted corpus at query time and instruct the model to answer only from them, with inline citations.
User query → embed with `google/gemini-embedding-001` (3072-d) → cosine-nearest 6 chunks from pgvector → passages injected into the system prompt with numeric IDs → model is told to cite [1] [2] and refuse when uncovered.
Store document embeddings in a purpose-built vector index; use approximate nearest-neighbour (HNSW / IVFFlat) with cosine distance for sub-second retrieval at scale.
Chunks are stored in Postgres `vector(3072)` with an HNSW cosine index. Retrieval is a stable-security-definer SQL function (`match_chunks`) scoped per-expert so corpora are isolated.
Surface uncertainty to end users. When the top-retrieved passage is a weak match, the answer is more likely to be extrapolation, not grounded expertise — signal that visibly.
The best cosine similarity from `match_chunks` is bucketed into High (≥ 0.78), Medium (≥ 0.65), Low (< 0.65) and rendered as a pill next to every answer. Bands are calibrated for gemini-embedding-001.
A second model call, acting as a skeptical critic, checks the draft against the source passages, lists issues, and revises. Reduces hallucination and unsupported claims.
After the primary generation, `critiqueAndRevise` prompts gemini-2.5-flash to return strict JSON `{issues[], verdict, revised}`. If `verdict === "revise"` we send the revised answer, and store both draft and critique in the audit log.
For expert-witness and regulated use, every AI-generated statement needs a defensible provenance trail: what was asked, what sources were used, what the model drafted, what a human reviewed and adopted.
Every reply writes an `answer_audits` row with question, retrieved chunks, draft, critique, final reply, confidence, model, and timestamp. Admins review each answer in `/admin/review` and mark it Adopted, Reviewed, or Rejected — creating the record a Daubert challenge expects.
Turn the human review queue into a real feedback signal by asking the reviewer to rate each answer on a numeric fidelity scale, not just Adopted/Rejected. The score becomes a trend line for corpus quality, persona drift, and model regressions.
Every answer in `/admin/review` gets an optional 1–10 fidelity score alongside Adopt / Reviewed / Reject. The dashboard shows the running average across all rated answers so a drop in fidelity is visible immediately — the same metric used in LLM eval harnesses like G-Eval and HELM.
For expert-witness, regulated, and legal use, the audit log itself must be defensible: any edit or deletion of a past answer must be detectable. Best practice is an append-only, hash-chained ledger — each row's sha256 hash depends on the previous row's hash, so altering any row invalidates every row after it.
A BEFORE INSERT trigger on `answer_audits` computes `row_hash = sha256(prev_hash ‖ canonical(row))` and refuses updates to any hashed column or deletions of any row. Reviewers can freely add review status, notes, and fidelity scores — but the question, sources, draft, critique, and final reply are frozen forever. `verify_answer_audits_chain()` replays the whole chain on demand and returns the first broken row, which the admin console exposes as a one-click 'Verify chain' button.
Separate 'who the expert is' from 'what the expert knows'. Persona lives in the system prompt; refusal-when-uncovered is enforced by app-owned rules so it can't be edited away.
`buildExpertSystemPrompt` composes persona + mode-specific rules (text vs. voice) + the retrieved SOURCE PASSAGES. The rules mandate first-person voice, inline citations, and a polite refusal + related-topic suggestion when passages don't cover the question.
For a digital twin to feel like the expert, use the expert's own cloned voice, low-latency streaming TTS, and low-error transcription. Preserve consent and revocation controls.
Speech-to-text: `openai/gpt-4o-mini-transcribe` via the Lovable AI Gateway. Text-to-speech: ElevenLabs `eleven_flash_v2_5` with a per-expert voice ID (falling back to a default). Voice IDs are stored per expert so a twin can be re-voiced or revoked.
Push-to-talk is friction. Energy-based VAD with pre-roll and a silence hangover gives natural, hands-free turn-taking without shipping a full-duplex realtime stack.
The browser captures 16 kHz mono PCM, runs an RMS-based VAD (start 0.018, continue 0.012, 700 ms hangover, 6-chunk pre-roll), encodes WAV, and ships one utterance at a time to the server.
For real work — legal matters, board disputes, M&A, patient files — end users must be able to bring private documents to an expert without exposing them to other users or leaking into the public corpus. Each engagement is its own siloed workspace with its own retrieval namespace and its own audit trail.
A signed-in user creates a Case Room bound to one expert. Files upload to a private `matter-docs` storage bucket keyed by `matter_id`. Embeddings go into `matter_chunks` scoped by `matter_id` with a dedicated `match_matter_chunks` SQL function. At query time we retrieve from BOTH the private matter and the expert's public corpus, tag every citation as CASE FILE vs. EXPERT CORPUS, and prioritize private context. RLS restricts view/write to the owner and site admins only; nobody else — including other paying users — can see the files, messages, or embeddings.
Multi-tenant expert data must not leak between experts or users. Enforce this at the database, not in application code — RLS + security-definer retrieval functions.
All app tables have RLS enabled. `match_chunks` and `get_expert_for_chat` are `SECURITY DEFINER` with `SET search_path = public`, scoped by `_expert_id`. Roles live in a separate `user_roles` table checked via `has_role()` — never on the profile.
Raw source material — transcripts, articles, decks, emails, book chapters — is noisy, redundant, and versioned inconsistently. Ingesting it directly produces bloated retrieval, weak citations, and unauditable knowledge. Best practice is a curated middle layer: raw sources live in topical folders owned by the expert's team; an AI synthesis step compresses each folder into one canonical markdown summary; only the summaries are embedded into the twin. Updates re-synthesize a single folder, not the whole corpus.
For Russ Rosenzweig, source materials live in 10 numbered Google Drive folders (00 Book & Bio, 01 Podcasts, 02 RTG/EWC Articles, 03 Admissibility & Daubert, 04 Scientific Evidence, 05 Reports & Testimony, 06 Ethics & Fees, 07 Reference Index, 08 New Book Chapters, 09 Emailed Files). An AI produces a V1 canonical summary per folder — structured with headers, key concepts, and machine directives. Those 10 summaries are embedded alongside the 7 persona files (voice, FAQ, refusals, case vignettes) for a 17-file knowledge base. When Russ's staff adds new raw docs to a folder, only that folder's summary is regenerated and re-uploaded. This separates raw source (Drive) from curated knowledge (twin) from persona (system prompt) — three independently editable layers.