Four signals from one conversation.

Voice to text. Older speakers are where general-purpose speech recognition fails most often — slower tempo, dialect, less distinct articulation. E:go therefore uses a managed speech-recognition service specialised for Korean (long-form recognition). The reason transcription quality matters this much is simple: every language-based axis in the cognitive engine below is computed on this text, so recognition error is the system's shared bottleneck. Utterance boundary detection (endpointing) is managed separately, because it feeds directly into the response-latency measures of the conversational-flow axis.

Photos and video to context. The vision model does two things. First, it classifies evidence of activity — a walking path, a meal, exercise. On the physical-activity axis, that evidence is weighted: video counts for more than a photo, a photo for more than words alone, so that reported activity and evidenced activity are not the same thing. Second, it seeds reminiscence questions that begin from the senior's own day — “That photo looks like you went out today. Where did you go?”

The record this produces is first-person, grounded in real life, and accumulated day by day. Those three properties are the raw material for everything below.

Before the evidence, a distinction about kinds of evidence. The figures in this item are reference studies on measurement validity — whether these markers distinguish cognitive decline. Whether daily cognitive-stimulation engagement is beneficial (the intervention evidence: WHO guidance, Cochrane review) is a separate question — see 'Designed on established ground' below. Keeping that distinction sharp is a rule we hold ourselves to.

🧠 Memory — core axis. What: how well a story holds together in meaning. How: ko-SBERT (an embedding model that represents the meaning of Korean sentences as vectors) computes semantic similarity between adjacent sentences (coherence), alongside the number of semantic clusters and repetition of the same content (perseveration). Evidence: in story-recall tasks, coherence markers distinguished healthy older adults from cognitive decline (including AD and MCI) at AUC 0.84 (Clarke et al., 2021). Cluster counts (Cintoli et al., 2024) and the memory-loading of perseveration (Pakhomov et al., 2018) provide supporting evidence.

📖 Vocabulary — core axis. What: diversity of word use and part-of-speech composition. How: KoNLPy (Mecab) morphological analysis extracts content-word lemmas, from which we compute MATTR and MTLD — lexical-diversity measures designed not to distort with utterance length. We rejected simple TTR because it swings with length, misreading “a quiet day” as vocabulary decline. Evidence: semantic-fluency (word-retrieval) models reach AUC 0.91 — the strongest single signal in the reference literature (Kaser et al., 2024).

🗨️ Conversational flow — reinforcing axis. What: response latency and the rhythm of speech. How: because the AI is the conversation partner, the system knows exactly when its question ends — a uniquely clean measurement setting where response latency carries no speaker-separation error. Parselmouth (the Python interface to Praat, the standard phonetics toolkit) analyses inter-pausal units (IPUs) and pause structure. Evidence: conversational-timing markers distinguish cognitive decline at AUC 0.80 (De Looze et al., 2021). One caveat we state plainly: that figure comes from human-to-human conversation research, and human-to-AI conversation requires calibration — which is on our own validation list.

Speech — supporting axis. What: speech rhythm (articulation time, pause ratio). How: derived from word-level timestamps of the speech-to-text output. Acoustic features (timbre, tremor) are not computed in the current version.

The axes themselves are an audited choice. An earlier 'attention' axis was removed from display after we concluded its signal overlaps memory and executive function — no independent information. And because the literature cannot fix weights across four axes (each study differs in cohort and task), we do not hard-code them; they are learned from our own baselines as deployment data accumulates.

A single score is not a signal. One conversation's score is one point on that person's baseline. What matters is a gradual longitudinal departure from their own baseline — which is why the pipeline is built not as a score calculator but as session → markers → personal time series → change detection.

Honest limits, stated ambition. The AUC figures above are reference results from different studies, cohorts and tasks — not clinical validation of the E:go pipeline itself. Korean older-adult norms and education adjustment are still being established, so the current version trusts within-person change only, never absolute scores. Claiming early detection of cognitive change is a goal we intend to earn — through concurrent- and predictive-validity studies and the software-as-a-medical-device (SaMD) regulatory pathway — not a claim we make today.

No single signal produces a risk grade on its own. Three streams of a different nature are combined — behaviour (usage and engagement patterns: an active user going quiet), language and mood (recurring expressions of low mood, listlessness or risk in conversation), and cognition (the longitudinal trend of the four-axis index). The grade rises when the three point the same way. The structure exists to reduce false alarms — a senior should never be flagged 'at risk' simply because they said less one day.

The output is a priority list, not a diagnosis. On the institutional dashboard, staff see who needs looking at first — as data. When engagement stops, an AI check-in call goes out first; repeated no-answers alert the assigned caseworker. Whether to visit, call, or keep watching is decided by a person, not the system. E:go does not replace the caseworker; it shows them the time between visits that no one could see before.

What it would take for these signals to one day earn the name 'early detection' is written plainly in the cognitive-engine item above — validity studies and a regulatory pathway. Until then, the name of this feature is signal, and the subject of its final sentence is always a person.

The four gates: ① Automatic removal of personal information at speech recognition — the transcription path is designed to handle identifying expressions such as names and contact details. ② Face and document blurring — images showing faces or identity documents are targets for automatic filtering and blurring in the analysis path. ③ Physical separation of identity and content — name, contact details and date of birth live only in a separate authentication layer; the service database holds nothing but a randomly generated anonymous ID. From the service database alone, no record can be tied to a person. ④ De-identified egress — data crossing the system boundary is limited to de-identified data.

Consent is never bundled. Consent to use the service, to analysis, and to external use are separate, each optional, each revocable at any time. The legal baseline is Korea's Personal Information Protection Act (pseudonymised-data provisions, Article 28-2).

One thing said precisely: these four gates are the system's design principles, and their implementation and auditing expand with each deployment stage. Maintaining a verifiable state rather than declaring completion — that is the posture of this entire page.