MoCA Solo: How to build reliable AI models despite limited data

When MoCA Cognition approached us to develop the intelligence behind MoCA Solo, their new digital version of the renowned cognitive test, a major challenge quickly emerged: there was almost no usable data to train AI models.

In a clinical context, this is a critical issue. It is impossible to automatically score a test if the models don’t have enough examples to learn what constitutes a correct or incorrect response. Faced with this reality, our close collaboration with MoCA and their partners led us to design a data-driven strategy that turned a nearly empty dataset into a robust, operational system.

The initial observation: lots of data… but very little usable

From the very beginning, we dove into the analysis of the dataset provided by MoCA.

What we discovered:

• Hours of videos and audio recordings of patients completing the test…
• … but the recordings often included multiple people speaking at once.
• Annotators were talking over the patients.
• It was impossible to isolate the voices and determine who was saying what.
• On the images, there were many artifacts, variations in angles, lighting, or simply unusable data.

In the end, despite the impressive volume of data, only a small fraction was actually suitable for training reliable models.

For audio in particular, the conclusion was clear: unusable data would risk producing unreliable results.

A different strategy was needed.

Our response: rebuild the data instead of struggling with It

When raw data is unusable, there are two options:

• Give up on performance
  
  
• Rebuild a clean, controlled training dataset
  

We obviously chose the second option.

A. For Drawings (Clock, Cube, Trail Making)

We developed a hybrid approach combining:

• Existing visual models (YOLO) to detect key elements
  
  
• Clinically coded expert rules to ensure compliance
  
  
• Synthetic data to compensate for the lack of real variety
  

The goal wasn’t to create perfect synthetic images, but enough diversity for the model to learn the important visual and clinical patterns.

Example: Clock Drawing

• Only a few dozen usable examples were available.
• MoCA requires very precise scoring (alignment, hand position, contour, numbers…).
• The possible variations are endless.

We generated hundreds of synthetic images, manipulating number positions, angles, line thickness, and more.

This allowed the model to generalize even without a large real clinical dataset.

Example: Cube Drawing

The cube is evaluated according to a very strict protocol: numbering, perspective, parallelism…

With almost no real data, we had to think differently.

Instead of relying on a complex model requiring thousands of images, we simplified the approach with rules capable of identifying the essential elements of the drawing and assessing their compliance with the protocol.

B. For Audio: Test, Iterate, Denoise

The biggest challenge of the project.

With recordings contaminated by multiple voices, we couldn’t train an in-house model from scratch.

We therefore:

• Tested Whisper, Parakeet, and other speech-to-text models
  
  
• Added a cleaning layer: noise removal, frequency filtering, and elimination of annotator speech
  
  
• Applied rules to ensure a recording was clean enough to interpret
  
  
• Developed specialized prompts to interpret MoCA responses (sequences, verbs, spatial orientation…)
  

Here, the main challenge wasn’t AI itself, but the ability to repair and structure the data before even processing it.

The importance of specialized prompts (and clinical judgment)

For auditory tasks such as word repetition, interpretation depends on the MoCA protocol.

Questions include:

• Is a mispronounced word acceptable?
  
  
• Is a verb inversion considered an error?
  
  
• If the sentence is complete but not exactly correct, is it acceptable?
  

We therefore designed highly detailed, tailored instructions that captured the clinical logic.

This went beyond AI engineering; it required a precise understanding of the clinical protocol to correctly interpret each response.

A key guideline: one model per task

A golden rule throughout the project: each MoCA task gets its own model—there is no generic model.

Why?

Because each sub-test of MoCA has:

• Unique rules
  
  
• Specific variations
  
  
• Distinct challenges
  

We trained and calibrated each model individually, which ensured far more stable performance and compatibility with future clinical validation.

The result: a reliable, validatable system

Despite the lack of data, we delivered:

• Models exceeding initial expectations
  
  
• Performance >90% on certain tasks from the first phase
  
  
• A robust audio pipeline
  
  
• Consistent visual scoring
  
  
• Architecture ready for regulatory validation
  
  
• Detailed documentation to support MoCA in the next clinical steps
  

Once cleaner data was available, performance quickly improved, with the architecture already in place to absorb these gains.

A tangible impact for MoCA

For MoCA, developing MoCA Solo represents more than just automating a test: it’s a scalable tool that doesn’t rely on hundreds of specialized annotators and provides more uniform scoring than human evaluation. This approach enables better longitudinal tracking and lays a solid foundation for regulatory validation, essential for safe clinical deployment. The result is a product ready for commercialization in a fast-growing global market.

For our team at Osedea, this project was an opportunity to demonstrate our expertise in a complex context. We were able to turn imperfect data into reliable results, adopt an iterative, pragmatic, impact-focused approach, and combine deep learning, synthetic data generation, and classical engineering. At the same time, we incorporated the constraints of medical products, reinforcing our ability to transform technical challenges into practical, operational solutions.

Key takeaways

Limited data could have blocked the MoCA Solo project. Instead, it pushed us to innovate and find creative technical solutions. We didn’t just train models—we built a full playground for them to learn in a clean, structured environment that complies with clinical standards.

With MoCA Cognition, we’ve shown that even in highly constrained contexts, a rigorous, technical, and collaborative approach can deliver reliable results ready to transform clinical practice. Read the full case study here.

If you want to explore how we can tackle your technical challenges and turn your ideas into practical solutions, contact us.