Conversation-aware Virtual Patient for Mixed Reality Medical Training


Healthcare - Telehealth, Medical Software & Imaging
Infocomm - Speech/Audio Processing
Show more >


Virtual Reality/Mixed Reality (VR/MR) solutions involve high setup costs and a long development time; typically requiring between 4 weeks to 6 months just to construct a training scenario which is frequently prohibitively limited in scope and flexibility. With such long lead times, the application of scalable medical training is a challenge for the medical training industry - limiting access for many medical practitioners.

This technology offer is an AI-powered medical training simulation utilising mixed reality and virtual reality to improve healthcare training for healthcare workers. It enables the creation of a wider variety of medical scenarios via its virtual patient engine and understands the verbal responses of medical learners via a conversational AI engine which also recognises a dictionary of medical phrases and drug names that are relevant to a clinical summary.


Virtual Patient Engine:

  • Able to simulate rare disease/symptoms and various types of patient medical history, derived from real-world statistics
  • Physiological problems are reflected virtually i.e. elderly with back pain, young patient with abdominal discomfort etc.
  • Built-in reusability and effortless customisability
  • Each patient profile is imbued with a unique set of presenting symptoms, history of illness, drug history, family history, social history which can be tweaked in real-time and neccessitate a differentiated line of questioning - forcing learners to change the examination approach and effect a different learning outcome
  • Reacts to user's line of questioning, interactions and choice of management

Conversational Artificial Intelligence:

  • Multilingual
  • Understands the correct order and flow of realistic conversations
  • Identifies the intent of conversations between medical practitioner and patient
  • Recognises medical phrases and drug names that are a requisite part of clinical summaries


This technology supports the learning needs of medical practitioners in the following broad areas:

  • Experiential - translate textbook content to virtual reality
  • Interdisciplinary training - peer-to-peer interaction with tutors, colleagues from anywhere and on any device
  • Real-time scenario controller - create and modify scenarios in real-time, resulting in observable changes to the virtual patient   
  • Assessment - standardized formative evaluation to assess student capabilities and remove accessor bias 

At present, the following medical training scenarios are supported (although not limited to):

  • Healthcare literacy (Orton-Gillingham card drills)
  • COVID-19 swab test supervision
  • Clinical examinations
  • Ophthalmology examinations (ocular)
  • Physical examinations
  • Paediatric distress recognition

Unique Value Proposition

Compared to existing techniques, the technology is unique with a fully customisable virtual patient that is readily configurable with different physiological profiles, paired with a conversational engine that understands and orchestrates the virtual patient's responses to a medical practitioner's line of conversation. The following benefits can be obtained from this the standardisation of medical training through the use of this technology:

  • Improved clinical outcomes - repeatable observational learning of complex procedures for reinforced enhanced clinical knowledge, competency, and knowledge retention
  • Decentralises the classroom - moving away from fixed learning places to remote learning spaces
  • Scalable - wide spectrum of simulated pathologies and adjustable symptoms
  • Assesses learner's critical thinking and adaptability - inject scenarios to assess how learners analyse, manage and adapt to unpredictable and unexpected scenarios
  • Reduced setup/development time (by up to 95%) and cost

The technology owner is keen on technology collaboration with medical institutions, hospitals, medical device companies, deep tech companies and VR/MR game developers to co-develop new products/services. Additionally, the technology owner is interested in opportunities for test-bedding/clinical studies.

Automated 3D Models from CBCT Segmentation
Ultra-Thin, Stretchable and Sensitive Fabric Sensor for Sports Monitoring
High Speed and Sensitive Artificial Olfactory Sensor
AI-Aided Analysis of Capsule Endoscopy Images
Contactless Palm Biometrics for Person Authentication and Identification
AI-Assisted Image Labelling Tool for Large Scale Data Labelling Efficiency
Hyper Realistic Virtual Lighting System for Video Productions/Photography
AI-enabled Virtual Modelling for Reduction of Energy, Carbon Dioxide Emission
Deep Neural Network (DNN) Approach for Non-Intrusive Load Monitoring (NILM)
High-performant Vector Database for Artificial Intelligence (AI) Applications