Background
A large US pharmaceutical company asked the agency I was working at to explore a “future” project: an iPad app for medical professionals monitoring appointments with chronic pain patients.
The brief was to push what the technology of the time could do. This was 2018, well before AI was in everyone’s pocket, and the client wanted voice recognition, machine learning and touch controls working together to improve the experience for practitioner and patient both.
The work
The central idea worked like dictation. While a patient talks to their practitioner, the app listens in the background and writes out the whole conversation. A machine learning model then reads the transcript, identifies the emotion behind key words, and highlights them against Plutchik’s Wheel of Emotion. We liked the wheel so much it ended up driving the colour theming of the entire app.
Recording sensitive medical conversations raises an obvious question, and we didn’t dodge it. Every patient sees a consent screen before anything is captured, giving the practitioner explicit permission to use the recording and the data from that appointment.
The result
The finished concept went beyond note-taking. Practitioners can replay the appointment audio straight from the app, and tapping a word in the transcript highlights that moment for later. A 3D mannequin lets patients point to exactly where the pain is; that gets saved to their case for the next doctor who opens it.
We knew early on this one was special. It was built to kick-start a bigger R&D programme, and it did its job.
