Jonathan O'Donnell | 01.26.17 | Insights

How to Improve Healthcare with Voice Interface Design

  • healthcare
  • voice interface

What if every hospital room had an Amazon Echo in it? We recently applied a mix of healthcare experience and research to design meaningful voice interactions for the hospital inpatient setting, even if some are not technically feasible today.

In our research, we found a succinct framework, created by design company Punchcut, that breaks down voice interface design into four primary capabilities: command, dictation, agency, and identification. With this framework in mind, we imagined interactions that would benefit patients, providers, and facilities.

Below is an overview of these four capabilities, along with some practical applications for them in a patient’s hospital room.


Examples: “Echo, turn on the lights.” “Echo, play my snacks playlist.”

Just as it sounds, a voice command is a short, simple phrase that is matched to a small, finite number of structured responses by the voice interface. Among many applications, it has potential in the inpatient setting to improve the patient’s physical environment, the patient’s and provider’s requests for care, and the clinician’s recall of information.

Application: modifying the patient’s physical environment and improving their comfort, convenience, and accessibility. While patient satisfaction ratings may or may not be linked to any clear clinical outcomes, it is certainly a differentiator when a patient chooses between healthcare providers in today’s consumer age. Some Command examples could address:

  • Patient room comfort (e.g. adjusting the temperature, lights, TV, or hospital bed positioning)
  • Patient hunger (e.g. ordering food from cafeteria)
    • N.B.: The specific case of an inpatient ordering food brought to light an interesting observation by our research team. If we were to ask our Echo to read us an entire menu, it would take far longer than the process we were looking to simplify. Through this, we realized that commands can be complemented by physical artifacts to enhance usability. So if you already have printed menu, or one on a screen, you can just order away!

Application: a patient asking for additional, prompt care, thus freeing up time-strapped hospital staff to attend to more pressing needs of other patients. Currently, this is most often achieved by pressing a call-button on the side of the hospital bed, alerting the head unit coordinator, who then has to find and alert the patient’s specific nurse, who then has to decide whether to leave their current work to attend to this undefined patient need…well, you get the point! Some impactful command examples could speed up the appropriate provision of:

  • As-needed (or “PRN”) medications (e.g. ibuprofen for pain or Ambien for sleep)
  • Nursing assistance (e.g. helping to triage the importance of the request, such as addressing a beeping peripheral IV line vs. attending to bloody bowel movements)
  • Nursing aid assistance (e.g. supplying a bedpan vs. shifting one’s body habitus in bed)

Application: a physician entering a patient’s room in which the patient has taken a sudden and critical turn for the worse

  • Page the respiratory therapist or rapid-response team (to escalate care and assemble a team immediately to a particular room)

Application: aiding a clinician’s recall of specific and updated clinical information

  • “What were the Basic Metabolic Panel results from this morning?”
  • “What were the last five blood glucose readings?”
  • “How many milligrams of hydromorphone were dosed in the past 24 hours?”


Example: “Echo, I’d like to journal.”

In dictation, the user is speaking and the voice interface is recording their words for use at a later time. Among several applications, it has potential for both patients and clinicians.

Application for patients: journaling, as a means to increase mindfulness and reduce stress

Application for clinicians: recording dictated findings from history-taking, physical exam, or other clinical impressions that can be stored and sent to:

  • The electronic health record for later review, editing, and signing (similar to how scribe programs work)
  • The patient and/or their caretaker (in order to enhance their recollection and communication surrounding their health)


Example: “Echo, my leg hurts!”

A stellar voice interface will act as the user’s agent. That is, when provided with certain information by the user, the interface should be able to intelligently process that information to provide a pertinent, timely, and helpful response. Even deeper than responding to a command, a voice interface acting as an agent will provide utility to the user even if there was no clear command or “ask.”

Application: for patients in pain, a voice interface agent responds to “My leg hurts!” by processing current patient-specific medication orders, dispensing history, and suggest available medications or alternative therapies.

  • “Your next dose of ibuprofen is available at 7:00pm, two hours from now. Would you like to try a guided breathing exercise in the meantime?”

In this way, the voice interface interpreted the veiled report of a vague symptom (pain) and a command to “treat my pain!” and then searched through the active orders to recommend a potentially useful alternative and safe treatment given the lack of availability of medication orders.

Application: for clinicians ordering a medication, the system checks it against the patient’s listed medication allergies, hospital availability, and time to arrival at the bedside. For a clinician who orders a particular type of intravenous fluids that is not readily available in stock on the hospital floor,

  • “Your order for 500 mL of normal saline mixed with 40 mEq of potassium chloride will take approximately 2 hours to arrive on the floor. 500 mL of normal saline mixed with 20 mEq of potassium chloride is in stock on this floor. Would you like to administer it instead?”

This is a simple and perhaps trivial example, but one that indicates a greater awareness of the environment in which the clinician functions, and can help enable safe and speedy care delivery.


Example: “Echo, I would like my oxycodone now.” “This is a 37yo white male with history of….”

For a voice interface to identify a particular user from among the many possible voices it hears opens up many possible secure interactions. This technology already exists in some form. For instance, “At Vanguard my voice is my password” is one way Vanguard customers can log securely into their accounts over the phone. Among many possible applications, voice identification ensures secure access to personal information and tasks for both patients and clinicians.

Application for patients: to authenticate a true patient request, versus one of a family member or friend, for sensitive information.

  • “Am I HIV positive?” – should the patient be sleeping, this would prevent an intrusion on their privacy and sensitive information by a malicious guest

Application for clinicians: to automatically authenticate spoken commands or log desired information

  • “Prescribe colchicine 1.2 mg once now and 0.6 mg in one hour” – given recognition of their voice, the physician is able to enter a new order effective immediately for the patient that is authenticated and pertains to the appropriate patient (by room-device association).
  • “Cardiovascular exam: regular rate and rhythm, no rubs or gallops, Grade 3 systolic murmur heard best at the apex, normal S1 and S2….” – the physician (authenticated by their voice into their electronic health record user account) may dictate physical exam or other notes to review later, improving their workflow and saving time.

These inherent voice interface capabilities – command, dictation, agency, and identification – open up exciting possibilities into present and future applications of this very human technology to improving both patient and clinician satisfaction and productivity. If you’d like to see it in action, don’t forget to check out our video!

Careers at Smashing Boxes

Open Positions

We don’t just make great products, we help build great companies.

Contact Us

Get exclusive access to Smashing Boxes news, case studies, and events.

Sign up now
close ×

Get exclusive access to Smashing Boxes news, case studies, and events. Sign up now!

* indicates required