4SINAI: COVID-19 Chatbot
2020
Role: Design Lead (with one product manager and one tech lead)
In this case study, I will discuss co-leading the design of a health system-wide chatbot that triaged patients to the resources they need while making sure that medical professionals are able to connect with the patients in most critical need during the COVID-19 crisis in NYC.
We created automated chatbot logic to help patients assess their symptoms with 3 main possible resolutions:
Health system endorsed (i.e. trusted) online COVID resources;
Entering a video based, urgent care services;
Live chatting with a registered nurse on staff.
I did the visual/UX design in this project independently and the conversational design with my product manager.
Process
It started the first week of March when I returned from Seattle (just before it was about to become the first coronavirus hotspot). Over the course of 1 week in fact and we had to use what we already had, but make an experience that could handle a significant, sustained amount of volume. The most important piece here was clarity upfront of what we were trying to achieve.
My product manager and I worked hand in hand to create an experience using our existing chatbot functionality (up until this point had only been used for triaging low-risk seasonal flu patients and self-scheduling appointments). Though the chat experience has been (and remains) imperfect, it was the optimal solution for a few reasons:
Mobile accessibility: According to Pew Research, amongst lower income households, 71% have access to a smartphone, but only 54% have access to a laptop or desktop computer.
Internationalization: Our chatbot had built in language translation technology, creating an even better experience for folks
After identifying the best medium, we got to work on creating the bot flow (decision tree). This required daily reviews with the Ambulatory practice team, a group of project managers and clinicians advising us on changes to staffing (who would staff the live chat portion of our chat flow), updates from the telemedicine team (managing video visits and whether they could have patients directed to them), as well as rapidly changing guidance from the CIty of New York and the CDC around COVID symptoms. Over the course of 2 months, we made changes everyday around: symptoms triage, resolution endpoints, chatbot behaviors.
Conversational UX map we created
Case study: Queuing
After about 1 week of launching our COVID-19 chat flow, we were quickly informed by the Patient Access Services team that their staff was getting crushed under the weight of incoming chat messages with the added complexity that staffing couldn’t scale up or down on the turn of a dime. If 500 people wrote in at noon, there were still only 10 nurses on staff, just the same as if 100 people wrote in at 9PM. On the patient side, there was no indication of a “waiting state,” so a lot of our initial reports that I reviewed saw a steep rate of abandonment during the connection between stepping up to livechat and actually connecting to an agent.
We solved this by creating a dynamic waiting queue based on the number of nurses logged into the chat platform, so that position in line represented was accurate. This queue was designed to ensure no one was ever waiting in line fruitlessly and was rather directed as quickly as possible to the best option the health system has to offer given capacity.
Results
At the peak of the COVID crisis in NYC, our chatbot was fielding an average of 150 unique entries to our chatbot, with 50 stepping up to speak with a nurse a day. We were able to offset, 800 were diverted to a video visit rather than one of our physical hospitals.
From Epic reporting, we learned that we helped drive a 700% increase in video visit traffic, contributing to lighter hospital visit loads.
The COVID-19 chatbot remains in place as we speak though the volume is understandably considerably down given that NYC has done an amazing job getting the virus under control. With a bit of breathing room, our team is now looking towards the Fall, when COVID will likely get worse again, to this MVP and make significant experience improvements, both to the service (having a dedicated COVID chat response team) and the platform (better accessibility across mobile and desktop, plus added internationalization).
4SINAI: COVID-19 Chatbot
2020
Role: Design Lead (with one product manager and one tech lead)
In this case study, I will discuss co-leading the design of a health system-wide chatbot that triaged patients to the resources they need while making sure that medical professionals are able to connect with the patients in most critical need during the COVID-19 crisis in NYC.
We created automated chatbot logic to help patients assess their symptoms with 3 main possible resolutions:
Health system endorsed (i.e. trusted) online COVID resources;
Entering a video based, urgent care services;
Live chatting with a registered nurse on staff.
I did the visual/UX design in this project independently and the conversational design with my product manager.
Process
It started the first week of March when I returned from Seattle (just before it was about to become the first coronavirus hotspot). Over the course of 1 week in fact and we had to use what we already had, but make an experience that could handle a significant, sustained amount of volume. The most important piece here was clarity upfront of what we were trying to achieve.
My product manager and I worked hand in hand to create an experience using our existing chatbot functionality (up until this point had only been used for triaging low-risk seasonal flu patients and self-scheduling appointments). Though the chat experience has been (and remains) imperfect, it was the optimal solution for a few reasons:
Mobile accessibility: According to Pew Research, amongst lower income households, 71% have access to a smartphone, but only 54% have access to a laptop or desktop computer.
Internationalization: Our chatbot had built in language translation technology, creating an even better experience for folks
After identifying the best medium, we got to work on creating the bot flow (decision tree). This required daily reviews with the Ambulatory practice team, a group of project managers and clinicians advising us on changes to staffing (who would staff the live chat portion of our chat flow), updates from the telemedicine team (managing video visits and whether they could have patients directed to them), as well as rapidly changing guidance from the CIty of New York and the CDC around COVID symptoms. Over the course of 2 months, we made changes everyday around: symptoms triage, resolution endpoints, chatbot behaviors.
Case study: Queuing
After about 1 week of launching our COVID-19 chat flow, we were quickly informed by the Patient Access Services team that their staff was getting crushed under the weight of incoming chat messages with the added complexity that staffing couldn’t scale up or down on the turn of a dime. If 500 people wrote in at noon, there were still only 10 nurses on staff, just the same as if 100 people wrote in at 9PM. On the patient side, there was no indication of a “waiting state,” so a lot of our initial reports that I reviewed saw a steep rate of abandonment during the connection between stepping up to livechat and actually connecting to an agent.
We solved this by creating a dynamic waiting queue based on the number of nurses logged into the chat platform, so that position in line represented was accurate. This queue was designed to ensure no one was ever waiting in line fruitlessly and was rather directed as quickly as possible to the best option the health system has to offer given capacity.
Results
At the peak of the COVID crisis in NYC, our chatbot was fielding an average of 150 unique entries to our chatbot, with 50 stepping up to speak with a nurse a day. We were able to offset, 800 were diverted to a video visit rather than one of our physical hospitals.
From Epic reporting, we learned that we helped drive a 700% increase in video visit traffic, contributing to lighter hospital visit loads.
The COVID-19 chatbot remains in place as we speak though the volume is understandably considerably down given that NYC has done an amazing job getting the virus under control. With a bit of breathing room, our team is now looking towards the Fall, when COVID will likely get worse again, to this MVP and make significant experience improvements, both to the service (having a dedicated COVID chat response team) and the platform (better accessibility across mobile and desktop, plus added internationalization).