The Pixel Watch 3 keeps a finger on the pulse with help from AI

Season 7 Episode 1 - Loss of Pulse Detection Transcript

Jake 00:00:00 So you have this fairly significant fraction of people who really have no chance of survival because simply they were unlucky enough to have no one around. And so in that regard, the medical community people that we spoke with were really excited about this.

Voiceover 00:00:18 Welcome to the Made by Google podcast where we meet the people who work on the Google products you love. Here's your host Rachid Finge.

Rachid 00:00:25 Welcome back to the Made by Google Podcast. This is the start of season seven and just as ever we'll be talking to the people that make your favorite Google products. We're kicking it off with one of the most profound innovations. I know it's a feature that we launched in Europe a while back, but now that it has FDA approval, it's also available in the US. Usually we want people to interact with the things we make, but this is a feature I hope you'll never, ever need. Let's welcome Director of Product Management, Florence Thng and Staff Research Scientist Jake Sunshine, to talk about an industry first life-saving innovation.

Voiceover 00:01:01 This is the Made by Google podcast.

Rachid 00:01:03 Flo, Jake, loss of pulse detection. What kind of people were you thinking about while building this feature? Maybe Flo, we can start with you.

Florence 00:01:11 Thanks Rachid. A big realization when Jake and I and others in the team were discussing this is just like this will impact everyone. So this is like a feature that I would want everyone, all my loved ones and everyone's loved one to have access to because when this event happen and if the person is not with someone who could actually get them help, this gives them a chance so that they will get help hopefully in time. So I think of everyone that I love and everyone really

Rachid 00:01:44 And we'll get into how loss of pulse addiction might be helpful in a minute. Jake, anyone or any group you had in mind?

Jake 00:01:50 I think about my family, my extended family, my parents, my in-laws. This is an issue that can impact people of all ages, people who are fit athletes and people who are older. That's a big reason why we developed this. It's 'cause it can affect such a large number of people.

Rachid 00:02:08 Jake, you're a research scientist and a professor at the University of Washington. Right. So what's a day in your life like?

Jake 00:02:13 I'm really fortunate to work with a great team and we're tasked with trying to build the future safely and responsibly using these consumer devices to see if they can be helpful in health applications, wellness applications. And we really focus on applications to help keep people safer, especially when life may bring the unexpected.

Rachid 00:02:38 And Flo, you're a director of product management. You've been at Google for I think 14 years. Are there things you worked on before building lifesaving Fitbit? Features things we might have heard of before?

Florence 00:02:49 I've been fortunate enough to have the opportunity to work across different aspects of health at Alphabet for the last 10 plus years. The theme is always around like how we can bring together AI technology, innovative hardware and thoughtful product design that is informed by scientific and clinical rigor so that we can empower people to be healthier. It means like making it easier to know what's going on, making sense of the different health data, which can sometimes be really confusing. So for example, I worked on the AI model for detecting diabetic retinopathy, which is basically diseases that you can get if you are living with diabetes that impact your eyes and it costs a lot of people to be blind when it's preventable if you detect them earlier.

Rachid 00:03:35 So loss of pulse detection on Pixel Watch 3, let's get into it. It's been available in about a dozen countries in Europe. It's launched in the US now as well. Can you quickly explain what loss of pulse detection is and how it works? So let's say that I have a loss of pulse event then what will Pixel Watch do for me?

Florence 00:03:53 Loss of pulse is basically a first to market feature that we made available on Pixel Watch 3, it's supposed to detect pulselessness, which can be caused by a number of conditions. Could be cardiac arrest, respiratory circulatory failure, it could be medication overdose. It could really happen based on a number of condition and could happen to anyone. You could be like a young perfectly healthy athlete and this could happen to you. And we've actually seen some of these happening in tv. Could be like you are alone somewhere and not feeling well and happens to you as well. And the goal is the way this feature works is the watch will detect it and when it does, it starts a countdown which could alert the users because we designed this very thoughtfully. So it will actually start a countdown with haptics so that if it's a false positive, the users will be able to dismiss it and then the alert will actually become a loud noise alert as well. Again giving the users a chance to dismiss it. But in the case where it's a real event, this actually will conclude with the call to the emergency services line. So the EMS could actually be called and provide help.

Rachid 00:05:12 So of course nobody will be able to speak then to emergency services. So what kind of information do the uh, medical people get from Pixel Watch?

Jake 00:05:20 Before we decided on what that message would be, we spoke with EMS systems, people who run these systems, people who are dispatchers and uh, accept these 911 calls downs to people who are responding to these emergencies kind of who are on the trucks, you know, with sirens blaring. And so we got feedback on them about what message should be conveyed and ultimately landed on a message that stated that a person had a loss of pulse event and that they were not responding so they were sort of non-responsive based on not clearing the alert, not moving, not manually deescalating and then that message is conveyed as well as their location and then a dispatcher is capable of sending appropriate resources to respond.

Rachid 00:06:08 I'm so curious about the origin story of loss of pulse detection. How did it come about? Do you like remember the first conversation about a feature like this?

Jake 00:06:16 Well I guess I can, you know, go maybe a little bit farther back. You know, I'm a physician by training a board certified anesthesiologist. So I've taken care of a lot of people during vulnerable health moments when they're having surgeries in the intensive care unit and we rely on equipment in those settings to help keep people safe, monitor vital signs, things like that. And I've always been struck by a lot of the technology that we use in those sort of health critical moments was developed decades ago and it's very, very good. But that hardware is not as sophisticated or as capable of now what is available to wear on your wrist, what's available on your phone, on your tablets. And so I've always been struck by, well if we can use these sort of primitive technologies in these health critical contexts, why can't we use more accessible devices? And then assembling a team to figure out how we could do this sort of responsibility and effectively that would pass the scrutiny of regulatory agencies.

Rachid 00:07:14 So how does this feature work? I mean it sounds really simple. You detect that there's no longer a pulse and then uh, you call emergency services. Of course in reality it's not that easy. So how did you pull it off?

Jake 00:07:25 Yeah, it can seem really straightforward but it's actually really, really hard and let me sort of explain why. The first is a consumer device is both great because it's distributed through a large portion of the population so it has the capability to have an impact from a public health standpoint. The downside of that is you also have a surface that is connected to say a 911 resource that is a public resource that we all rely on and the nature of these signals that are measuring your pulse all the time, you have to be reasonably confident that something has changed so you're not overburdening these precious resources. And so developing an algorithm that can not just measure the transition from a pulsatile state to a AL state but have multiple gates and multiple checks to confirm that something truly is different and may be amiss and then trying to alert the user, trying to get them to respond.

Jake 00:08:30 Doing that just so to detect the event of interest without burning 911 or an emergency health system is really, really challenging to do. And so it required collecting data on, you know, first off hundreds of thousands of hours of people where we know this event didn't occur to make sure that we were not creating opportunities for errant calls. And then we also had to be uh, really thoughtful and creative and collaborative to try to mimic what these events could look like given that we can't just invite someone to a lab and have their heart stop.

Rachid 00:09:05 Yeah, talk a little bit about that. How do you test this feature without, you know, likely being unable to actually test the, the real world occurrence of it?

Jake 00:09:13 It requires collaboration with cardiology partners to work with them and their patients in settings that are previously scheduled for them to have certain elective procedures that we were able to have people wear a watch under informed consents when they were having those procedures. And this enabled us to get signals while these events were happening that were previously planned elective procedures to help understand that the algorithm that we had developed that was measuring someone didn't have a pulse was actually occurring during these procedures under medical supervision to help confirm that. So that was the really critical piece to this that helped us understand that what we were building actually does look like some subset of pulseless events that can happen when the heart's temporarily speeding.

Rachid 00:10:10 You know, when I take off my watch to charge it. Why does loss of pulse detection not trigger in that case? Because surely it doesn't see a pulse anymore, right?

Jake 00:10:19 It gets back to the point of why this is not not always easy, right? You can't be calling an emergency health service if uh, you've just taken off your watch. So we have developed an algorithm to first identify that the watch is on the wrist and then also looking for subtle signatures that confirm that someone is having a pulsatile state and may have transitioned out of a pulsatile state and therefore if you take your watch off and put it on the counter, we have several checks to make sure that that does not look like a pulsatile signal and that allows us to build something that's not gonna make an emergency call when someone takes a watch off and there's no pulsatile signal present.

Rachid 00:11:02 I'm wondering what the medical community said when you were working on this. Were they excited about it? Were they maybe a little bit afraid for false alerts? Well how did they feel about it?

Jake 00:11:12 So you have this fairly significant fraction of people who really have no chance of survival because simply they were unlucky enough to have no one around. And so in that regard, the medical community people that we spoke with were really excited about this as long as it was implemented responsibly. And a big part of that being done responsibly is first making music aware that this could be life saving, it will not catch everything. So being clear about that and then also making sure that given the nature of these devices and there are so many of them, that it doesn't lead to a drain on a public resource like 911.

Florence 00:11:53 We did it thoughtfully. The team actually reached out to emergency services very early on for their feedback. We also engaged with regulators very, very early on as well to figure out the best pathway so that we could do this safely and thoughtfully. I would just say also the reception after we were able to actually announce this has also been really humbling.

Rachid 00:12:17 Flo, you mentioned earlier Pixel Watch 3 is the first device out there, the first smartwatch with a loss of pulse detection. Why is that? Is there anything specific to Pixel Watch 3 that made that possible?

Florence 00:12:28 We are continuously investing in our health sensing technology. For example, Pixel Watch 3 actually have our most advanced optical package, which enable us to have things like our most accurate heart rate. It's also the foundation so that we could have very high quality signals that enable loss of pulse detection. So Jake and his team is able to like bring together the signal here, AI technology and implement it in a safe way.

Rachid 00:13:02 So for Pixel Watch and Fitbit have always prioritized health and safety. How does loss of pulse detection fit in in the big picture?

Florence 00:13:10 Our general goal is to help people live longer, healthier, life empowering people. So more broadly across our strategy, we've been committed to helping users stay healthy, uh, stay safe or many years. Some of our features like high low heart rate notification, personally super helpful for my mom who is going through some health issues right now. Irregular rhythm notification, there's also safety check for when you're out late or those moments that you just might feel unsafe. And we also recognize while you know, loss of pulse won't detect every loss of pulse event, we do believe that it, when it does, it could have the opportunity to make a difference. It could provide help for people when they couldn't get it themself.

Jake 00:14:00 I think uh, loss of pulse builds off of a lot of the safety work that's come before this features that contemplated being able to take a signal from the watch, mostly motion based falls, car crash, things like that, and being able to connect to 911. So developing that infrastructure to do that responsibly has definitely come before this.

Florence 00:14:24 I think the general theme is how can we actually provide peace of mind for users, right? Like they're wearing these devices, they don't have to actually pay attention to the data that's coming from it. We are paying attention in the background and we're analyzing and we're going to give them alerts when something happens. There's different levels of this. This is one of the features that we would call emergency services, but if we look at our portfolio, there are also other features that would give you a signs when something is off. I have a call today. My Health Metric dashboard actually gave me a notification that my metrics are outside my regular baseline couple of days ago. So I was able to just like sleep more, drink up on vitamins. So I think in general that's the theme that we want to help people.

Rachid 00:15:12 You created something that has the potential to save someone's life, someone you'll never meet maybe on the other side of the world. What does it feel like working on those kind of things?

Jake 00:15:23 It’s incredibly gratifying. You're speaking with Flo and I, but there are so many other people who've spent a lot of time trying to build this and that happens when you know you have a mission driven team and trying to save somebody's life who you'll never meet is, uh, is a really powerful mission and I can't tell you how much work's gone into this, how many meetings about placing this text or this button in this orientation to be better for users, more responsible with integrating into this, you know, emergency community and its infrastructure. Just so much work goes into it. It's a privilege and honestly it's not hard to do when you're, when you're working on something that could potentially save someone's life and that's what makes it so enjoyable.

Rachid 00:16:10 What does it feel like for you Flo?

Florence 00:16:12 It's humbling. I feel privileged to be working with Jake and so many others who are experts and I can't second Jake enough. I think this is definitely a passion project for many folks because it's hard working. Working in health is hard. Working in health at Google is extra hard because it's not what Google typically does, but I really appreciate the opportunity. It's a privilege and it is an area that Google's investing more and more because it does have an opportunity to drive so much impacts in people's life and it's just been amazing to be able to work on this with Jake and all the other people that are not represented today.

Rachid 00:16:58 We love to offer our listeners a top tip, something they should do or be aware of with the, the feature we're talking about today, which is of course loss of pulse detections. Is there anything they should do on their Pixel Watch 3 to ensure the feature is active or anything else uh, you'd like them to know?

Florence 00:17:13 Yeah, this is actually a feature that most people probably don't even think about it, but I will reemphasize that it's something that can happen to everyone and this is also a feature that needs to be activated. So I would really encourage all those that have Pixel Watch 3 to look into how to enable this. And if you have loved ones that you care about that you want them to have this peace of mind, this airbags basically get them a Pixel Watch so that you can have that peace of mind.

Rachid 00:17:45 Amazing. It's such a profound feature loss of pulse detection available in the US now and as well in 14 countries in Europe. Flo, Jake, thank you so much for your time and hope to see you soon again.

Jake Thanks so much Rashid.

Flo Thank you.

Voiceover 00:17:57 Thank you for listening to the Made by Google podcast. Don't miss out on new episodes. Subscribe now wherever you get your podcasts to be the first to listen.