Measuring heart rate with consumer ultra-wideband radar

Consumer devices are becoming increasingly capable, featuring various sensors useful for monitoring fitness and wellbeing. A few years ago, we launched sleep sensing in the Nest Hub, which used radar technology, called Soli, to analyze sleep patterns[da2d84] while the device is placed near the bedside. More recently, we showed that the frequency modulated continuous wave (FMCW) radar technology underlying the Soli radar platform can track vital signs like heart rate and breathing rate during sleep and meditation in a fully contactless manner.

Today, in “UWB Radar-based heart rate monitoring: A transfer learning approach”, we present new research showing that the ultra-wideband (UWB) technology, already common in many mobile phones, can be used for radar-based heart rate measurement. While UWB is widely adopted for features like secure vehicle unlocking and precise item location, its potential for radar sensing has been largely untapped. We demonstrate how this existing hardware can be leveraged for vital sign monitoring, such as measuring heart rate (HR).

The radar systems that have shown the most promise for vital sign measurement from consumer devices include millimeter wave frequency-modulated continuous wave (mm-wave FMCW) and impulse-radio ultra-wideband (IR-UWB) radar systems. Google’s previous advances on sensing sleep, motion and gestures on the Soli radar platform used FMCW technology. This meant that we already had extensive datasets, studies, and machine learning algorithms trained for these tasks, including for heart rate monitoring using FMCW radar.

Meanwhile UWB — a multipurpose technology that has grown in popularity and is increasingly available on many current mobile phone models and other consumer devices, also offers radar capabilities. The radar capabilities of UWB have been thus far largely untapped, with current UWB applications leaning more on non-radar uses like localization and tracking, vehicle unlock features, or data transfer.

Detecting HR in a contactless manner with radar is challenging because the tiny movements of the chest wall caused by the heartbeat are easily obscured by the far larger movements from breathing and general body motion. This is where the distinct nature of the radar signal comes into play. Its spatial resolution works in three dimensions, using both distance and direction to focus its measurement. This allows the radar to define a precise “measurement zone” around a person's torso. As a result, it can isolate reflections coming from the chest area while ignoring stationary background objects or movements occurring outside this zone. Simultaneously, its high temporal resolution samples the signal fast enough (up to 200Hz) to capture the subtle, rapid motion of the heartbeat itself. We developed a new method that makes optimal use of these unique 2-dimensional spatio-temporal properties of the radar signal to achieve highly accurate heart rate measurement.

We investigated if we could transfer the features learned from FMCW radar — where we had the benefit of large existing datasets and studies — to the UWB radar. The two radar systems operate using completely different physical principles. Mm-wave FMCW transmits a continuous sinusoidal wave whose frequency increases linearly with time, periodically sweeping a frequency range, while UWB transmits very short pulses with duration on the order of a few hundred picoseconds to a few nanoseconds. Our study is the first to show that learned features can be transferred between radar types for vital sign measurement. We chose heart rate as an initial task, for both its high potential utility and level of challenge.

To accomplish this task, we developed a novel deep learning framework designed to model the complex spatial-temporal relationships in radar signals for HR estimation. The architecture first uses a 2D ResNet to process the input data, in which one axis represents time and the other represents the spatial measurements. This initial stage is designed to extract features from the fine-grained spatio-temporal patterns created by chest wall movements.

Following this step, the model collapses the spatial dimension via average pooling. The resulting feature set is then fed into a 1D ResNet, which is designed to analyze the signal exclusively along the temporal dimension. This second stage identifies the longer-range, periodic patterns characteristic of a heartbeat from the features extracted in the first stage.

When trained with our FMCW dataset, the model achieves a mean absolute error (MAE) of 0.85 beats per minute (bpm) for heart rate measurement. This finding represents a substantial gain over prior state-of-the-art results on this dataset, halving the previous error rate.

We then ran a study that collected UWB radar data, along with electrocardiogram (ECG) and photoplethysmogram (PPG) data as our ground truth for heart rate, using a setup that placed the UWB radar sensor in positions where users typically hold their phone, i.e., on a table in front of them or on their lap. Compared to the FMCW dataset, which was 980 hours of data, the UWB radar dataset was much smaller, with 37.3 hours. As the UWB radar configuration was close to what is feasible on a mobile phone, with a much lower bandwidth, its range resolution was far lower than the FMCW dataset.

To ensure that our model was optimized to transfer to the UWB dataset, we retrained it after performing additional pre-processing steps to modify the mm-wave FMCW radar data to better resemble the target IR-UWB data, effectively lowering its range resolution. We then fine-tuned this model on the IR-UWB dataset, achieving an MAE of 4.1 bpm and mean absolute percentage error (MAPE) of 6.3%, a 25% reduction over the baseline error rate. Our baseline for performance on UWB radar was 5.4 bpm MAE and 8.4% MAPE, achieved by selecting the best model trained from scratch on our UWB dataset. With transfer learning, we enabled the UWB radar to meet the Consumer Technology Association standards for heart rate measurement for consumer devices: an accuracy of up to 5 bpm MAE and 10% MAPE.

To make sure our model is both accurate and reliable, we analyzed its performance across the various scenarios and user conditions captured in each dataset. For both types of radar, we found that performance on heart rate measurement is consistent in situations that were adequately represented. For example, on the FMCW radar, which collected data during overnight sleep sessions, the performance is maintained across various sleep positions and even when a person is moving between positions. For UWB radar, heart rate measurement is equally accurate for both tested device positions relative to the user — on a table in front of them or in their lap. For more details on this subgroup analysis and other results, see the full research paper.

Heart rate measurement is useful for a range of health, fitness, and wellness applications, offering fundamental insight into an individual's cardiovascular status and physiological responses across various health conditions. This demonstration of heart rate measurement could be a step towards using mobile devices to measure even more complex and subtle health signals from the heart and large blood vessels.

While wearable devices like fitness bands and rings have popularized continuous monitoring of health and fitness, the ability to measure heart rate in a contactless manner with consumer-device–grade radar sensors allows the benefits of this technology to reach a much wider audience of smartphone users. For this study, we focused on heart rate while sleeping (for FMCW) and on a setup where the radar sensor was in positions where the phone is usually held during use (UWB). As technology evolves, continuous monitoring could extend to various daily settings, integrating seamlessly into a user's routine activities.

This work moves us closer to enabling contactless heart rate measurement using consumer devices, especially as ultra-wideband (UWB) technology becomes more prevalent in mobile phones. Although our study did not include direct testing using mobile phones in a real-world setting, this research establishes the crucial groundwork for such future applications.

A core finding of this work is the demonstration that a model trained on one type of radar (FMCW) can be successfully adapted for another (UWB) to measure heart rate. This transfer learning approach is a significant step forward. It suggests a more efficient path for future research and development, where the foundational knowledge from existing, large datasets can be leveraged for new devices. Instead of starting from scratch with extensive data collection for each new piece of hardware, this method allows for a more streamlined process, accelerating the timeline for bringing such features to consumer devices.

This research is a result of a collaborative effort between Google Research and Google Platforms & Devices teams. We would like to thank our co-authors Sebastien Baur, Matthew Baugh, Mathias Bellaiche, Sharanya Srinivas, Octavio Ponce, Matthew Thompson, Pramod Rudrapatna, Michael Sanchez, Lawrence Cai, Tim Chico, Robert Storey, Emily Maz, Umesh Telang, Shravya Shetty, and Mayank Daswani for their significant contributions. We are also grateful to Abhijit Guha Roy, Michał Matuszak, Florence Thng, Yun Liu, and Shwetak Patel for their expert review. We also want to thank Tiya Tiyasirichokchai for designing the graphic for this post.