Background

Nowadays, monitoring vital signs, especially heart rate is one of the most fundamental aspects of health. Monitoring the heart rate is an aspect of health care, fitness, and well being. However, traditional heart monitors rely on physical contact, requiring patients to wear chest straps or finger-clip pulse oximeters. While these contact-based sensors are highly effective, many people don't have the equipment or can find it uncomfortable to wear.

Beyond general discomfort, the requirement for physical contact presents barriers to vulnerable populations. From people with severe skin burns, to premature infants in the NICU, adhesive or clamp based sensors can damage skin or cause unnecessary pain. Along with that, with the rapid rise of telehealth or remote medicine, doctors struggle to get accurate results from virtual assessments because patients lack the special equipment in their homes.

---

Methodology: The rPPG Pipeline

The objective of this project is to extract the human heart rate and respiration rate from a standard webcam by using Python. To accomplish this, the software processes the live video through four stages.

---

Phase 1: Facial Tracking and Data Gathering

Firstly, the software has to acquire the areas on the human face that have the strongest capillary blood flow. The Python software uses a computer vision library known as MediaPipe to impose a digital mesh on the human face to track specific coordinates. There are also more external libraries like OpenCV, NumPy and SciPy. The software isolates the areas on the human face that it believes will have the strongest blood flow. It does by isolating the Regions of Interest (ROI) like the Forehead, Left and Right Cheeks for blood flow detection, while the Shoulders and Chest are used to detect movement. For each frame in the video feed, the software processes the video 30 times a second, and records the average RGB values in the ROI.

An example of Face Tracking using Computer Vision.

Phase 2: Signal Extraction (CHROM Method)

Tracking the "red" in the human face is a highly inaccurate process, as room lighting and slight head movement create massive amounts of signal "noise." To achieve the objective, the software uses the Chrominance (CHROM) algorithm to process the RGB values. CHROM uses a specific mathematical ratio to compare the color values against each other. The mathematical formula for CHROM is 3R - 2G and 1.5R + G - 1.5B. This effectively cancels out the "specular reflection" (glare and shadows), while isolating the exact color shifts of blood volume changes beneath the skin.

Images showing how CHROM works.

---

Phase 3: Signal Smoothing and Bandpass Filtering

Even after the CHROM extraction is complete, the wave of data still appears jagged. The software applies a Savitzky-Golay filter, which acts as a digital shock absorber to smooth the signal into a clean wave without eliminating the actual peaks of the heartbeat. Then, a bandpass filter is implemented. Since a normal human heart rate doesn't just go down to 10 BPM and suddenly spikes up to 300 BPM while the person is at rest, this filter strictly eliminates any signal outside the biological range of 45 BPM to 140 BPM.

An example of a Savitsky-Golay Filter used in graphs to smooth out data.

---

Phase 4: Fast Fourier Transform (FFT)

Lastly, the signal is translated to a usable number. The smoothed wave is fed into a Fast Fourier Transform algorithm. FFT is a complex algorithm that analyzes a signal over time to find the dominant frequency within it. By finding the primary wave repetitive wave within the filtered signal, it is able to find the exact frequency of the user's heart rate. This is multiplied by 60 to show the Beats Per Minute reading on the live dashboard.

A visualization of how Fast Fourier Transform works.

---

Hypothesis...

If you can pull physiological signals from a facial video recorded by a normal camera and analyze them using rPPG and chrominance-based methods, you can determine that the resulting heart rates and breathing rates are statistically similar (within 5 - 10 BPM) to those obtained by a commercial wearable, e.g., Apple Watch.

---

Accuracy Check

I will use the Heart Rate to compare the commercial device (Apple Watch) to the BioMonitor software. In order for the BioMonitor to be accurate, the Heart Rate has to be within 10 BPM of the Apple Watch.

Here, I have collected my Data:

The control device (Apple Watch) recorded a mean heart rate of 64.6 BPM, while the independent variable (BioMonitor webcam software) recorded a average heart rate of 67.33 BPM. Furthermore, the absolute difference in the fifteen readings was precisely 6.866 BPM.

It is interesting to observe that the ideal situation in which the subject was perfectly still (Numbers 2, 3, and 14), the BioMonitor recorded near-perfect accuracy with a difference of 0 to 1 BPM. Readings that recorded the most amount of error (Numbers 10 and 12), could be cause by small movements or minor lighting changes in the surrounding environment that could have affected the CHROM signal array. However, this table of data shows that it is evident that the viability and accuracy of remote photoplethysmography with a standard laptop lens are high.

As a result, my BioMonitor can be comparable to other commercial devices.

---

Conclusion

The objective of this project was to determine if a non-contact, chrominance (CHROM) based software utilizing a standard laptop webcam could measure human heart and respiration rate as accurately as a commercial contact device, like an Apple Watch. Based on the data, the hypothesis I made was correct and is fully supported.

The threshold for success was at a maximum of 10 BPM or less. Across 15 readings over two minutes, the BioMonitor recorded an absolute average difference of 6.866 BPM based off the data given by comparison to the Apple Watch. Furthermore, during readings where the subject was perfectly still, the software achieved near-perfect accuracy to that of an Apple Watch with a difference of 0 to 1 BPM. While the software had disadvantages like micro-movements or lighting changes that it was sensitive to, the overall mathematical systems and pipelines did separate the human pulse from micro-movements.

Ultimately, this proves that remote photoplethysmography (rPPG) is highly viable technology. By removing the need for expensive equipment ranging from hundreds to thousands of dollars, like pulse oximeters or Apple Watches, software like the BioMonitor could make physiological monitoring more accessible for telehealth, burn patients, or remote communities. Overall, this technology is improving and growing more as time passes, making this tech more accessible to those who need it.

Citations...

On a side note, I have also put the code in the attachments sections if you're curious to how my code works.

What is respiration rate?

How to measure your respiratory rate - Mayo Clinic

Rapid shallow breathing: MedlinePlus Medical Encyclopedia

How does chest expansion work?

Human respiratory system - Mechanics, Lungs, Airways | Britannica

Mechanics of Breathing - Inspiration - Expiration - TeachMePhysiology.

What is heart rate?

Heart rate: What's normal? - Mayo Clinic.

What’s a Normal Heart Rate?

Basics of optical sensing:

Measurement Sensors: Specular vs Diffuse | Panasonic Industrial Devices

Molecular Expressions: Science, Optics and You - Specular and Diffuse Reflection: Interactive Java Tutorial.

What is rPPG?:

Understanding rPPG | Rouast Labs.

Advancements in Remote Photoplethysmography.

Remote Photoplethysmography Technology for Blood Pressure and Hemoglobin Level Assessment in the Preoperative Assessment Setting: Algorithm Development Study - PMC.

Camera Basics:

Frame rate - Wikipedia.

What Is Exposure? Exposure in Photography Explained

The Science Behind Camera Resolution and Image Quality

What Does Resolution Mean? Image Resolution Guide | Adobe

Wh

at Is rPPG? Remote Health Tracking Explained.

Basics of Movement Detection in Computer Vision:

Computer Vision

How Computer Vision is Shaping Everyday Life: From Deepfakes to Self-driving Cars

“Noise” Sources in Optical Systems:

Camera Noise Sources

What BPM is Considered Accurate?:

Accuracy and Systematic Biases of Heart Rate Measurements by Consumer-Grade Fitness Trackers in Postoperative Patients: Prospective Clinical Trial

Real World Applications of Non-Contact Monitoring:

Non-Contact Video-Based Neonatal Respiratory Monitoring - PMC

Continuous non-contact vital sign monitoring of neonates in intensive care units using RGB-D cameras.

Contactless Vital Sign Monitoring System for In-Vehicle Driver Monitoring Using a Near-Infrared Time-of-Flight Camera

Studies of Non-Contact Respiration Monitoring:

Non-Contact Monitoring of Breathing Pattern and Respiratory Rate via RGB Signal Measurement

Contact and Remote Breathing Rate Monitoring Techniques: A Review | IEEE Journals & Magazine

Non‐contact Laser‐based Human Respiration Rate Measurement | AIP Conference Proceedings

Non-contact heart rate monitoring by combining convolutional neural network skin detection and remote photoplethysmography via a low-cost camera

Non-contact Respiratory Rate Monitoring Based on the Principal Component Analysis

What is FFT?:

Fast Fourier transform - Wikipedia

Fast Fourier Transformation FFT - NTi Audio Solutions for Audio & Acoustics

What is a Salvitsky-Golay Filter?:

Savitzky–Golay filter - Wikipedia

Introduction to the Savitzky-Golay Filter: A Comprehensive Guide (Using Python) | by Thomas Konstantinovsky

What is CHROM?:

Application of Chrominance Based rPPG in Estimation of Heart Rate from Video Signal | IEEE Conference Publication

A comprehensive review of heart rate measurement using remote photoplethysmography and deep learning - PMC

• I would like to thank D. Maruyama, my science teacher, helping me through my journey through any confusions or questions I had.
• I would also like to thank my family for giving me the right ideas and the right support.
• Another thanks to AI's like Gemini for correcting my grammar, and for checking errors in my code.