What this is
- Kenzen's wearable device accurately predicts (T) using real-time physiological data.
- The study involved 52 trials with 27 subjects across various environmental conditions.
- The device aims to help workers avoid heat-related injuries by providing continuous, non-invasive T monitoring.
Essence
- Kenzen's wearable device can accurately predict in real-time, meeting established accuracy criteria across a range of conditions. This innovation offers a non-invasive solution for monitoring T, crucial for preventing heat-related injuries in workers.
Key takeaways
- Kenzen's algorithm achieved a mean absolute error (MAE) of 0.25 °C compared to gold standard measurements. This accuracy is maintained even at higher temperatures where heat-related injuries are most likely.
- The device remains accurate across various environmental conditions (13–43 °C) and physical activity levels, demonstrating its versatility for real-world applications.
- The algorithm was validated using data from both male and female subjects, ensuring its applicability across biological sexes.
Caveats
- The study population was relatively young, and further validation is needed for older individuals who may wear multiple layers of clothing.
- Accuracy was slightly compromised at lower temperatures, where the device tended to underestimate T.
Definitions
- core body temperature (T): The internal temperature of the body, critical for maintaining physiological functions and preventing heat-related illnesses.
AI simplified
1. Introduction
As climate change is not only increasing average temperatures globally, but also the frequency of extreme heat events [1], an increasing number of workers (e.g., military, construction, agriculture, etc.) will be exposed to these hotter temperatures on a more frequent basis [2,3,4,5,6,7]. Workers experience a 2% loss in productivity for each 1 °C increase in wet bulb globe temperature (WBGT) ≥ 24 °C [6], and in addition to productivity losses, the number of heat-related injuries and illnesses at job sites is on the rise. For example, in construction settings, the risk of heat-related deaths has increased since the 1990s, and is predicted to continue to increase unless heat-mitigation strategies are adopted at these sites [7]. One such heat-mitigation strategy is to monitor core temperature (TC) and alert workers when they reach temperature thresholds that predispose workers to heat-related injuries and illnesses (i.e., 38.2–38.5 °C; [8]). While these TC thresholds are ~1 °C lower than the TC at which heat injuries and illnesses typically surface, research shows that if workers stop and rest to hydrate and cool down at these lower TC thresholds, they will be able to work for prolonged periods of time in the heat without ever reaching these detrimental TC [6,8,9]. However, accurately monitoring individuals’ TC on a daily basis at the job site is prohibitively invasive (i.e., rectal and esophageal probes) or expensive (i.e., gastrointestinal pill). Thus, it has been a long-standing goal of many entities to develop a way to non-invasively estimate TC using other physiological parameters like heart rate, skin temperature, and heat flux [10,11,12,13,14,15,16,17,18].
As computational and machine learning approaches have become faster and more accessible over the last decade, there has been a renewed interest in algorithms designed to continuously predict TC (in real-time) based on various environmental and physiological parameters [17]. Various research groups have published TC algorithms that meet field-established accuracy standards through the use of easily obtainable physiological measurements (e.g., heart rate and skin temperature) collected during physical activity [10,11,12,13,14,15,16,17,18]. However, while model accuracy is high, many of these algorithms were trained and validated on datasets involving primarily young fit males [11,12,13,14,16,18,19], or only hot conditions [19], or with minimal data for ground truth TC ≥ 38.5 °C [10,14,16,19]—the temperatures above which heat injuries and illnesses are most prevalent, and thus accuracy is of utmost importance.
Another important factor to consider is that many of the previous TC algorithms have been designed using various laboratory-grade physiological monitoring devices like chest straps for monitoring heart rate or thermocouples for measuring skin temperature [10,11,12,13,14,15,16]. While these physiological measurements using laboratory-grade equipment are highly accurate, these sorts of measurements are typically cumbersome to implement daily as they can interfere with worker comfort. Additionally, each physiological parameter being measured often has a separate device and output (e.g., one device for heart rate and a separate device for skin temperature), which makes real-time integration of these measurements into a TC prediction difficult. As such, a core temperature-monitoring algorithm implemented in occupational settings must generally meet the following criteria: (1) small enough not to interfere with the individual’s work tasks, (2) waterproof/sweat proof, (3) continuously (non-invasively) monitor various physiological parameters throughout the day from a single location, (4) be intrinsically safe, and (5) meet the field-established accuracy criteria across a range of body temperatures and environmental conditions [9,17]. While it has been suggested that such TC algorithms could be implemented in a wearable device [17], we are only aware of one recent study that reported the reliability and validity of a TC algorithm to be implemented in a wearable device using heat flux sensors [20]. The authors concluded that while this wearable device provided reliable TC measurements, it did not provide accurate TC measurements [20]. Consequently, we are unaware of any studies showing TC can be accurately implemented into a commercially available wearable device.
Thus, the purpose of this study was determine whether the Kenzen wearable device could accurately predict TC continuously during physical activity across a range of environmental conditions. The Kenzen device is a wearable device worn on the upper arm, attached by Velcro straps, and is designed to meet the wearable device criteria for the occupational settings listed above. Equally important, another goal of this study was to address some of the limitations of previous studies by developing a non-invasive estimate of TC that meets field-established accuracy criteria for (a) men and women, (b) physical activity in a wide variety of environmental conditions, and (c) TC ≥ 38.5 °C.
2. Materials and Methods
2.1. Subject Characteristics and Experimental Design
The Kenzen device is a sweatproof and waterproof wearable device worn on the upper arm. The device has a photoplethysmography (PPG) sensor to measure heart rate (HR), along with temperature and relative humidity sensors on the inner and outer side of the device to measure skin-side and ambient-side temperatures and relative humidity (RH), respectively. Additionally, a 3-axis accelerometer provides step rate measurements. Temperature and humidity measurements are collected from the same single sensor, and accurately measure temperature and relative humidity within ranges of 0–90 °C and 0–100% RH, respectively, with a reported accuracy of ±0.2 °C and 2% RH, respectively.
For each trial, subjects set up a user profile through the Kenzen application (iOS and Android), where the onboarding process involves inputting their age, height, body mass, and biological sex, and answering a brief medical history questionnaire about previous heat-related injuries and/or illnesses and current medications they are taking to treat cardiovascular and/or neurological conditions. In this dataset, all subjects were free from any diseases or conditions that would affect thermoregulation, sweating, and/or core body temperature. Subjects were also free from any heat-related injuries or illnesses in the last six months. All subjects’ data were de-identified (via the universities and/or through the Kenzen platform) before being used retrospectively for the algorithm development in this manuscript; thus, according to the Advarra IRB, the research project did not meet the DHHS definition of human subjects research under 45 CFR 46, and therefore, did not require IRB oversight.
Twenty-seven subjects (19 males and 8 females) completed 52 exercise trials (14 trials completed by females; subjects’ mean ± SD (range) for age = 28.9 ± 7.8 y (21–62 y), body mass = 75.2 ± 9.9 kg (53–96 kg), height = 176.1 ± 9.1 cm (149–191 cm), body surface area = 1.9 ± 0.17 m2). On average, subjects completed ~2 trials (mean ± SD = 1.92 ± 1.17; median = 2); however, depending on the study and experimental design, several subjects completed only one trial, while one subject completed a total of six trials in various environmental conditions. Body surface area was calculated according to Dubois and Dubois equation [21]. None of the subjects were heat acclimatized or heat acclimated.
Data to train and test the Kenzen TC algorithm comprised 52 trials from four different studies (i.e., Study 1–4), where subjects wore the Kenzen device on their upper arm during physical activity while ground truth TC was collected either via a gastrointestinal pill (19 trials) or rectal probe (33 trials) according to standard protocols [22]. Specifically, studies involving the rectal probe used a standard depth of >10 cm, and those involving the gastrointestinal pill required subjects to take the pill right before going to sleep the night before the morning trial (i.e., >8 h but <12 h before trial start). Of these 52 trials, a total of 4036 min of data were collected from subjects during low- to high-intensity physical activity, ranging from 32–110% of the subject’s age-predicted maximum HR (mean ± SD = 71 ± 16% [23]), in a variety of environmental conditions from 13.4–43.2 °C (mean ± SD = 28.5 ± 7.2 °C), and trial lengths spanning 40 to 107 min (mean ± SD = 77 ± 19 min).
Study 1 (n = 5) involved 10 min seated rest, followed by two 25 min cycling bouts at 50–65% of age-predicted maximum HR, with a 5 min seated rest after each bout, and then a 15 min high-intensity cycling bout at >76% of age-predicted maximum HR followed by a 15 min seated rest. Study 2 (n = 14) involved 60 min of running on a treadmill, where for the first 45 min subjects ran at 70% of their ventilatory threshold and the last 15 min were a self-paced work trial. Study 3 (n = 15) involved a 10 min seated baseline followed by 60 min walking on a treadmill at 6 km/h with a 3% gradient, a 10 min rest, 15 min walking at 6 km/h with a 3% gradient, and a final 10 min seated recovery. Study 4 (n = 18) involved walking (on a treadmill) for 5 min at 3 mph with a 1% gradient, followed by two 15 min periods of running at a moderate intensity (64–76% age-predicted maximum HR), with 5 min of walking at the same speed and gradient in between periods. Subjects then sprinted (rating of perceived exertion of 17–19) for 15 s with 30 s recovery in between, continuing this sprint-recovery cycle until TC exceeded at least 38.25 °C (with a goal of reaching 39.0 °C). For Study 1–3, subjects wore minimal clothing layers (i.e., a t-shirt and shorts, or just shorts and sports bra/no shirt). For Study 4, subjects wore impermeable clothing to assist in elevating core temperature faster.
2.2. Data Preprocessing
Python version 3.8 with Jupyter Notebook were used for all data processing, algorithm development, statistical analyses, and graphics (including the built-in packages of pandas, matplotlib, glob, scikit-learn, scipy stats). The de-identified data were collected from all four studies (i.e., Kenzen data and ground truth TC), parsed, and time-aligned so that sampling rates were similar across variables. Data from the Kenzen device has sampling rates as follows: skin and ambient temperatures, along with skin and ambient relative humidity are collected every 5 s, while HR and step rate are collected every second. After the trials were completed, Kenzen device data were queried from an autonomous database where Kenzen data are stored. The ground truth TC values were collected at different sampling rates based on each study design (see above), ranging from 5 to 300 s apart. Due to variable sampling rates for the ground truth and Kenzen data, an interval of 1 min (60 s) was selected for aggregation and time alignment of all data. This meant that the Kenzen data were aggregated each minute, where the one-minute aggregation for temperature and relative humidity comprised 12 samples, and the one-minute aggregation for HR and step rate comprised 60 samples. For ground truth TC data where the sampling rate of the study exceeded 60 s, the Python interpolate() function was used to interpolate ground truth TC values to obtain a value each minute that would time-align with the Kenzen physiological data for that minute. These one-minute values were used to build the Kenzen TC model, and therefore, to calculate the model accuracy metrics (presented below) which compared the Kenzen TC vs. ground truth TC.
Once data were time aligned, we began feature engineering on the physiological data as inputs to the models. Kenzen HR, skin and ambient temperatures, skin and ambient relative humidity, and step rate were engineered in different ways for each one-minute aggregation, including transformations such as higher-degree polynomials, variable interactions, log transformations, rolling averages, minimum, maximum, median, standard deviation, etc. In order to select the best features for the Kenzen TC algorithm, an iterative process of feature engineering and feature selection were performed on each of these variables and tested separately for each model listed below (see below for details).
2.3. Algorithm Development and Accuracy Criteria
Previous research has used a variety of accuracy criteria to conclude that their algorithm is accurate compared to ground truth TC measurements, including a mean absolute error (MAE) ≤ 0.3 °C, mean bias < 0.1 °C, root mean squared error (RMSE) ≤ 0.35 °C, standard error of the measurement (SEM) ≤ 0.2 °C, limits of agreement (LOA) ± 0.58 °C, and/or a Pearson r correlation r ≥ 0.7 [12,14,19,24]. For algorithm development purposes (i.e., to identify which algorithm was performing best during the training phase), we used the criteria of MAE ≤ 0.3 °C to determine whether iterative modifications of our algorithm were improving model accuracy. However, all accuracy metrics were evaluated in the final model and are presented in the results below.
We used a common machine learning approach called leave one out cross validation (LOOCV) to split our dataset into training and test datasets and avoid overfitting [25]. Briefly, in the LOOCV method, one trial or subject is left out of the training dataset and used as the test dataset; the model is created on the training dataset and then tested on the one dataset that was left out. This process continues in an iterative manner, where each iteration leaves out a different trial. In this case, there were 52 iterations for any given model that was created (e.g., multiple regression with heart rate, skin temperature, and skin relative humidity). The MAE and mean bias from the 52 iterations of the test datasets are then averaged to give an overall model accuracy. In this manner, different models can be tested fairly quickly, where the best model has the lowest MAE. Once the best model was selected, the coefficients of all training iterations were averaged to get the overall TC model. Then, this overall model was implemented on all 52 datasets to obtain the accuracy metrics outlined in the results below.
Note that as each subject completed a different number of trials (ranging from 1–6, with a median of 2 trials), we completed this LOOCV process by leaving a single trial out each time or by leaving out an entire subject’s set of trials for each iteration (i.e., LOPO, leave one participant out). In doing so, we found no difference in the performance of the algorithm with either method; thus, we used the coefficients for the model based on the LOOCV where each trial was left out.
2.4. Machine Learning Models
The primary purpose of the algorithm was to predict the ground truth TC given the physiological data collected by the Kenzen wearable device and the physical characteristics of the subjects that were collected through the Kenzen app when subjects created a user profile. Several different machine learning models (described below) were trained and tested on the dataset using the LOOCV method (using sklearn LeaveOneGroupOut, where the groups input was the trials), and the accuracy criteria listed above were used to determine whether each model’s performance was better or worse than the previous model. This iterative model selection phase continued until a model met the key accuracy criteria (outlined above); in particular, we wanted to ensure that these accuracy criteria were met when TC was ≥38.5 °C, because temperatures above these TC predispose workers to heat injuries and illnesses.
First, all features were checked for multicollinearity (using python’s statsmodels variance_inflation_factor (VIF)), where features with VIF > 5 were excluded. For each of the models that were tested, feature selection was completed using python packages and functions like the ordinary least squares (OLS) statsmodel package, where variable coefficients with p ≤ 0.05 were selected as being significant predictors in the models (e.g., backward regression). After obtaining the features from the OLS package, the SelectKBest function from the sklearn package was used with the f_regression scoring to further validate that these features should be implemented in the model moving forward. Additionally, we tried other types of machine learning models, such as Lasso and Ridge regressions, XGBoost regressor, Random Forest regressor, and an Extended Kalman Filter (EKF) model, all using the LOOCV method to obtain the model accuracy and select the best model.
It should be noted that while this problem may be approached using deep learning models, there were two main limitations in taking this approach. First, in the field, many workers do not carry phones with them and/or are not near Wi-Fi throughout the day, and so the Kenzen TC algorithm must be implemented on the device firmware and utilize as little memory as possible. Deploying the algorithm on the device firmware ensures that workers will be alerted in real-time when their TC is too high. Second, many of the workers at these sites have inconsistent work schedules (e.g., work for 2 weeks, off for 2 weeks), and so traditional deep learning models that rely on 24 h wear time for continual learning were not possible, and therefore these models would likely not be as accurate compared to simpler machine learning models.
2.5. Statistical Analyses
To evaluate the performance of different Kenzen TC models compared to ground truth TC, different statistical metrics and plot-based representations were used to identify the best-performing model. These statistics were first calculated on each one-minute datapoint for each individual trial (e.g., calculated MAE for a given subject’s trial for a specific category), and then all the one-minute values were averaged across that subject’s trial. Next, the averages from all 52 trials were averaged to give an overall performance value for that model. Each metric—MAE, RMSE, SEM, Pearson r correlation, and mean absolute percent error—were all calculated in the same manner, and the details of these calculations are described below.
The MAE was calculated by taking the absolute value of the difference between the Kenzen TC and the ground truth TC for each minute, and then obtaining the mean from all minutes for that trial. The RMSE was calculated by squaring each minute-based residual, and then calculating the square root of the mean of the residuals for that trial. The Pearson r correlation coefficient was also used to compare the ground truth TC with the predicted Kenzen TC using the pearsonr function from the scipy.stats package in python to obtain a Pearson r correlation coefficient for each trial. The mean absolute percent error (MAPE) was calculated by taking the mean absolute error for each one-minute timepoint and dividing this value by the ground truth TC at that minute, and then taking the average of all one-minute values and multiplying by 100 to obtain a percentage for each trial. Then, all 52 trials’ MAPE was averaged to obtain a final percent error value. The standard error of measurement (SEM) was calculated by taking the standard deviation of all residuals for all minutes in each trial; the SEM from each trial was averaged to get an overall mean SEM.
A modified Bland Altman plot was used to identify any patterns or biases in the residuals across the range of ground truth TC. The mean bias was calculated by taking the mean of all of the one-minute errors overall, where ground truth TC was subtracted from Kenzen TC. Thus, a positive mean bias indicates the Kenzen TC overestimated ground truth TC, and a negative mean bias indicates the Kenzen TC underestimates ground truth TC. The 95% LOA were calculated as 1.96 × standard deviation of all errors.
3. Results
3.1. Model Selection
The best-performing models were the random forest regressors and the EKF. However, given the feasibility of implementing the EKF (vs. the random forest model) on the device firmware, the EKF model was selected. In the EKF model, the initial seed value (i.e., initial core temperature value) can be set at a standard TC value (e.g., 37 °C), or it can be set based on the learning for that specific person. Through various iterations of seed values, and then evaluating the model performance, we found that the seed values that led to the most accurate EKF models were based on a separate linear regression model where biological sex was the sole input. Thus, the final Kenzen TC algorithm is an EKF which includes a combination of each individual’s real-time physiological data (collected through the Kenzen device) and their user profile data (entered initially into the Kenzen application). The algorithm continuously predicts TC each minute the person is wearing the device.
To implement the final algorithm into the device firmware, the model coefficients were rounded down to five floating points to minimize the complexity and optimize the memory requirements of the device. Before implementation, we re-ran the statistical analyses to confirm that the rounding of coefficients to five decimal places did not affect model accuracy. The model was then translated into an assembly level C code to be implemented on the device firmware.
3.2. Model Accuracy Based on Ground Truth TRanges C
The data outlined below demonstrate Kenzen’s TC algorithm accuracy for the 52 trials. Table 1 shows the Kenzen TC algorithm is accurate at each core temperature bin ranging from 36.24–40.20 °C, as it meets the field-established accuracy criteria where MAE ≤ 0.3 °C. From TC > 37.0 °C, the Kenzen TC meets the accuracy criteria for mean bias, LOA, and RMSE (<0.1 °C, ±0.58 °C, and ≤0.35 °C, respectively). The MAPE is also <1%, SEM < 0.2 °C, and r > 0.7 for all TC bins. Figure 1 and Figure 2 further demonstrate Kenzen TC algorithm accuracy.
Figure 1 shows the modified Bland-Altman plot, where data fall within acceptable LOA, and the overall mean bias is <0.1 °C (i.e., 0.08 °C). In designing the algorithm, we purposefully chose to slightly overestimate TC (versus underestimate TC), as the goal is to protect workers from heat-related injuries and illnesses and alert them when their TC is exceeding safe levels (Figure 1 and Figure 2). Thus, alerting a worker too soon is preferable to missing an alert or underestimating TC. That being said, the errors are similar across TC ≥ 37 °C (i.e., there is no pattern in the residuals in the Bland–Altman plot); and especially at higher core temperatures (≥38.5 °C), the algorithm remains accurate and is well within the field-established standards for MAE, RMSE, LOA, and mean bias. Thus, at high TC where heat injuries and illnesses are most prevalent, the Kenzen TC is highly accurate.
Modified Bland-Altman plot showing ground truth Tvs. Kenzen device T. All 52 trials (i.e., 4036 min of data) are shown here. At T≤ 37.0 °C, the Kenzen device slightly overestimates T; however, at T≥ 37.0 °C, there is no discernable pattern in the residuals, and the algorithm meets pre-established accuracy criteria compared to ground truth T. The 95% LOA on the figure include the mean bias (i.e., mean bias ± 1.96 × SD). Mean bias = 0.07 °C. C C C C C C
Kenzen Talgorithm performance vs. ground truth T. () Correlation between ground truth and Kenzen Tfor all 52 trials (4036 min of data;= 0.94,0.001). Dotted line represents a correlation of 1.0, showing that the Kenzen algorithm tends to slightly overestimate Tat higher core temperatures. () Histogram showing the mean error distribution for all data based on one-minute averages of Kenzen Tminus ground truth T(rounded to one decimal place). ~70% of data had an MAE ≤ 0.30 °C, the pre-established accuracy criteria. The histogram also shows that the Kenzen Ttends to slightly overestimate (vs. underestimate) ground truth Tto protect workers from heat-related injuries and illnesses. C C C C C C C C a b r p <
| Ground Truth TRange (°C)C | Total Min of Activity | PearsonCorrelation Coefficientr | RMSE (°C) | MAE (°C) | Mean Bias ± LOA (°C) | SEM (°C) | MAPE (%) |
|---|---|---|---|---|---|---|---|
| 36.2–37.0 | 355 | 0.71 | 0.2 | 0.19 | 0.23 ± 0.55 | 0.02 | 0.52 |
| >37.0–38.0 | 2040 | 0.89 | 0.32 | 0.29 | 0.07 ± 0.53 | 0.09 | 0.78 |
| >38.0–39.0 | 1331 | 0.88 | 0.26 | 0.24 | 0.04 ± 0.57 | 0.03 | 0.62 |
| >39.0–40.2 | 310 | 0.79 | 0.16 | 0.15 | −0.03 ± 0.41 | 0.01 | 0.38 |
| All data | 4036 | 0.94 | 0.3 | 0.25 | 0.07 ± 0.62 | 0.18 | 0.67 |
3.3. Model Accuracy for Each of the Four Studies
Each of the four studies had different experimental protocols which involved varying combinations of walking, jogging, running, sitting, and stationary cycling across a range of environmental conditions. This variability in experimental design led to different mean ground truth TC and exercise intensities for each of the four studies. Based on the accuracy criteria of RMSE, MAE, and Pearson r correlation, Table 2 shows that the Kenzen TC algorithm meets the accuracy criteria for Studies 2–4 but just misses the criteria for Study 1. This is likely because Study 1 had a small sample size (n = 5) and involved cycling instead of walking/jogging/running. Still, Study 1 still shows strong accuracy as the mean bias is <0.1 °C, the SEM < 0.1 °C, and Pearson r = 0.90. Figure 3 shows representative plots from each study.
Time-averaged plot of Kenzen vs. ground truth Tfor each Study 1–4. These plots are a visual representation of the Kenzen T(orange) vs. ground truth T(black). Trials from each Study were time-aligned, where the trial start was minute 0, and each point represents the minute-based average of all trials within that specific study. Error bands (light orange and gray shading around the mean lines) represent the 68% confidence interval (±1 SD). Study 1 (), Study 2 (), Study 3 (), Study 4 (). C C C top left top right bottom left bottom right
| Study | Total Min of Activity in Range | Mean Ground Truth T(°C)C | Mean % Max HR | PearsonCorrelation Coefficientr | RMSE (°C) | MAE (°C) | MeanBias (°C) | SEM (°C) | MAPE (%) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 492 | 37.54 | 59.9 | 0.9 | 0.36 | 0.33 | −0.06 | 0.04 | 0.88 |
| 2 | 836 | 38.4 | 88.4 | 0.98 | 0.27 | 0.23 | 0.16 | 0.27 | 0.6 |
| 3 | 1454 | 37.53 | 60.7 | 0.94 | 0.29 | 0.25 | 0.06 | 0.08 | 0.67 |
| 4 | 1254 | 38.09 | 76.8 | 0.95 | 0.31 | 0.25 | 0.08 | 0.24 | 0.66 |
3.4. Model Accuracy across Heart Rate Zones and Environmental Temperatures
It was also important to ensure that the Kenzen TC algorithm met the field-established accuracy criteria during a range of low- to high-intensity physical activities typical of manual labor jobs. Table 3 shows that the Kenzen TC algorithm is accurate for light to vigorous HR zones; however, during lighter HR zone activities (e.g., sitting), Kenzen TC slightly underestimates ground truth TC, as the mean bias, MAE, and Pearson r correlation coefficient slightly exceed the pre-determined accuracy criteria.
Moreover, it was important that the Kenzen TC algorithm also passed all accuracy criteria during activities spanning a wide range of environmental conditions. In this case the temperatures and relative humidity ranged from 13–43 °C and 11–75%, respectively. Table 4 shows that the accuracy criteria are met for MAE, RMSE, mean bias, and Pearson r correlation coefficients across a wide range of temperatures, demonstrating that the Kenzen TC algorithm is accurate in a range of environmental temperatures and humidity.
| HR Zone | % of Max HR | Total Min of Activity | PearsonCorrelation Coefficientr | RMSE (°C) | MAE (°C) | Mean Bias ± LOA (°C) | SEM (°C) | MAPE (%) |
|---|---|---|---|---|---|---|---|---|
| Lighter | ≤57 | 737 | 0.58 | 0.3 | 0.28 | −0.15 ± 0.59 | 0.03 | 0.76 |
| Light | >57 to ≤64 | 687 | 0.89 | 0.28 | 0.27 | 0.00 ± 0.65 | 0.03 | 0.72 |
| Moderate | >64 to ≤75 | 979 | 0.87 | 0.27 | 0.25 | 0.12 ± 0.65 | 0.04 | 0.67 |
| Vigorous | >75 | 1633 | 0.94 | 0.27 | 0.24 | 0.16 ± 0.50 | 0.12 | 0.63 |
| TRange (°C)a | TMean ± SD (°C)a | RH Mean ± SD (%) | Total Min of Activity | PearsonCorrelation Coefficientr | RMSE (°C) | MAE (°C) | Mean Bias ± LOA (°C) | SEM (°C) | MAPE (%) |
|---|---|---|---|---|---|---|---|---|---|
| ≤20.0 | 17.3 ± 2.2 | 30.4 ± 15.2 | 645 | 0.91 | 0.3 | 0.25 | 0.05 ± 0.64 | 0.2 | 0.65 |
| 20.1–30.0 | 27.2 ± 2.5 | 40.7 ± 15.3 | 2121 | 0.95 | 0.27 | 0.23 | 0.10 ± 0.56 | 0.21 | 0.61 |
| >30.1 | 37.2 ± 3.7 | 33.4 ± 11.2 | 1147 | 0.91 | 0.34 | 0.31 | 0.06 ± 0.69 | 0.07 | 0.81 |
3.5. Model Accuracy When Comparing Gastrointestinal Pill vs. Rectal Probe Measurements
As there were two types of equipment used to collect ground truth TC measurements (i.e., 19 trials with the gastrointestinal pill and 33 trials with the rectal probe), it was important to demonstrate the Kenzen TC algorithm was accurate when compared to both methods of the gold-standard TC measurements. Table 5 shows that the Kenzen TC algorithm is accurate compared to both types of ground truth measurements based on the MAE, mean bias, Pearson r, and RMSE criteria.
| Type of Ground Truth TMeasureC | Total Min of Activity in Range | PearsonCorrelation Coefficientr | RMSE (°C) | MAE (°C) | Mean Bias ± LOA (°C) | SEM (°C) | MAPE (%) |
|---|---|---|---|---|---|---|---|
| Rectal Probe | 2708 | 0.94 | 0.3 | 0.25 | 0.06 ± 0.63 | 0.17 | 0.67 |
| GI Pill | 1328 | 0.96 | 0.29 | 0.26 | 0.07 ± 0.62 | 0.21 | 0.67 |
3.6. Model Accuracy Based on Biological Sex
Although there was not an equal number of males and females in this study, a large percentage of trials were completed by females (i.e., 27% or 14 out of 52 trials). The comparisons for males vs. females were as follows: mean bias ± LOA = 0.07 ± 0.60 vs. 0.07 ± 0.69 °C, MAE = 0.24 vs. 0.28 °C, RMSE = 0.29 vs. 0.32 °C, and Pearson r = 0.96 vs. 0.92. Thus, the Kenzen TC algorithm met the pre-established accuracy criteria for both sexes.
4. Discussion
The aim of this study was to demonstrate that an algorithm to predict core body temperature could successfully be implemented on a wearable device and produce accurate non-invasive TC estimates in real-time. Here, we show that the Kenzen wearable device can accurately predict TC compared to gold-standard measurements of TC via a rectal probe or gastrointestinal pill. Importantly, the Kenzen algorithm maintains high accuracy as TC increases above the point at which heat injuries and illnesses become more prevalent (i.e., ≥38.5 °C). To maintain strong accuracy at higher TC (when workers are at greatest risk of heat-related problems), accuracy is slightly comprised at lower core temperatures, where TC is underestimated (i.e., at rest, TC range of 36–37 °C). Furthermore, the Kenzen TC algorithm remains accurate throughout a wide range of environmental conditions, for both biological sexes, and for physical activity spanning light to vigorous HR zones.
Overall, Kenzen’s TC accuracy is similar to, and in some cases, more accurate than previously reported TC algorithms. Seng et al. (2016) also used an EKF model to predict TC and reported a mean bias of 0.11 °C (vs. Kenzen = 0.08 °C) and an RMSE of 0.29 °C (vs. Kenzen = 0.30 °C) [19]. Seng et al. reported that ~87% of their data fell within ≤0.3 °C MAE, while we show that ~70% of Kenzen data fall within MAE ≤ 0.3 °C (Figure 2). This slightly higher accuracy reported by Seng et al. may be due to the fact that they built and tested their algorithm only on men and within a tight ambient temperature range (30–35 °C), whereas Kenzen’s dataset included women and a much wider temperature range (13–43 °C). Moreover, Kenzen’s algorithm demonstrates a slightly stronger correlation with ground truth TC than Seng et al. (2016): 0.94 vs. 0.89, respectively [19]. Although the Kenzen TC algorithm and Seng et al.’s (2016) algorithm are not the same, they both use an EKF to predict TC from commonly measured physiological variables. The performance is similar between these two algorithms; however, the Kenzen algorithm was built and tested on a wider variety of environmental conditions, women, and a larger range in ground truth TC.
Buller et al. (2013) also have a TC algorithm that uses an EKF model. However, different to Seng et al. and Kenzen’s model, Buller et al.’s model only uses heart rate as the single input variable [12]. Buller et al. reported an overall mean bias ± LOA of –0.03 ± 0.63 °C, which is similar to Kenzen’s mean bias of 0.07 ± 0.62 °C. Buller et al.’s reported RMSE was 0.30 ± 0.13 °C, which is comparable to that of Kenzen (Table 1), and they reported a correlation coefficient of 0.84 for the observed vs. predicted TC, which is slightly lower than Kenzen’s correlation coefficient of 0.94. Although Buller et al. had a much larger test dataset (vs. Kenzen’s), which included a variety of clothing layers and acclimatization states, the training and test datasets only included one female whereas Kenzen’s train/test dataset was comprised of 27% females and a wider range in ages. The environmental conditions for the datasets from Buller et al. (2013) and Kenzen are similar: 13–43 and 9–45 °C, respectively, with slightly colder temperatures in the test dataset for Buller et al. (2013). As the Buller et al. algorithm only uses one input variable, HR, one question that might arise is why it would be necessary to incorporate more variables into the model if the accuracy is similar (between the Buller and Kenzen algorithms). As HR is susceptible to many acute physiological changes (e.g., caffeine, stress and anxiety, drugs and medications, fatigue, etc.), a model that incorporates multiple input variables (especially when they are easily measured simultaneously in a wearable device) will help minimize false alerts and therefore more accurately identify workers that are at core temperature limits where a break is required.
Kenzen’s overall MAE is 0.25 °C throughout a range of exercise intensities and environmental conditions, for both sexes, and a wide range in TC. Importantly though, the Kenzen TC algorithm maintains its accuracy (and even becomes more accurate) as the ground truth core temperature exceeds 38.5 °C. This is important because NIOSH recommends workers stop and take a break (to avoid heat injury and illness) at 38.2 and 38.5 °C if workers are unacclimatized or acclimatized, respectively [8]. In the case of manual labor work settings, where workers typically endure 8+ hour shifts and must stay safe in the heat, it is preferable to slightly overestimate TC (vs. underestimate) in order to protect these workers from heat-related injuries and illnesses throughout the day. As such, Kenzen’s TC algorithm was designed to slightly overestimate (vs. underestimate) TC at these dangerous TC ranges in order to protect workers, whereas previous algorithms or wearable devices have been shown to underestimate TC at these higher temperatures [20], which can put workers at even greater risk for heat injury and illness if solely relying on the wearable device for safety feedback. Another point to note is that this slight overestimation of core temperature at TC ≥ 38.5 °C may be even greater when compared to the gold-standard TC measurement of esophageal temperature, since gastrointestinal and rectal temperatures tend to be ~0.24–0.28 °C higher than esophageal temperatures during lower body exercise in the heat which causes a heat sink [22]. Of course, the magnitude of error between esophageal TC and Kenzen TC would also be dependent on the environmental conditions, type and duration of exercise, and therefore further research is needed to better understand how Kenzen TC compares to esophageal TC at these higher TC.
There are many strengths of the current study where the Kenzen TC algorithm was trained and tested. These strengths include an adequate sample size (52 trials with 27 unique subjects including females), a variety of exercise intensities within and among trials, a wide range of environmental conditions, and a dataset where ~40% of ground truth TC data were ≥38.0 °C. As a result, the Kenzen TC algorithm is equally accurate for both biological sexes, for a range of cool to hot environmental conditions, and maintains accuracy at higher TC which predispose workers to heat injuries and illnesses. Some limitations of the current Kenzen TC model include minimal clothing layers for most of the studies and a relatively younger subject population. Many of the worksites include older individuals (e.g., >50 y) and workers often wear 2–3 layers of clothing [3,4,5,6,9,27]. Thus, future research should validate the Kenzen TC algorithm in older adults and individuals wearing heavy clothing layers while working in the heat. Further research should also explore a core temperature algorithm that accounts for subjects being heat acclimatized vs. non-heat acclimatized. Based on previous research [28], if using an EKF model, it is likely that the seed value may need to be shifted to a lower starting point in heat-acclimatized (vs. unacclimatized) individuals.
5. Conclusions
In summary, this is the first study we are aware of to show that an EKF model can be successfully implemented in a wearable device worn on the upper arm to non-invasively and accurately predict TC continuously throughout work or exercise. We demonstrate that the Kenzen TC algorithm accurately predicts TC compared to ground truth measurements of TC via a gastrointestinal pill or rectal probe throughout a range of environmental conditions, exercise intensities, TC ranging from 37.0–40.2 °C, and for both biological sexes. Thus, the Kenzen device can be used to accurately monitor TC during work in cool to hot environments so that individuals can avoid heat-related injuries and illnesses via personalized alerts when an individual’s TC reaches unsafe levels.
Acknowledgments
The authors thank the many subjects that participated in this study. We thank Skip Orvis for his engineering support throughout this work & Priyanka Joshi for advice during the algorithm development phase. The authors also thank Cory Butts, Janessa Stephens, and Breana Cutler who executed the data collection for Study 4 and supplied the ground truth core temperature data to Kenzen.
Author Contributions
Conceptualization, N.E.M., R.C.B., B.T., L.A.H., O.J., T.M.; methodology, N.E.M., R.C.B., B.T., L.A.H., O.J., T.M.; validation, N.E.M. and R.C.B.; formal analysis, N.E.M. and R.C.B. investigation, B.T., L.A.H., O.J., T.M.; data curation, R.C.B.; writing—original draft preparation, N.E.M.; writing—review and editing, N.E.M., R.C.B., B.T., L.A.H., O.J., T.M.; visualization, N.E.M. and R.C.B.; supervision, N.E.M., T.M., O.J.; project administration, L.A.H. and B.T. All authors have read and agreed to the published version of the manuscript.
Funding
This study was funded by Kenzen Inc., Kansas City, MO, USA.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
No new data were created or analyzed in this study. Data sharing is not applicable to this article.
Conflicts of Interest
N.E.M. was a paid employee of Kenzen Inc. at the time of writing this manuscript and R.C.B. is currently a paid employee of Kenzen Inc. All Kenzen wearable devices were provided at no cost to the universities for data collection purposes. Study 1 was conducted by Kenzen employees and with Kenzen equipment and funds. Studies 2 and 3 were not paid for by Kenzen, and the researchers conducted the studies for alternate reasons (i.e., had completely different research questions unrelated to this manuscript or Kenzen interests); the researchers placed Kenzen devices on subjects for the trials, had them fill out the profile information in the Kenzen app, and uploaded ground truth core temperature data for Kenzen employees. For these two studies Kenzen wearable devices and technological support (troubleshooting app or device, etc.) were provided at no cost to the researchers. Study 4 materials and equipment (e.g., rectal probes, subject payments, etc.) was paid for by Kenzen; however, none of those researchers are authors on this manuscript. No Kenzen data were shared with any of the researchers for these studies as the researchers were not interested in the Kenzen data for their study purposes. L.A.H. was on a partial graduate scholarship from Kenzen during the time this study was conducted, but not during the writing of the manuscript.
Footnotes
References
Associated Data
Data Availability Statement
No new data were created or analyzed in this study. Data sharing is not applicable to this article.