Short-term effects of a self-monitoring smartphone app on university students' mental health

Mental health problems are a growing public health concern worldwide. Around 14% of the global disease burden is attributed to neuropsychiatric disorders, which primarily include depression, alcohol-substance abuse, and psychosis [1]. Although the burden of mental health disorders spans across all age groups, the highest proportion of disability-adjusted life years occurs in adolescents and young to middle-aged adults (aged 10–29 years) [2]. An estimated 10–20% of children and adolescents experience mental disorders worldwide [3]. In addition, half of all lifetime mental disorders start by the mid-teens and three quarters by the mid-20s [4, 5]. Suicide is also a serious public health problem that affects many young people and is the second leading cause of mortality in people aged 10 to 24 years [6]. Suicidal behavior is closely related to the presence of mental disorders. A meta-analysis of psychological autopsy studies using a profiling method that reconstructs a person's psychological state before their death showed that 87.3% of suicide cases had been diagnosed with a mental disorder [7]. Given that mental disorders and suicide are major health problems for young people, enacting mental health measures for young people is an urgent issue.

Despite the widespread recognition of the importance of mental health promotion and prevention of psychiatric disorders in young people, the current evidence indicates that only a small proportion of young people with a mental disorder receive mental health care [8, 9]. Similarly, the World Health Organization reported that only 16.4% of the college students with mental disorders received a 12-month healthcare treatment for their mental disorders [10], which suggests that university students are perhaps hesitant to consult a psychiatrist or psychologist for their mental health problems. Furthermore, online surveys completed by college students at 26 campuses in the US showed that only half of the students with depression received adequate care [11]. There are several explanations as to why young people do not seek treatment for their mental health problems: 1) They do not pay attention to their mental health; 2) They are not aware of any consultation systems including medical services; 3) They are embarrassed to consult with others about their personal problems (fear of stigmatization); 4) They are too busy to consult with mental health specialists; and 5) They have restricted mobility due to physical conditions or apathy [12, 13]. One of the most cited barriers to treatment-seeking behavior is the stigma attached to mental health disorders and those who have received treatment for these issues. Furthermore, it has been suggested that the stigma attached to mental disorders is more pronounced in young people than the elderly [14]. Addressing these barriers to treatment could help university students consult with others about their mental health problems. Without treatment, students with mental health disorders are at a higher risk for lower grade point averages, school dropout, and unemployment than those without such disorders [15]. Thus, promoting early interventions for mental disorders could improve the quality of life of university students with mental health problems.

With the advances in information and communications technology, digital tools provide new avenues for changing health-related behaviors and managing chronic conditions [16]. In particular, smartphone applications (apps) are being widely used to support health care; it has been estimated that one-third of cellphone owners use mobile phones to obtain health information and 19% of smartphone users have at least one health app [17]. With the increasing use of mobile phones, mobile interventions have become a viable option to treat individuals with mental health problems. For young people, including university students, smartphones and apps are familiar tools that offer an effective and ubiquitous platform for delivering mental health information [18]. Many studies have examined the effect of smartphone apps for mental disorders, including depression, anxiety disorder, post-traumatic stress disorder (PTSD), addictive disorders (alcoholism and tobacco addiction), and developmental disorders [19 –22]. There are now a variety of apps available for different purposes, including cognitive behavioral therapy (CBT), self-monitoring, screening of mental disorders, psychoeducation, and mindfulness [23 –25]. Thus, smartphone apps have clinical relevance; however, currently, no mental health app with a user-centered design exists for university students.

To increase university students' mental health support, we independently developed a smartphone app called the 'Mental App', which was designed for university students according to the results obtained from a questionnaire [26]. This user-centered designed app was equipped with a self-monitoring and self-screening function for mental disorders aimed to tackle the aforementioned obstacles (i.e., lack of attention to their mental condition, a paucity of information about consulting systems, lack of knowledge about mental illness, and not having enough time for consultation). In the present study, using psychological tests, we examined the app's effect on the mental state of university students. Log data were also collected to assess the Mental App's usage, including the frequency and behavioral patterns. Finally, we conducted a questionnaire survey on the overall design impression and usability of the Mental App.

In total, 68 students participated in this study (app user group, n = 35; non-user group, n = 33). Eleven students did not participate in the post-study (app user group, n = 7; non-user group, n = 4). Therefore, we used data from the 57 subjects (app user group, n = 28; non-user group, n = 29; Fig 1). The demographic characteristics of the study participants who completed this study are shown in Table 1. There were no significant between-group differences in age, sex, academic degree, and major in the university.

First, we analyzed the log data to check how often the subjects used the Mental App (Table 2). For this log data analysis, we examined data from 35 app users, which included subjects who did not complete the post-study. The mean number of days spent using the app was 5.66 ± 3.16 (mean ± SD; Fig 3A). Forty percent of the subjects used the app for more than 7 days. The mean total screen time of the app was 9:03 ± 6:41 (min:sec). A histogram of the app's screen time is shown in Fig 3B. Eighty-nine percent of subjects used the app for less than 15 min. The mean number of total actions was 161.91 ± 107.34. Sixty-six percent of subjects tapped or swiped the screen 100 to 300 times (Fig 3C). All subjects used the self-monitoring function, and 23 out of the 35 subjects used the self-screening function (Table 2).

Table 3 shows the mean scores of all psychological tests. There were no significant between-group differences at either pre- or post-study time points (pre-study LSS score: app user vs. non-user, 28.9 ± 7.51 vs. 29.6 ± 5.19, p = 0.548; pre-study CES-D score: app user vs. non-user, 22.0 ± 12.8 vs. 16.2 ± 12.1, p = 0.070; pre-study GHQ-12 score: app user vs. non-user, 4.86 ± 3.54 vs. 3.10 ± 2.85, p = 0.056; post-study LSS score: app user vs. non-user, 28.8 ± 6.38 vs. 28.9 ± 5.98, p = 0.823; post-study CES-D score: app user vs. non-user, 17.6 ± 11.8 vs. 15.8 ± 11.2, p = 0.330; post-study GHQ-12 score: app user vs. non-user, 2.64 ± 3.00 vs. 2.93 ± 3.37, p = 0.826).

We also compared the test scores between the pre-study and post-study time points. There was a significant within-group difference in the CES-D score for the app user group (22.0 ± 12.8 vs. 17.6 ± 11.8, p = 0.004). There was also a significant within-group difference in the GHQ-12 score in the app user group, whereby the pre-study score was significantly higher than that in the post-study score (4.86 ± 3.54 vs. 2.64 ± 3.00, p < .001). There was no significant within-group difference in the LSS score in the app user group (28.9 ± 7.51 vs. 28.8 ± 6.38, p = 0.556). In the non-user group, there were no significant within-group differences for any of the test scores (non-user LSS score: 29.6 ± 5.19 vs. 28.9 ± 5.98, p = 0.365; non-user CES-D score: 16.2 ± 12.1 vs. 15.8 ± 11.2, p = 0.161; non-user GHQ-12 score: 3.10± 2.85 vs. 2.93 ± 3.37, p = 0.472).

Next, we compared the variation before and after the intervention (Δvalue) in each psychological test between the app user and non-user group to examine the effect of the Mental App on each psychological test. There was no significant between-group difference in the ΔLSS (-0.11 ± 4.28 vs. -0.59 ± 3.30, p = 0.496; Fig 4A). Similarly, there was no significant between-group difference in the ΔCES-D (-4.39 ± 7.13 vs. -2.07 ± 8.78, p = 0.143; Fig 4B). However, there was significant between-group difference in the ΔGHQ (-2.21± 2.23 vs. -0.17 ± 2.69, p = 0.007; Fig 4C).

To exclude the possibility that the difference in the ΔGHQ was caused by a regression to the mean, univariate ANCOVA was used to assess the effect of using the Mental App. The ANCOVA analysis found a statistically significant difference [F (1, 54) = 5.72, p = 0.020] between the app user and non-user group in GHQ-12. However, there were no significant differences between the app user and non-user group in the LSS [F (1, 54) = 0.11, p = 0.74] and CES-D [F (1, 54) = 0.12, p = 0.74] (Table 4).

Finally, we surveyed the users' impressions of the Mental App using a questionnaire. Most users gave the app a good evaluation (Table 5). For example, 92.8% of users were "slightly satisfied" or "satisfied" with the overall design of the app and 85.7% were "slightly satisfied" or "satisfied" with the app's operability. Furthermore, 60.7% of the users reported that they would like to continue using the app.

In the present study, we independently developed a smartphone app to provide and assess the mental health support of university students. This user-centered designed app was equipped with a self-monitoring function (daily user's condition: appetite, exercise, sleep, and mood), self-screening function (14 mental disorders), and a referral function that referred the users to psychiatrists or other consultation services. Furthermore, we examined the effect of the app on the mental health of students. The app usage improved the GHQ-12 score, although the subjects used the app for only 2 weeks. There were no improvements observed for the depressive symptoms (CES-D score) or the prejudice towards people with mental illness (LSS score). This indicates that the current app may provide effective mental health care, even in the short-term.

Some observational studies have examined the mental conditions of university students using a smartphone app; however, this is the first interventional study that independently developed an app for university students and evaluated its effect on their mental health. Sano et al. developed a new system that provided objective physiological and behavioral assessments using smartphones and wearable sensors [33]. In that study, the authors recruited 201 college students and collected data from wearable sensors (skin conductance, skin temperature, three-axis acceleration, activity levels, and sleep/wake patterns), a smartphone app (location, receivers, senders, and timings of calls and SMS text messages, the screen's on or off durations, and phone app usage), and electronic diaries with two entries per day. Furthermore, each student filled out self-reported questionnaires on stress and mental health before and after the study. The authors found that the data obtained from each device could identify stress levels and mental conditions of the college students with high accuracy; however, the study was observational and did not compare the efficacy of an app on mental health with a control group. Geyer et al. also developed an app for college students to examine how emotional ratings during social interactions later predicted the perceptions of those interactions from the viewpoint of social anxiety [34]; however, this was also an observational study. Using commercially available apps, Huberty et al. examined the efficacy of the mindfulness meditation app "Calm" on mental health in college students [35]. In this randomized controlled trial (RCT), the use of the app resulted in significant improvements in all outcomes, including perceived stress levels, mindfulness scores, and self-compassion scores.

Apart from research conducted in university students, many other interventional studies have reported improvements in psychiatric symptoms after using smartphone apps in other populations. For example, Mantani et al. evaluated the efficacy of a CBT app as adjunctive therapy in patients with antidepressant-resistant major depression. In that study, a medication switch plus CBT app group showed significant improvements in depression scores compared to a control group (medication switch alone) [36]. Arena et al. reported that a cognitive training app and problem-solving therapy app group exhibited greater benefits on mood than the control group [37]. Ivanova et al. investigated the efficacy of an Internet-delivered acceptance and commitment therapy-based treatment program using a smartphone on anxiety disorders; the treatment group had reduced general and social anxiety disorder, but no decrease in panic symptoms compared to the waiting-list group [38]. Overall, several interventional studies have examined the effect of CBT (in a broad sense) using smartphone apps and confirmed its efficacy [23, 39].

The main function of our app was the self-monitoring system (daily appetite, exercise, sleep, and mood), and our app did not include a CBT program. According to previous reports, it is still controversial as to whether self-monitoring by a mobile device is effective for reducing psychiatric symptoms. For example, Reid et al. conducted an RCT that examined the mental health benefits of self-monitoring using a mobile phone. The authors found no significant differences in depression, anxiety, or stress scores between an intervention and control group, although the use of the mobile phone program was significantly associated with an increase in the emotional self-awareness (ESA) score [40]. Faurholt-Jepsen et al. examined whether the use of daily electronic self-monitoring using smartphones could reduce depressive and manic symptoms in patients with bipolar disorder; however, there were no significant effects of self-monitoring on depressive or manic symptoms [41]. In contrast, an RCT conducted by Kauer et al. reported that self-monitoring mood, stress, and coping strategies by mobile phone increased ESA and that this ESA increase was associated with a significant decrease in depressive symptoms [42]. Bakker et al. examined the efficacy of a self-monitoring mobile phone app that provided daily prompts to complete a short mood questionnaire and found that app engagement ratings predicted decreases in depression and anxiety. In addition, the authors found that the app increased the mental well-being scale score and that these effects were mediated by increases in ESA [43]. In the present study, the CES-D scores decreased after using the app, although this decrease (Δvalue) was not significant compared to that in the non-user group. Furthermore, we found that the GHQ-12 score, which is a measure of current mental health, improved significantly in the app user group compared to that in the non-user group. Considering previous results from studies that have used a self-monitoring app, it is possible that our app improved the GHQ-12 score via an increased ESA. ESA is the ability to understand one's own emotions, which can improve emotional self-regulation [44]. A low ESA is common in both anxiety and depressive disorders [45]. In addition, given that a high GHQ-12 score is associated with psychiatric symptoms, including depression and anxiety, our findings of a decreased GHQ-12 score in the app users may reflect an increased ESA. Moreover, our app provided contact details for psychiatrists and other consultation services when necessary and this referral function may have also relieved the users' anxiety. However, there is not enough data to clarify by what mechanism our app improved the mental health of its users and this warrants further study.

In general, the cut-off point for CES-D is 15/16 and that for GHQ-12 is 2/3; however, the mean scores of these tests were relatively high in our studies. There may be two reasons for this discrepancy. First, all subjects in the present study were student volunteers and some students with mental health problems may have participated in the study. Students with mental health problems may have been interested in our research due to concerns regarding their psychological state. Therefore, students with mental health problems may have raised the mean score of these psychological tests. Second, cultural background in Japan may have also further impacted the results of this study. Several studies report a relatively high score of CES-D and GHQ-12 in Japan. For example, Doi et al. performed GHQ-12 on the Japanese general adult population (1,808 subjects) and found that young adults (20–29 years) exhibited relatively high scores compared to the elders (men: 2.77± 2.86, women: 3.88± 3.16) [32]. Ohtsu et al. evaluated the mental health status of 1,619 medical students and found the mean (± SD) score of male students to be 3.00 (± 2.94) and that of female students to be 3.82 (± 3.10) [46]. Concerning CES-D, a previous study examined the prevalence of depression among 2,200 Japanese people using CES-D and found the mean (± SD) CES-D score to be 16.09 (± 8.61) [47]. Furthermore, another study in Japan showed that 60.1% of college students exceeded the cut-off point of CES-D (≥ 16) [48]. Therefore, the psychological tests that examine mental problems may tend to score relatively higher in Japan.

We also examined whether the app could improve the students' stigma of psychiatric disorders. The app had a self-screening function for mental disorders that provided general information about each mental disorder. In the general information, the app explained that mental disorders are common and that anyone can develop a mental disorder; however, there were no differences in the LSS score between the app user and the control group. One RCT with 262 subjects found that E-learning improved the stigma against mental illness [49]. In that study, participants were sent weekly E-mails via personal computers for a month to help them complete an E-learning CD. In contrast, our app did not prompt students to use the self-screening function and only 23 out of 35 students spontaneously used this function. In addition, the duration of our intervention was shorter than this beforementioned study. Thus, it is possible that with a longer intervention time, we would have observed a larger anti-stigma effect. Given the scarcity of studies that have investigated the removal of stigma using digital devices, further research is required to confirm whether smartphone apps are a useful tool in this context.

Regarding adherence with smartphone apps for mental health, previous reports have been inconsistent. For example, Huberty et al. examined the effect of an 8-week mindfulness course in college students using a mobile app and only 78.6% (44/56) of the students completed the post-intervention survey [35]. Meanwhile, Mak et al. found that only 23.5% (508/2161) of the participants completed the assessment after the 4-week program [50]. Concerning the number of days spent using apps, Kenny et al. reported that on average, participants used the app on 4 out of 7 days in their study; they examined the efficacy of an emotional self-monitoring app [51]. Matthews et al. found that users of a mobile mood diary demonstrated significantly higher levels of compliance (mean 8.12 days) compared to a control group who charted their mood using a pen and paper (mean 5.44 days) [52]. In our study, 28 out of 35 subjects in the app user group (80.0%) completed the study and the average days spent using the app was 5.66 ± 3.16 days out of the 14 days of intervention. Considering that we did not prompt students to use the Mental App, adherence to our app is not necessarily poor compared to other apps used for mental health. That said, the conditions of the present study were different from those of previous reports. We developed the Mental APP based on a questionnaire survey conducted before this study and that may have resulted in good adherence as the app's design was heavily influenced by the user's input.

The present study has some limitations. First, this study was not an RCT, so the results may have been affected by the potential confounding factors. In our study, only iPhone users could participate in the intervention group (app user group) and other types of mobile phone users were allocated to the control group (non-user group). A previous study found that there may be minor differences in personality between iPhone and Android users and that such minor differences may reflect the differences in sociodemographic factors [53]. It is, therefore, possible that there were sociodemographic differences between the two groups that may have affected the present results. In the future, we would like to create an Android version of the Mental App to confirm the effect of the app on mental health in students and distribute this app to the university students of various backgrounds. Second, all participants were recruited from a single university. In addition, the sample size of this pilot study was relatively small and our findings may not be generalizable to the majority of the university students. Third, the duration of this study was too short to evaluate the efficacy of the Mental App. A longer observation period may reveal additional benefits or even adverse effects of our app. In particular, we think that a longer intervention time may be required for university students to experience changes in the stigma. Finally, the Mental App did not include any function that dealt with academic problems (e.g., academic performance, mild developmental disorders, harassment, and employment after graduating the university). In addition, students of various ages, including middle-aged graduate students, also attend the university. Therefore, it is necessary to further expand the app's functions to encompass such additional problems and diverse users at the university.

We independently developed a smartphone app for university students and the use of the app improved the GHQ-12 score. However, app use did not improve the CES-D or LSS scores. This pilot study indicates that a self-monitoring app could improve mental health in university students. Further studies are needed to evaluate the efficacy of our app, particularly rigorously controlled research designs such as RCTs, that have longer intervention periods. Given the mental health burden in young adults, smartphone apps could be a promising option for improving the mental health of university students.