Chiu DT, Solano JJ, Ullman E, Pope J, Tibbles C, Horng S, Nathanson LA, Fisher J, Rosen CL. The Integration of Electronic Medical Student Evaluations Into an Emergency Department Tracking System is Associated With Increased Quality and Quantity of Evaluations. J Emerg Med 2016;Abstract
BACKGROUND: Medical student evaluations are essential for determining clerkship grades. Electronic evaluations have various advantages compared to paper evaluations, such as increased ease of collection, asynchronous reporting, and decreased likelihood of becoming lost. OBJECTIVES: To determine whether electronic medical student evaluations (EMSEs) provide more evaluations and content when compared to paper shift card evaluations. METHODS: This before and after cohort study was conducted over a 2.5-year period at an academic hospital affiliated with a medical school and emergency medicine residency program. EMSEs replaced the paper shift evaluations that had previously been used halfway through the study period. A random sample of the free text comments on both paper and EMSEs were blindly judged by medical student clerkship directors for their helpfulness and usefulness. Logistic regression was used to test for any relationship between quality and quantity of words. RESULTS: A total of 135 paper evaluations for 30 students and then 570 EMSEs for 62 students were collected. An average of 4.8 (standard deviation [SD] 3.2) evaluations were completed per student using the paper version compared to 9.0 (SD 3.8) evaluations completed per student electronically (p < 0.001). There was an average of 8.8 (SD 8.5) words of free text evaluation on paper evaluations when compared to 22.5 (SD 28.4) words for EMSEs (p < 0.001). A statistically significant (p < 0.02) association between quality of an evaluation and the word count existed. CONCLUSIONS: EMSEs that were integrated into the emergency department tracking system significantly increased the number of evaluations completed compared to paper evaluations. In addition, the EMSEs captured more "helpful/useful" information about the individual students as evidenced by the longer free text entries per evaluation.
Torous J, Staples P, Fenstermacher E, Dean J, Keshavan M. Barriers, Benefits, and Beliefs of Brain Training Smartphone Apps: An Internet Survey of Younger US Consumers. Front Hum Neurosci 2016;10:180.Abstract
BACKGROUND: While clinical evidence for the efficacy of brain training remains in question, numerous smartphone applications (apps) already offer brain training directly to consumers. Little is known about why consumers choose to download these apps, how they use them, and what benefits they perceive. Given the high rates of smartphone ownership in those with internet access and the younger demographics, we chose to approach this question first with a general population survey that would capture primarily this demographic. METHOD: We conducted an online internet-based survey of the US population via mTurk regarding their use, experience, and perceptions of brain training apps. There were no exclusion criteria to partake although internet access was required. Respondents were paid 20 cents for completing each survey. The survey was offered for a 2-week period in September 2015. RESULTS: 3125 individuals completed the survey and over half of these were under age 30. Responses did not significantly vary by gender. The brain training app most frequently used was Lumosity. Belief that a brain-training app could help with thinking was strongly correlated with belief it could also help with attention, memory, and even mood. Beliefs of those who had never used brain-training apps were similar to those who had used them. Respondents felt that data security and lack of endorsement from a clinician were the two least important barriers to use. DISCUSSION: RESULTS suggest a high level of interest in brain training apps among the US public, especially those in younger demographics. The stability of positive perception of these apps among app-naïve and app-exposed participants suggests an important role of user expectations in influencing use and experience of these apps. The low concern about data security and lack of clinician endorsement suggest apps are not being utilized in clinical settings. However, the public's interest in the effectiveness of apps suggests a common theme with the scientific community's concerns about direct to consumer brain training programs.
Torous J, Staples P, Fenstermacher E, Dean J, Keshavan M. Corrigendum: Barriers, Benefits, and Beliefs of Brain Training Smartphone Apps: An Internet Survey of Younger US Consumers. Front Hum Neurosci 2016;10:253.Abstract
[This corrects the article on p. 180 in vol. 10, PMID: 27148026.].
Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry 2016;3(2):100-2.
Torous J, Baker JT. Why Psychiatry Needs Data Science and Data Science Needs Psychiatry: Connecting With Technology. JAMA Psychiatry 2016;73(1):3-4.
Hsin H, Torous J, Roberts L. An Adjuvant Role for Mobile Health in Psychiatry. JAMA Psychiatry 2016;73(2):103-4.
Gay K, Torous J, Joseph A, Pandya A, Duckworth K. Digital Technology Use Among Individuals with Schizophrenia: Results of an Online Survey. JMIR Ment Health 2016;3(2):e15.Abstract
BACKGROUND: Despite growing interest in the use of digital technology by individuals with schizophrenia, little is known about how these individual relate to, own, and use technology in their daily life and in the context of their symptoms. OBJECTIVE: The goal of this study is to better characterize technology use in those with schizophrenia. METHODS: A Web-based survey of individuals' use of and attitudes toward technology for those 18 years and older self-identifying as having schizophrenia, schizoaffective disorder, or schizophrenia spectrum disorders was conducted. Consumer input was sought in the design of the survey. RESULTS: In total, 457 individuals responded to this Web-based survey. Ninety percent owned more than one device (personal computer, landline telephone, tablet, public computer, mobile phone without applications or Internet, or smartphone), with many reporting high utilization of multiple devices, and 61% having 2 devices. The respondents reported that Web-based technology helped with support from family and friends, as well as in gathering information. Many respondents used Web-based technology to help identify coping strategies (24% very often or often) including music to help block or manage voices (42%), while others used technology to set alarms/reminders for medication management (28%). Younger respondents in particular anticipated the role of technology growing over time with respect to their recovery. CONCLUSIONS: Survey respondents reported that technology access was common, with utilization involving coping, reminders for medications and appointments, and connection. Overall, attitudes were largely positive. Overuse was a concern for 30% of respondents. The study is limited in its generalizability as the population was highly engaged in mental health treatment (87%), self-identified as living with the disorder, and had awareness of their illness. This survey demonstrates high engagement for a subset of technology-oriented individuals living with schizophrenia. It is not known what percent of individuals with schizophrenia are represented by these technology-oriented survey respondents.
Firth J, Torous J, Yung AR. Ecological momentary assessment and beyond: The rising interest in e-mental health research. J Psychiatr Res 2016;80:3-4.
Halpern Y, Horng S, Choi Y, Sontag D. Electronic medical record phenotyping using the anchor and learn framework. J Am Med Inform Assoc 2016;Abstract
BACKGROUND: Electronic medical records (EMRs) hold a tremendous amount of information about patients that is relevant to determining the optimal approach to patient care. As medicine becomes increasingly precise, a patient's electronic medical record phenotype will play an important role in triggering clinical decision support systems that can deliver personalized recommendations in real time. Learning with anchors presents a method of efficiently learning statistically driven phenotypes with minimal manual intervention. MATERIALS AND METHODS: We developed a phenotype library that uses both structured and unstructured data from the EMR to represent patients for real-time clinical decision support. Eight of the phenotypes were evaluated using retrospective EMR data on emergency department patients using a set of prospectively gathered gold standard labels. RESULTS: We built a phenotype library with 42 publicly available phenotype definitions. Using information from triage time, the phenotype classifiers have an area under the ROC curve (AUC) of infection 0.89, cancer 0.88, immunosuppressed 0.85, septic shock 0.93, nursing home 0.87, anticoagulated 0.83, cardiac etiology 0.89, and pneumonia 0.90. Using information available at the time of disposition from the emergency department, the AUC values are infection 0.91, cancer 0.95, immunosuppressed 0.90, septic shock 0.97, nursing home 0.91, anticoagulated 0.94, cardiac etiology 0.92, and pneumonia 0.97. DISCUSSION: The resulting phenotypes are interpretable and fast to build, and perform comparably to statistically learned phenotypes developed with 5000 manually labeled patients. CONCLUSION: Learning with anchors is an attractive option for building a large public repository of phenotype definitions that can be used for a range of health IT applications, including real-time decision support.
Powell AC, Torous J, Chan S, Raynor GS, Shwarts E, Shanahan M, Landman AB. Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation Apps. JMIR Mhealth Uhealth 2016;4(1):e15.Abstract
BACKGROUND: There are over 165,000 mHealth apps currently available to patients, but few have undergone an external quality review. Furthermore, no standardized review method exists, and little has been done to examine the consistency of the evaluation systems themselves. OBJECTIVE: We sought to determine which measures for evaluating the quality of mHealth apps have the greatest interrater reliability. METHODS: We identified 22 measures for evaluating the quality of apps from the literature. A panel of 6 reviewers reviewed the top 10 depression apps and 10 smoking cessation apps from the Apple iTunes App Store on these measures. Krippendorff's alpha was calculated for each of the measures and reported by app category and in aggregate. RESULTS: The measure for interactiveness and feedback was found to have the greatest overall interrater reliability (alpha=.69). Presence of password protection (alpha=.65), whether the app was uploaded by a health care agency (alpha=.63), the number of consumer ratings (alpha=.59), and several other measures had moderate interrater reliability (alphas>.5). There was the least agreement over whether apps had errors or performance issues (alpha=.15), stated advertising policies (alpha=.16), and were easy to use (alpha=.18). There were substantial differences in the interrater reliabilities of a number of measures when they were applied to depression versus smoking apps. CONCLUSIONS: We found wide variation in the interrater reliability of measures used to evaluate apps, and some measures are more robust across categories of apps than others. The measures with the highest degree of interrater reliability tended to be those that involved the least rater discretion. Clinical quality measures such as effectiveness, ease of use, and performance had relatively poor interrater reliability. Subsequent research is needed to determine consistent means for evaluating the performance of apps. Patients and clinicians should consider conducting their own assessments of apps, in conjunction with evaluating information from reviews.
Firth J, Cotter J, Torous J, Bucci S, Firth JA, Yung AR. Mobile Phone Ownership and Endorsement of "mHealth" Among People With Psychosis: A Meta-analysis of Cross-sectional Studies. Schizophr Bull 2016;42(2):448-55.Abstract
There is currently growing interest in using mobile phones to support the treatment of psychotic disorders, such as schizophrenia. However, the widespread implementation of these interventions will ultimately depend upon patients' access to mobile devices and their willingness to engage with mobile health ("mHealth"). Thus, we conducted a systematic review and meta-analysis to assess mobile phone ownership and interest in mHealth among patients with psychosis. An electronic search of Ovid MEDLINE, Embase, PsycINFO, CENTRAL, AMED, Health Technology Assessment Database, and Health Management Information Consortium Database was conducted, using search terms synonymous with mobile phones and psychotic disorders. The initial literature search yielded 2572 results. Fifteen studies matched eligibility criteria, reporting data from 12 independent samples of psychiatric patients (n = 3227). Data pertaining to mobile phone ownership, usage, and opinions on mHealth among patients with psychotic disorders were extracted from these studies, and meta-analytic techniques were applied. The overall mobile phone ownership rate was 66.4% (95% CI = 54.1%-77.6%). However, we found strong statistical evidence that mobile phone ownership has been significantly increasing since 2007, and the rate among patients surveyed in the last 2 years was 81.4% (n = 454). Furthermore, in surveys of mHealth acceptability, the majority of patients responded in favor of using mobile phones to enhance contact with services and support self-management. Considering the increasing availability of mobile phones and the broad acceptability of mHealth among patients, there is now a need to develop and evaluate mHealth interventions to enhance healthcare services for people with psychosis.
Torous J. Mobile telephone apps first need data security and efficacy. BJPsych Bull 2016;40(2):106-7.
Torous J, Kiang MV, Lorme J, Onnela J-P. New Tools for New Research in Psychiatry: A Scalable and Customizable Platform to Empower Data Driven Smartphone Research. JMIR Ment Health 2016;3(2):e16.Abstract
BACKGROUND: A longstanding barrier to progress in psychiatry, both in clinical settings and research trials, has been the persistent difficulty of accurately and reliably quantifying disease phenotypes. Mobile phone technology combined with data science has the potential to offer medicine a wealth of additional information on disease phenotypes, but the large majority of existing smartphone apps are not intended for use as biomedical research platforms and, as such, do not generate research-quality data. OBJECTIVE: Our aim is not the creation of yet another app per se but rather the establishment of a platform to collect research-quality smartphone raw sensor and usage pattern data. Our ultimate goal is to develop statistical, mathematical, and computational methodology to enable us and others to extract biomedical and clinical insights from smartphone data. METHODS: We report on the development and early testing of Beiwe, a research platform featuring a study portal, smartphone app, database, and data modeling and analysis tools designed and developed specifically for transparent, customizable, and reproducible biomedical research use, in particular for the study of psychiatric and neurological disorders. We also outline a proposed study using the platform for patients with schizophrenia. RESULTS: We demonstrate the passive data capabilities of the Beiwe platform and early results of its analytical capabilities. CONCLUSIONS: Smartphone sensors and phone usage patterns, when coupled with appropriate statistical learning tools, are able to capture various social and behavioral manifestations of illnesses, in naturalistic settings, as lived and experienced by patients. The ubiquity of smartphones makes this type of moment-by-moment quantification of disease phenotypes highly scalable and, when integrated within a transparent research platform, presents tremendous opportunities for research, discovery, and patient health.
Torous J, Keshavan M. The role of social media in schizophrenia: evaluating risks, benefits, and potential. Curr Opin Psychiatry 2016;29(3):190-5.Abstract
PURPOSE OF REVIEW: Patients with schizophrenia suffer from numerous social problems often because of negative symptoms of the illness and impairments in social cognition. Social media and social networks now offer a novel tool to engage and help patients navigate and potentially improve social functioning. In this review, we aim to explore how impaired neural networks in schizophrenia impair social functioning, examine the evidence base for social networks and social media to help in the role, consider the evidence for current risks and benefits of use, and discuss the future of social media and social networks for schizophrenia. RECENT FINDINGS: Patients with schizophrenia are increasingly connected to and engaged with social media. There is strong evidence that they own, use, and accept digital tools like smartphones and already use social media services like Facebook at high rates, especially among those who are younger. Less is known about the clinical risks and benefits of social media use in schizophrenia, although there are increasingly more social networking platforms being designed specifically for those with mental illness. SUMMARY: Social media tools have the potential to offer a plethora of new services to patients with schizophrenia, although the clinical evidence base for such is still nascent. It is important to ensure that both clinicians and patients are aware of and educated about the risks of using social media. Going forward, it is likely that social media will have an expanding role in care, with social media offering new pathways to address negative symptoms and impairments in social cognition in schizophrenia.
Safran C. Time for Transitions. Int J Med Inform 2016;92:v.
Crotty BH, Slack WV. Designing online health services for patients. Isr J Health Policy Res 2016;5:22.Abstract
Patients are increasingly interacting with their healthcare system through online health services, such as patient portals and telehealth programs. Recently, Shabrabani and Mizrachi provided data outlining factors that are most important for users or potential users of these online services. The authors conclude convincingly that while online health services have great potential to be helpful to their users, they could be better designed. As patients and their families play an increasingly active role in their health care, online health services should be made easier for them to use and better suited to their health-related needs. Further, the online services should be more welcoming to people of all literacy levels and from all socioeconomic backgrounds.
Minen MT, Torous J, Raynowska J, Piazza A, Grudzen C, Powers S, Lipton R, Sevick MA. Electronic behavioral interventions for headache: a systematic review. J Headache Pain 2016;17:51.Abstract
BACKGROUND: There is increasing interest in using electronic behavioral interventions as well as mobile technologies such as smartphones for improving the care of chronic disabling diseases such as migraines. However, less is known about the current clinical evidence for the feasibility and effectiveness of such behavioral interventions. OBJECTIVE: To review the published literature of behavioral interventions for primary headache disorders delivered by electronic means suitable for use outside of the clinician's office. METHODS: An electronic database search of PubMed, PsycINFO, and Embase was conducted through December 11, 2015. All eligible studies were systematically reviewed to examine the modality in which treatment was delivered (computer, smartphone, watch and other), types of behavioral intervention delivered (cognitive behavioral therapy [CBT], biofeedback, relaxation, other), the headache type being treated, duration of treatment, adherence, and outcomes obtained by the trials to examine the overall feasibility of electronic behavioral interventions for headache. RESULTS: Our search produced 291 results from which 23 eligible articles were identified. Fourteen studies used the internet via the computer, 2 used Personal Digital Assistants, 2 used CD ROM and 5 used other types of devices. None used smartphones or wearable devices. Four were pilot studies (N ≤ 10) which assessed feasibility. For the behavioral intervention, CBT was used in 11 (48 %) of the studies, relaxation was used in 8 (35 %) of the studies, and biofeedback was used in 5 (22 %) of the studies. The majority of studies (14/23, 61 %) used more than one type of behavioral modality. The duration of therapy ranged from 4-8 weeks for CBT with a mean of 5.9 weeks. The duration of other behavioral interventions ranged from 4 days to 60 months. Outcomes measured varied widely across the individual studies. CONCLUSIONS: Despite the move toward individualized medicine and mHealth, the current literature shows that most studies using electronic behavioral intervention for the treatment of headache did not use mobile devices. The studies examining mobile devices showed that the behavioral interventions that employed them were acceptable to patients. Data are limited on the dose required, long term efficacy, and issues related to the security and privacy of this health data. This study was registered at the PROSPERO International Prospective Register of Systematic Reviews (CRD42015032284) (Prospero, 2015).
Torous JB, Chan SR, Yellowlees PM, Boland R. To use or not? evaluating ASPECTS of smartphone apps and mobile technology for clinical care in psychiatry. J Clin Psychiatry 2016;Abstract
In this commentary, we discuss smartphone apps for psychiatry and the lack of resources to assist clinicians in evaluating the utility, safety, and efficacy of apps. Evaluating an app requires new considerations that are beyond those employed in evaluating a medication or typical clinical intervention. Based on our software engineering, informatics, and clinical knowledge and experiences, we propose an evaluation framework, "ASPECTS," to spark discussion about apps and aid clinicians in determining whether an app is Actionable, Secure, Professional, Evidence-based, Customizable, and TranSparent. Clinicians who use the ASPECTS guide will be more informed and able to make more thorough evaluations of apps.
Ayers K, Li Z, Quintana Y, Van Kirk Villalobos A, Klosky JL. St. Jude Cancer Education for Children Program: The Impact of a Teacher-Led Intervention on Student Knowledge Gains. J Cancer Educ 2016;Abstract
In 2006, St. Jude Children's Research Hospital (Memphis, Tennessee) began developing a school-based outreach program known as the St. Jude Cancer Education for Children Program (SJCECP). The aim of this program is to teach Memphis-area children about cells, cancer, and healthy habits that can prevent the development of cancer in adulthood. Initial plans for delivery of the program was for St. Jude staff to present the program at local schools. This plan for disseminating instruction was not feasible due to the limited availability of St. Jude staff. As a next step, during the 2012-2014 academic years, we conducted a study entitled SJCECP2, utilizing the SJCECP curriculum, with the objective of evaluating the impact of the educational intervention on knowledge acquisition and retention among fourth-grade students participating in a modified, teacher-led version of the program. Eighteen teachers and 426 students from 10 local schools in the greater Memphis area participated in the program evaluation. This study used a single-group, pre-test/post-test design to determine the impact of the SJCECP intervention on changes in knowledge scores among fourth-grade students. Testing was on cells, cancer, and healthy living. The mean scores increased from 6.45 to 8.12, 5.99 to 7.65, and 5.92 to 7.96 on cell, cancer, and health behaviors units, respectively (all p values <.001). Preliminary evidence suggests that the SJCECP2 intervention is a useful tool for teachers to improve student knowledge of knowledge of cells, cancer, and healthy living concepts at the fourth-grade level.
Addressing the Challenges of Aging: How Elders and Their Care Partners Seek Information.
Walker J, Crotty BH, O'Brien J, Dierks MM, Lipsitz L, Safran C. Addressing the Challenges of Aging: How Elders and Their Care Partners Seek Information. Gerontologist 2016;Abstract

PURPOSE: Elders in retirement communities face many challenges concerning information and communication. We know little about whether or how online technologies help meet their medical and social needs. The objective of this study was to gain insights into how these elders and their families manage health information and communication. DESIGN AND METHODS: Qualitative analysis of 10 focus groups with elders and family members. Participants were 30 elders at least 75 years of age residing in 5 senior living communities in and near Boston, MA, and 23 family members. RESULTS: Elders and families turned first to their personal networks when they needed information or help. They stayed informed about elders' health primarily by talking directly with providers. They used online resources infrequently, including portal access to medical records. They wanted online access to medication lists and visit notes, up-to-date information about local services and social activities, and a way to avoid the overwhelming nature of Internet searches. IMPLICATIONS: Elders in senior living communities and their families piece together information primarily from word of mouth communication. In the future, electronic social and collaborative technologies may make information gathering easier.