Cautionary Findings in Study of Web-Based Tool for Children with Flu-Like Illness

While parents and caregivers reported that a web-based triage tool for children with influenza-like illness was easy to understand and use, the tools underlying mathematical formula needs to be improved so it does not result in more, rather than fewer, children seeking emergency department care, according to a study published Online First by Archives of Pediatrics & Adolescent Medicine, a JAMA Network publication. The study is being published early because its public health importance.

Emergency department (ED) crowding is an issue, especially when demand for ED care is high, such as during an influenza epidemic. One potential strategy is to develop automated algorithms so patients can self-triage to determine if they need to visit the ED, according to the study background.

Rebecca Anhang Price, PhD, of the RAND Corporation, and colleagues conducted a pilot validation study during the 2012 influenza season to get feedback on the Strategy for Off-site Rapid Triage (SORT) for Kids, a web-based tool intended to triage patients affected by the flu.

An adult version of the triage tool was made available to the public during the 2009 influenza pandemic, but the Centers for Disease Control and Prevention (CDC) and the American Academy of Pediatrics (AAP), which jointly devised the pediatric algorithm, would not endorse the effort to make such a tool available for pediatric patients without evidence of its safety.

Researchers were interested in parental feedback on the SORT for Kids websites usability and the sensitivity and specificity of its underlying algorithm. The study, which included 294 parents and adult caregivers of children with influenza-like illness (ILI), was conducted between February 8 and April 30, 2012 at two pediatric emergency departments in the metro Washington, D.C. area. After parents used the prototype website to enter information about their childs illness, all of the children received ED evaluation and treatment. This allowed researchers to compare the computers assessment of risk to that of experienced healthcare professionals.

According to the study results, 90 percent of the participants reported that the website was very easy to understand and use. Based on responses provided by parents and adult caregivers, the SORT for Kids algorithm classified 10.2 percent of patients as low risk, 2.4 percent as intermediate risk and 87.4 percent as high risk.

Safety was the chief concern of the CDC/AAP working group that developed the clinical algorithm; therefore the algorithm was designed to avoid misclassification of high-risk cases. As a consequence, SORT for Kids deemed the vast majority of children with ILI in our study as high risk, sacrificing specificity for the sake of sensitivity, the researchers comment. An unintended consequence of such a cautious approach might be to significantly overtriage mildly and moderately ill children to hospital EDs, worsening ED crowding in the process and quite possibly contributing to a range of adverse events. This is not what the algorithms authors had in mind.

The algorithm correctly classified 93 percent of pediatric patients with ILI who made necessary ED visits and all children who made a second ED visit for ILI within the subsequent week, according to the study.

Our findings present a cautionary tale regarding the potential effects of self-triage tools. Although a diverse set of consumers found the SORT for Kids website easy to use, the underlying algorithms specificity was poor. Had it been made available to the public in its current form, it might have led more, rather than fewer, parents to bring their children to an ED, thereby worsening, rather than ameliorating, ED crowding, the authors conclude.

The authors suggest further research is needed: Because the goal of self-triage is to reduce health system surge, not increase it, a much larger prospective study, examining a wider range of clinical questions, will be needed to refine the algorithm to achieve a higher level of specificity without compromising patient safety, they conclude.

Reference: Arch Pediatr Adolesc Med. Published online December 12, 2012. doi:10.1001/jamapediatrics.2013.1573.