UCLA Researchers Use Online Crowd-Sourcing to Diagnose Malaria

Online crowd-sourcing in which a task is presented to the public, who respond, for free, with various solutions and suggestions has been used to evaluate potential consumer products, develop software algorithms and solve vexing research-and-development challenges. But diagnosing infectious diseases?

Working on the assumption that large groups of public non-experts can be trained to recognize infectious diseases with the accuracy of trained pathologists, researchers from the UCLA Henry Samueli School of Engineering and Applied Science and the David Geffen School of Medicine at UCLA have created a crowd-sourced online gaming system in which players distinguish malaria-infected red blood cells from healthy ones by viewing digital images obtained from microscopes.

The UCLA team found that a small group of non-experts playing the game (mostly undergraduate student volunteers) was collectively able to diagnosis malaria-infected red blood cells with an accuracy that was within 1.25 percent of the diagnostic decisions made by a trained medical professional.

The game, which can be accessed on cell phones and personal computers, can be played by anyone around the world, including children.

"The idea is, if you carefully combine the decisions of people even non-experts they become very competitive," says Aydogan Ozcan, an associate professor of electrical engineering and bioengineering and the corresponding author of the crowd-sourcing research. "Also, if you just look at one person's response, it may be OK, but that one person will inevitably make some mistakes. But if you combine 10 to 20, maybe 50 non-expert gamers together, you improve your accuracy greatly in terms of analysis."

Crowd-sourcing, the UCLA researchers say, could potentially help overcome limitations in the diagnosis of malaria, which affects some 210 million people annually worldwide and accounts for 20 percent of all childhood deaths in sub-Saharan Africa and almost 40 percent of all hospitalizations throughout that continent.

The current gold standard for malaria diagnosis involves a trained pathologist using a conventional light microscope to view images of cells and count the number of malaria-causing parasites. The process is very time-consuming, and given the large number of cases in resource-poor countries, the sheer volume presents a big challenge. In addition, a significant portion of cases reported in sub-Sahara Africa are actually false positives, leading to unnecessary and costly treatments and hospitalizations.

By training hundreds, and perhaps thousands, of members of the public to identify malaria through UCLA's crowd-sourced game, a much greater number of diagnoses could be made more quickly at no cost and with a high degree of collective accuracy.

"The idea is to use crowds to get collectively better in pathologic analysis of microscopic images, which could be applicable to various telemedicine problems," says Sam Mavandadi, a postdoctoral scholar in Ozcan's research group and the study's first author.

Ozcan and Mavandadi emphasized that the same platform could be applied to combine the decisions of minimally trained healthcare workers to significantly boost the accuracy of diagnosis, which is especially promising for telepathology, among other telemedicine fields.

The new UCLA study, "Distributed Medical Image Analysis and Diagnosis Through Crowd-Sourced Games," has been accepted for publication in the journal PLoS ONE.

In addition to developing the crowd-sourced gaming platform that allows players to assist in identifying malaria in cells imaged under a light microscope, Ozcan's research group created an automated algorithm for diagnosing the same images using computer vision, as well as a novel hybrid platform for combining human and machine resources toward efficient, accurate and remote diagnosis of malaria.

"The most exciting aspect is that this is an entirely novel approach in the area of visual diagnostics, which really challenges diagnostic algorithms used to date," says Karin Nielsen, a professor of infectious diseases in the department of pediatrics at the Geffen School of Medicine. "It is diagnostics outside the box that is, the study introduces an entirely new concept in diagnostics with the use of games for this purpose. The potential applications of this new approach are immense."

How the game works:

Before playing the game, each player is given a brief online tutorial and an explanation of what malaria-infected red blood cells typically look like using sample images. After completing a short training phase, players go through the actual game, in which they are presented with multiple frames of red blood cell images and can use a "syringe" tool to "kill" the infected cells one-by-one and use a "collect-all" tool to designate the remaining cells in the frame as "healthy."

Within each frame, there are a certain number of cells whose status (i.e., infected or not) is known by the game but not by the players. These control cell images allow Ozcan's team to dynamically estimate the performance of gamers as they go through each frame and also helps the team assign a score for every frame the gamer passes through.

"I believe that, similar to other very innovative ideas, one of the major challenges will be the skepticism of traditional microscopists, pathologists and clinical laboratory personnel, not to mention malaria experts, who will initially view with suspicion a gaming approach in malaria diagnostics," says Nielsen, also an author of the study. "It is a very revolutionary proposal and it might take a few clinical studies in the field to document the efficacy of this platform in order to convince traditional microbiologists and other infectious disease colleagues."

"Scaling up accurate, automated and remote diagnosis of malaria through a crowd-sourced gaming platform may lead to significant changes for developing countries," Ozcan says.

"It could eliminate the current overuse and misuse of anti-malarial drugs, improve management of non-malaria fevers by ruling malaria out, lead to better use of existing funds, and reduce risks due to long-term side-effects of anti-malarial drugs on patients who don't need treatment," Mavandadi adds.

Ozcan's team hopes to bring the platform into the field through clinical trials to help validate its use and facilitate implementation of the technology worldwide. Nielsen and Ozcan plan to implement it at clinical sites in countries such as Mozambique, Malawi and Brazil.

In addition, the same crowd-sourcing and gaming-based micro-analysis and medical diagnosis platform could be further scaled up for a variety of other biomedical and environmental applications in which microscopic images need to be examined by experts, the researchers said.

Other authors of the study included Stoyan Dimitrov, Steve Feng, Frank Yu, Uzair Sikora, Oguzhan Yaglidere and Swati Padmanabhan, all with the UCLA Department of Electrical Engineering.

Funding for Ozcan Research Group is provided by the Presidential Early Career Award for Scientists and Engineers (PECASE), the Army Research Office Young Investigator Award, the NSF CAREER Award (BISH Program), the Office of Naval Research Young Investigator Award 2009 and the NIH Director's New Innovator Award.