Web-based app identifies insects around the world and around the farm

A farmer notices an unfamiliar insect on a leaf. Is this a pollinator? Or a pest? Good news at harvest time? Or bad? Need to be controlled? Or not?

That farmer can snap a picture, use a smartphone or computer to feed the photo into a web-based application called InsectNet and, with the help of machine learning technology, get back real-time information.

“The app identifies the insect and returns a prediction of its taxonomic classification and role in the ecosystem as a pest, predator, pollinator, parasitoid, decomposer, herbivore, indicator and invasive species,” said a scientific paper describing InsectNet recently published by the journal PNAS Nexus. Iowa State University’s Baskar Ganapathysubramanian and Arti Singh are the corresponding authors.

InsectNet—which is backed by a dataset of 12 million insect images, including many collected by citizen-scientists—provides identification and predictions for more than 2,500 insect species at more than 96 percent accuracy. When the application isn’t sure about an insect, it says it is uncertain, giving users more confidence when it does provide answers.

And, because the application was built as a global-to-local model, it can be geographically fine-tuned using expert-verified local and regional datasets. That makes it useful to farmers everywhere.

So, beware, armyworms, cutworms, grasshoppers, stink bugs and all the other harmful insects. And, hello, butterflies, bees and all the other pollinators. Good to see you, lady beetles, mantises and all the other pest predators.

“We envision InsectNet to complement existing approaches, and be part of a growing suite of AI technologies for addressing agricultural challenges,” the authors wrote.

A village of researchers

InsectNet’s ability to be fine-tuned for specific regions or countries make it particularly useful, said Singh, an associate professor of agronomy.

In Iowa, for example, Singh said there are about 50 insect species particularly important to the state’s agricultural production. To identify and provide predictions about those insects, Singh said the project used about 500,000 insect images.

That could happen for farmers all over the globe. And wherever there isn’t sufficient data—these sophisticated models often require millions of images—for local fine-tuning, the global dataset is still available for farmers.

InsectNet isn’t just for farmers, though. Singh said it could also help agents at ports or border crossings identify invasive species. Or it could help researchers working on ecological studies.

So, the app is usable and flexible. But is it accessible?

You can’t go to an app store and download a version just yet, said Ganapathysubramanian, the Joseph and Elizabeth Anderlik Professor in Engineering and director of the AI Institute for Resilient Agriculture based at Iowa State. But the app is running on a server at Iowa State. With a QR code (see sidebar) or this URL, users can upload insect pictures and get an identification and prediction.

This works throughout the stages of an insect’s life: from egg to larva to pupa to adult. It works with look-alike species. And it works with diverse image qualities and orientations.

The bottom line for any user is basic information about an insect: “Is this a pest?” Singh said. “Or is it a friend?”

Developers demonstrated the app during last August’s Farm Progress Show in Boone, Iowa. And now the research paper is introducing it to a broader, scientific audience.

But aren’t there already apps that help identify insects?

Yes, said Ganapathysubramanian, but they’re not to the scale of InsectNet and aren’t capable of global-to-local applications. And they’re also not open-source applications with technology that can be shared.

“Making InsectNet open source can encourage broader scientific efforts,” he said. “The scientific community can build on these efforts, rather than starting from scratch.”

The project also answered a lot of technical questions that could be applied to other projects, he said.

How much data is enough? Where can we get that much data? What can we do with noisy data? How much computer power is necessary? How do we deal with so much data?

“Lastly, it takes a village of expertise to get to this point, right?” said Ganapathysubramanian.

It took agronomists and computer engineers and statisticians and data scientists and artificial intelligence specialists about two years to put InsectNet together and make it work.

“What we learned working with insects can be expanded to include weeds and plant diseases or any other related identification and classification problem in agriculture,” Singh said. “We’re very close to a one-stop shop for identifying all of these.”