A majority of problems that AI pilots encounter are not only solvable but with the right planning ahead of time, they can be avoided altogether. Alyssa Rochwerger, former VP of AI of Appen, and Wilson Pang, the CTO of Appen, take you through two examples of pilots to show you what to do and avoid to increase your odds of success.
Only 20% of AI pilots in the real world make it to production. The other 80% fail. These are sobering statistics. Developing AI models can be costly and time-consuming for businesses, and you don’t want to pour time, money, and energy into a project only for it to ultimately fail. Fortunately, there’s good news. Many of these challenges can be avoided with the right planning. At Appen, we work with companies from all types of industries to help them deploy AI confidently, and our average pilot-to-production deployment rate for the past two years has been 67 percent—far better than the 20% general average. Now, let us take you through two examples of pilots to show you what to do and what not to do to increase your odds of success.
A Car Company’s Failed Pilot
A national media and automotive dealer conglomerate—relied upon by nearly every person buying or selling cars in the country—once tried to get into the AI game to mine more information from the big library of images of the used cars they sold. They figured, if they could automatically identify dents, rust, and other kinds of damage, they would be able to find all kinds of uses for the information, from scheduling repairs more efficiently to providing accurate descriptions to customers to handling accident claims.
So they asked an AI team to create a model that could find dents and rust in images of cars. The AI team started building the model, but because they knew this was a complex problem, they also subcontracted with several other machine learning companies at the same time, hoping that at least one of them would do the job quickly and accurately.
Over a year and a few million dollars later, none of the AI models could achieve better than 60 to 70% accuracy. The images didn’t have uniform lighting, which led to the shading being inconsistent. The model wasn’t able to reliably distinguish between dents, rust stains, or simple shadows.
Learn More: Implementing AI: Moving Beyond the Hype
Where They Went Wrong: What Not to Do
This failure didn’t happen for lack of talented computer vision scientists working on the problem. Computer vision problems are notoriously difficult to solve. They require a huge investment in data labeling and complex models that many companies find hard to justify—unless the business value is similarly huge.
This company’s mistake was common: they started with an AI problem rather than a business problem. They didn’t ask the AI team to focus on a solution for a single business pain point. Instead, they handed them a big, general problem and asked them to get it up and running with a certain quality level, and they’d figure out exactly how to use it later. It’s no wonder that the team wasn’t able to deliver.
Learn More: Why We’re Seeing an Uptick in Data Science Architecting
A Successful Pilot
In April 2015, California was experiencing a terrible drought. Governor Jerry Brown issued mandatory water restrictions: municipalities had to cut their water usage by 25% within the next few months.Â
This presented an enormous challenge for local governments. They could ask everyone to use less water, but some wouldn’t be able to or wouldn’t listen. If the city could find people who were using more than their fair share, however, they could reach out directly to help them cut back. Unfortunately, few cities in California had the sophistication or metering to understand exactly where all their water was going.
OmniEarth was a small California-based startup that analyzed public satellite imagery to provide data about water usage. They looked at the color of lawns—if a lawn was too green, it probably meant that a lot of water went into keeping it that way. A house with solar panels on its roof was an indicator the home was statistically more likely to have other green efficiencies inside, like low-flow showers or toilets.Â
Considering these and other factors, OmniEarth was able to provide very granular consumption data to California’s water district about how much water every property actually needed. They then integrated this information with actual billing data to show which properties were using more water than they needed.
As a result of this integration, California cities were able to target their direct-mail campaigns very narrowly to only those people who were overconsuming water. This efficiency allowed their budgets to stretch further and ultimately help them be more successful at achieving the aggressive goal the governor had mandated.
Learn More: How Modern Data Centers Can Leverage Object Migration Software
What They Did Right: What to Do
So what did OmniEarth do right with their pilot?
First, they started small. They didn’t start with the entire state of California; they started with a single county and worked their way up. If they’d tried to apply the same model everywhere in California, it wouldn’t have worked. The meaning of “too green†is very different in the Sierra Nevada than it is in the Los Angeles or Bay Area.Â
While they started small, they prepared for eventual scaling. For example, since they were working in a single county, they could have acquired far more detailed imagery by sending up drones. This might have made it easier to get their pilot model to succeed, but it clearly wouldn’t have been possible for the entire state. So instead, they relied on publicly available U.S. Geological Survey data, which was available for the state.
OmniEarth’s approach was ultimately considered very successful, and the company was acquired by EagleView in 2017.
Learn More: Women in Tech Are Fighting More Than Just a Pandemic
Creating a Great AI Pilot
For your business to reap the benefits of AI, you first need a successful AI pilot, which requires thoughtful planning and design. Start by identifying a single business pain point that can be reasonably, cost-effectively solved using AI. Then start small, with a narrow scope for your pilot that can be scaled up later.
Creating a great AI pilot is never going to be easy. The odds are against you. But by following these guidelines, you can surely be successful.
Did you enjoy reading the article? Let us know your thoughts in the comments section below or on LinkedInOpens a new window , TwitterOpens a new window , or FacebookOpens a new window . We would love to hear from you!