There are few things more alluring than the idea of putting an AI system "on the edge." It's easy to imagine the use cases - smart energy management, autonomous manufacturing, responsive infrastructure. Indeed, in a world with perfect technology, one could even conjure up a utopian vision autonomous vehicles and drone delivery.
Compounding this is a general sense of a huge market opportunity. IDC put out a report in 2019 that predicted that Edge AI investment would top $1.3 trillion by 2023.
While the opportunity is real, a commonsensical approach is called for. First, as a recent Gartner report pointed out, there are many pieces of technology that limit the ability for some more advanced AI use cases: These include the limited compute on the edge (especially compared to the cloud) and the availability of good-quality and usable data.
Many companies have sought to solve these problems with hardware. It makes sense. With standardized hardware comes standardization in the data type, format and set. Also, the architects of the system would only have to design for one type of silicon, which avoids a much more complex set of problems associated with having software that can fit onto various types of hardware.
In various interviews across large companies, though, we've found three significant additional problems to Gartner's list:
Let me give an example. A large building management company shared with us how they keep Edge AI small, and therefore capable of managing it on older hardware. Essentially, they keep it very focused on specific use cases. This means that each chiller they have has its own AI model, each room, each chatbot, on and on.
This may help with them keeping the models small, but it has a knock-on effect of having many dozens, and potentially one day, hundreds, if not thousands or more, of models to manage in production.
But models on the edge aren't the same as models trained for increasing ad return or basket size. This is why understanding the risk and uncertainty within a model, as well as having methods to explain how it works and what might have gone wrong, are crucial to ensuring a system that doesn't go down.
This point is especially critical for the industrial manufacturing companies we've spoken to and worked with - some of the largest in the world. They consistently have said that keeping their machines running without any disruption is their primary focus.
In fact, we spoke to a large health care company that was the only to make a critical piece of medical equipment used around the world. If their factories went down, then it would endanger lives.
These are the kinds of problems that a sophisticated MLOps solution can handle. Risk, uncertainty, monitoring, automatic retraining, even optimization, are all key criteria in making AI successful at large edge companies. We've seen our MLOps platform brought into large manufacturers to manage their ML, and we've embedded our MLOps capabilities to enhance our clients' offerings.
Overall, we've listened closely to the needs of our customers and research, and it has informed our human-centered AI strategy, our platform and our products.