The Vianai team was a sponsor at the Toronto Machine Learning Summit, and we had a blast meeting attendees at our booth, and listening to lightning talks, workshops, and technical sessions.
The Carlu, where the event was held, provided an intimate setting that fostered networking and engagement. The coffee bar and snack tables were well-stocked to fuel discussion, and the conference organizers did a fantastic job. Shout-out to Faraz Thambi for a great event!
The @Whova app made it so easy to see who else was attending, connect with attendees, and even share nerdy jokes like this:.
A poll at the beginning of the event revealed only 58% of respondents had five or fewer models in production, and 25% had between 6-10. This, along with discussions we had with other attendees, confirmed what we have been seeing across the ML community at large, and with our own customers and colleagues–
ML is messy, and each company seems to do things a bit differently.
There’s not a one-size-fits-all solution or approach, nor do people seem to be looking for one. Instead, users want tools and techniques that help them gain deeper insight to model behavior quickly, know what to do to ensure fairness, mitigate bias, and accelerate performance.
The Issue with ML Models and Vianai’s Answer
Deploying more models to production faster and keeping models trustworthy seemed to be the biggest challenges we heard from people at the Summit. Many of the attendees who visited our booth shared that keeping models trustworthy is often an afterthought, becoming a significant barrier across the ML lifecycle.
At Vianai, we are deeply passionate about this as these problems become acute at scale – scale in terms of more models, more features per model, more inferences. Our focus is on helping users quickly identify:
- Why models degrade
- How to identify root cause
- Understand what is going on
- Steps to take to fix the problem
We see this as a continuous cycle of operations that includes monitoring, retraining models, and validating them before redeployment.
Some data scientists (DS) we spoke with owned the full ML model lifecycle and were keen on gaining a deep understanding of each step in the process to learn how to continuously improve and build better models. Other DS saw their role as discrete from the rest of the MLOps process but struggled to manage clean hand-offs to machine learning engineers (MLE) and spent too much time answering questions and bridging the gap between the roles. One DS found that our simplified UX was exactly what she had been looking for so any other stakeholder could import a model and automate its deployment.
Our team was able to attend some pre-conference workshops and several sessions as well. I listened to Jesse Cresswell’s talk “Navigating the Tradeoff Between Privacy and Fairness in ML” where he shared how privacy-enhancing technology can increase unfairness and bias in ML models, using federated learning and differential privacy as examples, and then suggested how to address the challenges.
I also listened to Hien Luu’s talk “Scaling and Evolving the Machine Learning Platform at DoorDash” which gave insight into many lessons and technical decisions and tradeoffs that were made in building their platform to handle massive scale. There was a great selection of other topics that included:
- Industry-specific solutions
- The role of alternative data in investing
- Using NLP for financial markets, managing AI in regulatory environments
- Serving large-scale knowledge graphs
We also met many accomplished and passionate students and recent graduates from the University of Toronto, University of Waterloo, and York University who were excited about ML – we hope to hire some of them! Overall, this event was thought-provoking, informative, educational, and fun! We can’t wait to go back next year.