The most common factor determining machine learning success or failure isn’t the accuracy of models or the quality of data – it’s how effectively the technology is adopted and applied in teams. A key element can be the right training and education. In this month’s blog, we discuss how to get the balance right – upskilling for success without requiring your team to become data science or statistics experts.
Here’s a shocking statement from a company dedicated to Machine Learning (ML): ML sometimes doesn’t work; indeed, it can make things worse. There’s even a GitHub repository recording examples of ML failure, which range from the amusing to the tragic. And there are plenty of good articles (like this one) exploring the causes of failure: asking the wrong questions, problems or misalignments in the data, flawed deployment projects. But, before you give up on that ML project you were considering, there are also many examples of its transformative potential – we have quite a few of these at Intellegens. Indeed, it’s fair to argue that you can’t opt-out of the AI revolution and expect to survive in the competitive world of R&D. So how do we adopt ML successfully and avoid the risks? Part of the answer, as is so often the case, is education.
This doesn’t mean every user must learn how to be a data scientist. The objective of using ML is to facilitate your work, not to add to it. It does mean that research organizations should ensure their people develop the appropriate level of knowledge to support effective deployment of the technology. It’s been three years since we participated in industry research that identified educational initiatives as one key element needed to enable adoption of ML in the chemicals and materials industries. Since then, through our work with customers in these sectors, we’ve done a lot of thinking about what that means in practice. Here’s five areas in which every research-based organization could consider upskilling its staff to make them ML-ready:
1. Senior management training. Leaders need to learn how to guide ML-enabled R&D successfully, without needing to become technical experts. This includes asking the right questions, defining realistic data requirements, supporting teams effectively, avoiding common implementation mistakes, and turning results into clear, actionable decisions.
2. A framework for deployment. Project managers need to understand enough of the foundational concepts of ML to appreciate its capabilities and limitations. How can it assist experimental design? Why is uncertainty quantification important? How do we frame problems to enable data-driven decision-making? The goal is not theory for its own sake, but supporting a practical implementation roadmap.
3. How to implement. The teams charged with turning roadmaps into reality can learn from the experience of others. The right training can help you to specify initial projects that deliver useful results. Understanding how to get your data ML-ready can avoid many problems. It’s also important to learn through your own experience – the right support as you begin to implement these projects helps teams build expertise and confidence in independent use.
4. Using results. ML needs humans-in-the-loop, thinking critically about the results and drawing the right lessons to make research decisions. The skills required here are not unusual for scientists; they are fundamental research skills – critical thinking, good scientific method, integrating domain knowledge with quantitative analysis. Nevertheless, it can be useful, early in your ML journey, to get structured, hands-on training that helps you to apply these skills to realistic ML scenarios.
5. Embedding ML in your workflows. ML isn’t magic. It’s a tool that works best for research when fed with the right data and embedded in a virtuous cycle in which it can guide what experiments you do next and the results of those experiments can then both validate ML predictions and be used to continually improve the ML model. This approach is called adaptive Design of Experiments. A good grounding in its concepts and application can increase your ML productivity.
At Intellegens, we’ve responded to customer demand for education in these topics through our Alchemite™ Academy initiative.
Alchemite Academy provides customer organizations with structured training as they specify, deploy, and apply machine learning. Although hands-on use of the software is part of these courses, the focus is not on ‘which buttons to press’ – it’s our job to make our Alchemite™ software sufficiently intuitive that such coaching is irrelevant. Instead, we help the different roles in a team – leaders, project managers, and end-users – to develop an appropriate level of understanding of where ML can be successfully applied and how to achieve that goal.
Done right, ML adoption should equal turbo-charged R&D processes, avoiding frustrations and mis-direction. If you’re interested in sharing experience – of success or lessons learned – get in touch!
