In the previous issue, I discussed why Machine Learning projects fail
. In this issue, let’s start figuring out how to build successful Machine Learning products. The first step is understanding when Machine Learning is more effective than traditional programming.
Traditional programs have deterministic logic to solve a problem. But Machine Learning is probabilistic. It leverages patterns in data to tune the logic.
For a problem:
- If you have a deterministic logic that solves it with 100% accuracy, then obviously that is cheaper, easier, and more accurate than any ML model one can make.
- If you have some stable heuristic rules that solve it most of the time, the extra work/complexity of ML might not be worth it.
- If your heuristics does not work up to the desired accuracy and requires constant updates, then ML can be a good bet.
Always evaluate the tradeoffs of additional complexity and cost against performance gains to determine if it is really worth it.
Some of the problems that are better solved with ML:
- Search result ranking
- Mail classification and spam detection
- Expected Time of Arrival in maps
- Clustering similar news stories
- Web/App Ads click-through rate prediction in Google, Facebook, etc.
- Product recommendations in Amazon, Netflix, Facebook, Quora, etc.
- Personal Assistants like Siri, Alexa, Google
- Fraud transaction detection
- Pricing/Financial Modeling
- Customer segmentation and churn prediction
- Equipment failure prediction
- Network intrusion detection
- Text sentiment analysis
- Language translation
- Image/video analysis
- Speech/audio understanding
It is not possible to design a 100% correct logic for these problems. Earlier, many of these problems were solved with heuristic rules that were updated constantly. Instead, collecting data and training an ML model is easier and better. (However, these heuristic solutions are a good starting point for collecting the needed data.)