From the course: AWS Certified Machine Learning Engineer Associate (MLA-C01) Cert Prep

Traditional vs. AI methods for solving problems

From the course: AWS Certified Machine Learning Engineer Associate (MLA-C01) Cert Prep

Start my 1-month free trial Buy for my team

Traditional vs. AI methods for solving problems

“

(serene music) - [Instructor] Hello guys, and welcome again. So in today's lesson, we are going to discover the suitability of using machine learning in your problems and whether to use machine learning in your problems or just using a simple rule-based algorithm. And what are the factors to consider for that. So when it comes to cost optimization in machine learning, a critical step is identifying whether machine learning is suitable for your specific problem. Because not every challenge requires a machine learning approach. While sometimes traditional algorithms or even simple rule-based systems could achieve the desired outcomes more cost effectively. Another best practice is to perform a trade off analysis between custom and pre-trained models. While custom models can be tailored to your specific needs, offering high accuracy, but they often require extensive computation and resources, time and expertise. But pre-trained models on the other side are generally more cost effective since they've already been trained on vast data sets and can be fine tuned for your use case. One of the key steps in cost optimization is properly framing the machine learning problem before even diving into any solutions. Adopting best practices at this stage could make a significant difference in cost efficiency, and this includes clearly defining the problem you aim to solve, identifying the most relevant data sources and choosing the right algorithms and models. So asking yourself a question of is machine learning the right solution? Well, you need to see first if you could or cannot code the rules yourself. So when rules depend on too many factors and many of those rules overlap or even need to be tuned very finely then you would need a machine learning solution. Or if you cannot scale. So machine learning solutions are effective at handling large scale problems. You should also evaluate alternatives. So what if using a simple rule-based approaches, because it might be more effective, you could even consider if machine learning adds unnecessary complexity. You should also be weighing costs. So what is the cost of adopting machine learning versus the opportunity cost of not using machine learning? And you need to be aware that specialized resources for machine learning can be expensive and limited. So when might AI or machine learning not be appropriate for your solutions? So for situations requiring specific outcomes. So if you need for exact deterministic results over the predictions, remember, machine learning is all about statistics and probabilities. So if you need an exact solution, exact deterministic result, then AI and machine learning may not be appropriate for your problem. Examples of this would be compliance checks and fixed logic operations for example. For the cost benefit analysis, while high costs may outweigh the benefits in some scenarios, and you need to assess if machine learning provides a significant value addition or not. So specialized resource constraints are a major factor in AI and machine learning projects. For human resources, data scientists are expensive and short supply, and also their time significantly impacts the model development and the time to market. When it comes to hardware, choosing the cost effective options could limit the speed of experimentation, and it's crucial to find the balance between keeping the costs low and maintaining development efficiency. Talking about the implementation plan, you need to start as simple as you could. So first of all, you need to articulate the problem through clearly defining objectives and expected outcomes. You would need to identify the data sources and assess the data availability and quality. You would also need to consider associated costs. So you have costs associated to data design and preparation expenses, and you have storage costs for machine learning data sets. Considering the model training costs it's dependent on your hardware choices. So high performance hardware may increase your costs of training the models. Also costs related to data labeling. It's necessary for supervised learning models which stake in labeled data, and they could be significantly large if large data sets are involved. Also worth mentioning some of the potential hidden costs. So first of all, you have the iterative retraining due to bias. So bias could lead to repeated model adjustments in order to solve that problem. And this increases the time and financial investment. Also for hosting and maintenance. So there are some ongoing costs for model deployment and there are monitoring and updating models costs over time. So you would need to weigh the costs against the opportunity. So you would need to assess the total investment in machine learning. So that includes some of all the costs involved in machine learning adoption. And don't forget the potential hidden costs like the monitoring, updating, or even retraining of the model. Also, for the opportunity cost of not using machine learning, you need to assess the potential benefits missed by not transforming with machine learning and using simple based algorithms. When making decisions about adopting machine learning or not, start by asking yourself if machine learning is outperforming your current methods. If it's not delivering better results, it might be wise to reconsider investing heavily in machine learning. Another key criterion is the availability of prebuilt solutions. Access to prebuilt models and one click deployment options can significantly reduce both the complexity and the cost of the implementation, making the machine learning adoption more feasible for you. So in conclusion, not every problem requires a machine learning solution. It's crucial to conduct a thorough cost benefit analysis in order to ensure that machine learning really adds real value to your problem. You need to start simple, which could save both time and resources, and ultimately you'd align your technology choices with your business goals and constraints.

Contents