Training an AI model

- Assume that you work for a nationwide used vehicle retailer and want to train an ML model to take the guesswork out of estimating the market value of used cars. How would you go about it? As the first step, you need to assemble a robust dataset containing relevant data on recent transactions in several thousand used cars. Let's say you have good data on close to 10,000 cars. For each transaction, your database should include data on the output variable, that is sales price, and a number of input features for each car. The features might include make and model, launch year, mileage, color, secondary options such as type of seat, entertainment system, et cetera, exterior and interior condition, repair history, accident history, and so forth. You would then get the data cleaned up and the various labels and formats standardized across all transactions. Since you already have an answer key in your training dataset, that is for each car, values of input features and the resulting sales price, it is clear that this context requires a supervised learning algorithm. It'll generally be wise for you to pick a few different, but pertinent supervised learning algorithms. Train an ML model on each algorithm and then pick the algorithm that beats others in yielding the best model. The next step is to specify several hyperparameters for each algorithm. These are attributes that are specified ex ante to guide the algorithm in carrying out the training task. Some of the critical hyperparameters include a measure of model performance, also called the loss function, that is how accurately the model estimates sales prices when compared with actual sales prices. How the dataset should be split between a training set, a validation set, and a test set. Training set refers to data used for training the machine learning model. Validation set refers to data used for measuring the performance of a trained model and to decide how to fine tune the hyperparameters before running the retraining again. Finally, test set refers to data used to measure and compare the accuracy of the best trained ML model yielded by each of the different algorithms, the number of epochs, that is the number of times you want the entire training set to go through the ringer to improve the accuracy of the ML model being trained for that particular algorithm. Within each epoch, there often will be a number of iterations, each using a smaller batch from the larger training set. Specify the number of iterations and the batch size for each iteration. The number of hidden layers in the neural network. Hidden layers increase the model's ability to look for complex interrelationships among the various input features that determine the output value. It's the multiplicity of hidden layers that differentiates deep neural networks from shallow neural networks. Once the hyperparameters are specified, you let each algorithm train its ML model. After each round of training, you would use the validation data to measure the accuracy of the model, and then fine tune the hyperparameters, redo the training, measure the accuracy again, and repeat the process until you feel that you have the best possible model for this algorithm. You should thus end up with the best trained ML model from each of the different algorithms. You would then assess and compare the performance of these best models using the test set. After this, you would know which is the best of the best model and thus the best algorithm. The final step is to decide whether even the best ML model is good enough when compared with human judgment. If it's better, you may make this your primary valuation tool. If it's good enough, but not better, you may use it to augment human judgment, but if it's much worse, you may shelve it until better data or better algorithms become available.