Performance Evaluation Metrics for Machine-Learning Based Dissertation

Abstract

Evaluation metric plays an important role in obtaining the best possible classifier in the classification training. Thus, choosing an appropriate evaluation metric is an essential key for obtaining a selective and best possible classifier. The associated evaluation metrics have been reviewed systematically that are specifically considered as a discriminator for optimizing a classifier. In general, many possible classifiers use accuracy as a measure to classify the optimal solution during the classification evaluation. Thus, the measurement device that measures the performance of a classifier is considered as the evaluation metric. Different metrics are used to evaluate various characteristics of the classifier induced by the classification method.

Introduction[1] 

An importantaspect of the Machine Learning process is performance evaluation. The right choice of performance metrics is one of the most significant issues in evaluating performances. It is also a complex task. Therefore, it should be performed cautiously in order for the machine learning application to be reliable. Accuracy is used to assess the predictive capability of a model on the testing samples. Machine learning and data mining are the fields that use this major metric. Another alternate metric that has been used in pattern recognition and machine learning is the ROC curve. Thus, there are many performance metrics that have been developed for assessing the performance of ML algorithms. ¹

Evaluation of Machine Learning

The evaluation of categorized tasks is usually done by dividing the data set into a training data set and a testing data set. The machine learning method is then trained on the first set of data, while the testing data set calculates the performance indicators to assess the quality of the algorithm. ML algorithm’s common issue lies in accessing the limited testing and training data. Thus, overfitting can be a serious issue when assessing these programs. In order to tackle this problem, a common method is, to employ an X-Fold Cross-Validation. The cross-Validation method describes the process of dividing the entire data set into X parts and employing each set consecutively as the test data set while merging the other sets to the training data. Then the performance indicators are normalized overall validation processes. There is no ideal performance indicator for every topic that concerns the evaluation of machine learning algorithms since every method has its own flaws and advantages. ³

Image source[3] : Evaluating Learning Algorithms ⁸

Performance measures of ML[4] 

a. Confusion Matrix: The performance of a classification problem can be measured easily using this metric. Here, the output can be of two or more classes. A confusion matrix is a table with two dimensions i.e., “Actual” and “Predicted” and also, both the dimensions have “True Positives (TP)”, “True Negatives (TN)”, “False Positives (FP)”, “False Negatives (FN)”

b. Accuracy:  Accuracy is a metric to measure the accuracy of the model.

Accuracy = Correct Predictions / Total Predictions

Accuracy = Correct Predictions / Total Predictions

c. Precision & Recall: Precision is the ratio of True Positives (TP) and the total positive predictions. The recall is a True Positive Rate. All the positive points that are predicted positive are explained here.

The mean of precision and recall is termed as F measure.

d. ROC & AUC: ROC is a plot between True Positive Rate and False Positive Rate that is estimated by taking several threshold values of probability scores from the reverse sorted list given by a model.

Bayesian Inference

The recent development in machine learning has led many IT professionals to focus mainly on accelerating associated workloads, especially in machine learning. However, in the case of unsupervised learning, the Bayesian method often works better than machine learning with a limited or unlabelled data set, and can influence informative priors, and also have interpretable approaches. Bayesian inference model has become the most popular and accepted model over the years as it is a huge compliment to machine learning. Some recent revolutionizing research in machine learning accepts Bayesian techniques like generative Bayesian neural networks (BNN), adversarial networks (GAN), and variational autoencoders.

Recommended Algorithms

Through visual assessment, it has been proved that naive Bayes was the most successful algorithm for evaluating programming performance. Many detailed analyses were carried out statistically to find out if there were any considerable differences between the estimated accuracy of each of the algorithms. This is important as involved parties may prefer for choosing an algorithm that they would like to execute and must know if the use of such algorithm(s) would result in a significantly lower performance evaluation. The analysis identified that all of the ML algorithms, naive Bayes had comparably best performance evaluation and thus could be used to assess the performance of ML dissertation. Naive Bayes has been recommended as the best choice for predicting program performance. ⁵

Future Topics:[5] 

1. Evaluating and modifying performance measurement systems.

Performance measurement has become an emerging field during the last decades. Organizations have many motives for using performance measures but the most crucial one would be that they increase productivity when utilized properly.

2. Performance enhancement: a technique to support performance enhancement in industrial operations.

The main of this research is to: Build and assess a method that supports performance enhancement in industrial operations. This is performed through many case studies and literature research. The outcome is a systematically evaluated method for Performance Improvement.

3. Determining performance measures of the supply chain: prioritizing performance measures The main aim is to decrease costs and boost the profitability of organizations to thrive in the market of competition.

 4. A current state analysis technique for performance measurement methods.

Many organizations use the performance measurement (PM) method to support operational management and strategic management processes. This is chiefly important as it leads to modifications in organization strategy and PM systems.

5. Dynamic Performance Measurement Methods: A framework for organizations Approaches are dynamic naturally, while the current measurement systems are predictable and stable. Merging strategies with measurement methods is absurd and has created issues for organizations as the strategic framework modifies.

Conclusion[6] 

Improving the evaluation performance of an emerging workload, the most proficient way is to make use of existing systems. Another important research implemented is generic Bayesian frameworks for GPUs. As of now, Bayesian inference is considered the best combination of algorithm and hardware platform for performance evaluation. Performance evaluation aims to approximate the generalization accuracy of a model in future unknown data. In future research, research work can be carried out to improve the evaluation metrics even further. It would be better to test those metrics on various Machine Learning  cloud services to assess the services, to check how easy it is to use the metrics, and what type of data can be obtained using the metrics. Research work must be carried out in this direction to build a framework that would help in prioritizing the metrics and identify a set of conditions to join results from various metrics. ⁶

References:

1. A REVIEW ON EVALUATION METRICS FOR DATA CLASSIFICATION EVALUATIONS Hossin, M. and Sulaiman, M.N
2. AL-HAMADANI, MOKHALED N. A., M.S. Evaluation of the Performance of Deep Learning Techniques Over Tampered Dataset.
3. Benchmarking Machine Learning Methods for Performance Modeling of Scientific Applications Preeti Malakar, Prasanna Balaprakash, Venkatram Vishwanath, Vitali Morozov, and Kalyan Kumaran
4. Saurabh Raj, 2020, Evaluating the performance of Machine Learning models.
5. Statistical and Machine Learning Models to Predict Programming Performance by Susan Bergin.
6. Wang, Yu, 2020, Performance Analysis for Machine Learning Applications.
7. Yangguang Liu, Yangming Zhou, Shiting Wen, Chaogang Tang, 2019, A Strategy on Selecting Performance Metrics for Classifier Evaluation.
8. Nathalie Japkowicz & Mohak Shah, 2020, Evaluating Learning Algorithms: A Classification Perspective

Tutorsindia Academic Brand which assists the numerous Uk Reputed universitie’s students offers Machine Learning  dissertation and  Assignment help. A Genuine Academic Company with a  presence across the World including the US, UK & India.If you are looking for creative topics and full dissertation  across all the subjects. No doubt, we have a subject-Matter Expertise help you in writing the complete thesis. Get Your Master’s or PhD Research from your Academic Tutor with Unlimited Support!