Supervised Learning - Traditional Methods

Data Mining Supervised Learning delves into traditional machine learning models, a crucial step within the CRISP-ML(Q) methodology. This section focuses on supervised learning, a powerful form of predictive modeling encompassing various techniques like classification models, shallow machine learning models, ensemble models, and regression models, including the black box approach.

We explore fundamental concepts like probability, joint probability, and the Bayes Rule, with a practical application of the Naive Bayes algorithm. While effective, Naive Bayes isn't ideal for datasets with numerous numeric features, as these require conversion to categorical data via discretization or binning. However, Naive Bayes excels at handling missing values and operates under the assumption of class-conditional independence. To address the issue of zero probabilities for unseen words, we introduce the Laplace Estimator, developed by French mathematician Pierre-Simon Laplace. The Laplace estimator, with a default value of 1 (though any value is applicable), prevents zero probabilities and improves model robustness.

Next, we examine the K-Nearest Neighbor (kNN) Classifier, also known as lazy learning, memory-based reasoning, example-based reasoning, instance-based learning, case-based reasoning, and rote learning. We differentiate k-means from kNN and analyze the impact of different k values – 1, 2, 3, and 7 nearest neighbors. The k parameter, a crucial hyperparameter, determines the number of neighbors considered. We also discuss baseline models (accuracy equal to the majority class) and the importance of achieving accuracy above 80% for prediction models. Finally, we analyze the bias-variance trade-off in the context of kNN and explore its practical applications and importance.

The Decision Tree algorithm, a rules-based algorithm, is then explored. We cover the fundamentals of decision trees and the process of building decision trees, focusing on the greedy algorithm for optimal tree construction and attribute selection. A decision tree is a tree-like structure where internal nodes represent attributes, branches represent outcomes, and leaf nodes represent class labels. We detail the three node types: root, branch, and leaf. The tree-building process involves using training data to determine which variable to split at each node, the optimal split value, and the decision to stop or continue splitting. Terminal nodes are assigned labels. The greedy algorithm employs a top-down, recursive divide-and-conquer approach. We further analyze the Greedy Approach, Entropy, and Information Gain – key concepts in decision tree construction.