MLP: Machine Learning Practice

Topics covered

• Introduction to Pandas

  • Purpose of Pandas library.
  • Difference between NumPy arrays and Pandas objects.
  • Handling tabular data.

• Pandas Series

  • Creating Series.
  • Index and values.
  • Index-based selection.
  • Basic operations on Series.

• DataFrames

  • Creating DataFrame.
  • Rows and columns.
  • Column names and index.
  • Inspecting the DataFrame.

• Data selection

  • loc and iloc.
  • Conditional filtering.
  • Selecting multiple columns.
  • Accessing rows and columns.

• Data modification

  • Dropping Columns.
  • Filtering Rows.
  • Initial inspection of missing values.

• Statistical summaries

  • describe(), mean(), median(), std().
  • Unique values and value counts.
  • Introduction of group-wise summary.

Topics covered

• Scikit-learn introduction

  • Role of scikit-learn.
  • Estimator, transformer and predictor.
  • fit(), transform() and predict() methods.

• Data loading

  • Built-in datasets.
  • load_*, fetch_* and make_* functions.
  • Dataset features and targets.

• Missing-value handling

  • Identifying missing values.
  • Mean, median and most-frequent imputation.
  • SimpleImputer.
  • Basic idea of KNNIMputer.

• Categorical data

  • Nominal and ordinal features.
  • One-hot encoding.
  • Ordinal encoding.
  • OneHotEncoder and OrdinalEncoder.

• Numerical transformations

  • Need for feature scaling.
  • Standardization.
  • Min-max scaling.
  • StandardScaler, MinMaxScaler.

• Outlier handling

  • Identifying outliers.
  • Impact of outliers on model performance.
  • Outlier-sensitive transformations.

• Feature selection

  • Filter-based feature selection.
  • Wrapper-based feature selection.
  • Correlation and role of statistical scores.

• Dimensionality reduction

  • Practical introduction of PCA.
  • Explained variance.
  • Reducing feature space.

Topics covered

• Data transformation workflow

  • Applying multiple preprocessing steps in sequence.
  • Consistent transformations between training and testing data.
  • Concept of data leakage.

• Scikit-learn pipelines

  • Pipeline.
  • Transformer chaining.
  • Pipeline through model training.

• Composite transformations

  • ColumnTransformer.
  • Separate transformations on numerical and categorical columns.
  • Combining multiple preprocessing branches.

• Feature extraction and imputation

  • Data extraction.
  • To impute missing values.
  • Visualizing feature distributions.

• Feature selection and PCA

  • Integrating feature selection into the pipeline.
  • Using PCA as a preprocessing step.

• Class imbalance

  • Imbalanced classification datasets.
  • Majority and minority classes.
  • Introduction of oversampling and undersampling.
  • Importance of evaluation metrics for imbalance.

Topics covered

• Linear regression

  • Regression problem and target variable.
  • Linear model and coefficients.
  • Prediction and residuals.

• Model evaluation

  • Training and testing data.
  • Regression metrics: MAE, MSE, RMSE and R².
  • Model performance comparison.

• Polynomial regression

  • Modeling non-linear relationships.
  • Polynomial features.
  • Underfitting and overfitting.

• Regularization

  • Controlling model complexity.
  • Introduction of Ridge and Lasso.
  • Effect of regularization strength.

• Classification models

  • Logistic regression.
  • Naive Bayes.
  • K-Nearest Neighbours.
  • Support Vector Machines.
  • Decision trees.

• Ensemble learning

  • Voting classifiers.
  • Bagging.
  • Random forests.
  • Boosting: AdaBoost and gradient boosting.

• Neural networks

  • Multi-Layer Perceptron (MLP).
  • Hidden layers and activation functions.
  • Scikit-learn implementation of Neural networks.

Topics covered

• Dataset exploration

  • California Housing dataset.
  • Features and target variables.
  • Data distributions and correlations.
  • Exploratory Data Analysis (EDA).

• Baseline model

  • Simple baseline prediction.
  • Comparison of baseline and trained model.

• Linear regression

  • LinearRegression.
  • Model fitting and prediction.
  • Interpreting coefficients.

• Gradient descent

  • Iterative optimization.
  • SGDRegressor.
  • Learning rate and convergence.

• KNN regression

  • Prediction based on nearest neighbors.
  • Distance metrics.
  • Effect of Sankhya of Neighbors.

• Tree-based regression

  • Decision tree regressor.
  • Bagging and random forest regressor.
  • AdaBoost and gradient boosting regressor.

• MLP regression

  • Neural network-based regression.
  • Hidden layers and model training.
  • Performance comparison.

Topics covered

• Classification fundamentals

  • Classification problem.
  • Binary and multiclass classification.
  • Features and class labels.

• Perceptron

  • Linear decision boundary.
  • Binary classification.
  • Multiclass perceptron.

• Logistic regression

  • Probability-based classification.
  • Sigmoid function.
  • Decision threshold.

• Ridge classifier

  • Regularized linear classification.
  • Zero detector example.

• Naive Bayes

  • Bayes theorem.
  • Conditional independence assumption.
  • Basic use case of text classification.

• Softmax regression

  • Multiclass probability prediction.
  • Softmax function.
  • MNIST digit classification.

• KNN classification

  • Distance-based classification.
  • Choice of k.
  • Classification boundaries.

• Support Vector Classifier

  • Maximum-margin classification.
  • Introduction of Kernel-based classification.

• Decision trees and ensembles

  • Decision tree classification.
  • Random forest classifier.
  • AdaBoost and gradient boosting classifiers.

• MLP classifier

  • Neural network-based classification.
  • Hidden layers and output probabilities.

Topics covered

• Large-scale machine learning

  • Model training on large datasets.
  • Batch-based learning.
  • Concept of incremental learning.
  • Use of partial_fit().

• Confusion matrix

  • True Positive (TP).
  • True Negative (TN).
  • False Positive (FP).
  • False Negative (FN).

• Classification metrics

  • Accuracy.
  • Precision.
  • Recall.
  • F1-score.

• Metric interpretation

  • Trade-off of precision and recall.
  • Limitations of accuracy in unbalanced data.
  • Selecting metrics for different classification objectives.

• Multiclass evaluation

  • Multiclass confusion matrix.
  • Concept of macro, micro and weighted averaging.
  • Per-class performance.

• Model comparison

  • To compare the performance of different classifiers.
  • Identifying error patterns.
  • Deciding next steps based on evaluation results.

Topics covered

• Support Vector Machines

  • Maximum-margin hyperplane.
  • Support vectors.
  • Margin and decision boundary.

• Linear and nonlinear classification

  • Linear SVM.
  • Nonlinear decision boundaries.
  • Concept of kernel trick.

• SVM kernels

  • Linear kernel.
  • Polynomial kernel.
  • RBF kernel.
  • Effect of kernel parameters.

• Model parameters

  • Regularization parameter C.
  • Kernel coefficient γ.
  • Hyperparameter selection.

• Model tuning

  • Cross-validation.
  • Grid search.
  • Randomized search.
  • Validation performance comparison.

• Classification workflow

  • Data preprocessing.
  • Scaling before SVM.
  • Training and evaluation.

Note: The practical placement of SVM and tuning may change depending on lecture playlists and course versions. This week is considered a revision-oriented topic grouping, not an official fixed week title.

Topics covered

• Problem understanding

  • Converting business problem into ML task.
  • Identifying regression, classification or clustering.
  • Defining success metrics.

• Data exploration

  • Dataset overview.
  • Feature distributions.
  • Missing values and outliers.
  • Feature relationships.

• Data preparation

  • Train-test split.
  • Missing-value imputation.
  • Encoding and scaling.
  • Preprocessing pipelines.

• Model selection

  • Baseline model.
  • Multiple candidate algorithms.
  • Cross-validation.

• Hyperparameter tuning

  • Grid search and randomized search.
  • Comparing model parameters.
  • To inspect overfitting and underfitting.

• Project workflow

  • Introduction of Kaggle platform.
  • Notebook-based workflow.
  • Documenting the results.
  • Preparing final model and predictions.

Topics covered

• Introduction to recommender systems

  • Recommendation problems.
  • User-item interactions.
  • Rating-based recommendation.
  • Introduction to the MovieLens dataset.

• Exploratory data analysis

  • Analysis of users, items and ratings.
  • Rating distributions.
  • User-item interaction matrix.

• Content-based filtering

  • Item features.
  • Similarity-based recommendations.
  • Recommendations based on user preferences.

• Collaborative filtering

  • User-user collaborative filtering.
  • Item-item collaborative filtering.
  • Similarity measures.

• Matrix factorization

  • Sparse user-item matrix.
  • Latent factors.
  • Basic idea of Matrix decomposition.

• Recommendation evaluation

  • Predicted ratings and actual ratings.
  • Recommendation quality.
  • Introduction of cold-start problem.

Topics covered

• Time-series data

  • Time-indexed observations.
  • Representing time-series data in Pandas.
  • Date-time indexing.

• Resampling

  • Frequency conversion.
  • Daily, weekly and monthly aggregation.
  • Concept of upsampling and downsampling.

• Time-series visualization

  • Time plots.
  • Trends and seasonal patterns.
  • Moving averages.

• Classical decomposition

  • Trend component.
  • Seasonal component.
  • Residual component.
  • Additive and multiplicative decomposition.

• STL decomposition

  • Seasonal and Trend decomposition using Loess.
  • Separating seasonal patterns.
  • Comparison with classical decomposition.

• Stationarity

  • Stationary and non-stationary series.
  • Time-dependent changes of mean and variance.
  • Impact of trend and seasonality on stationarity.

• Making data stationary

  • Differencing.
  • Transformations.
  • Decomposition-based understanding.

Topics covered

• Introduction to transfer learning

  • Concept of transfer learning.
  • Pretrained models.
  • Feature extraction and fine-tuning.

• Neural network recap

  • Neural network architecture.
  • Forward propagation.
  • Overview of training and optimization.

• Transfer learning workflow

  • Loading pretrained model.
  • Feature extraction.
  • To replace the final classification layer.
  • Adapting the model to the new dataset.

• Fine-tuning

  • To unfreeze selected layers.
  • Role of learning rate.
  • Feature reuse and retraining.

• Practical implementation

  • Image dataset preparation.
  • Use of pretrained CNN.
  • Training and validation.
  • Performance comparison.

• Transfer learning evaluation

  • Training from scratch vs transfer learning.
  • Data requirements.
  • Computational cost and accuracy trade-offs.