Differences

This shows you the differences between two versions of the page.

--- machine_learning_roadmap [2023/10/23 16:55] – created demiurge
+++ machine_learning_roadmap [2023/10/23 17:02] (current) – [Learn Advanced Python] demiurge
@@ Line 1: / Line 1: @@
-# Machine Learning Roadmap
+====== Machine Learning Roadmap ======
-So you want to *learn*  Machine Learning? It will be a long journey - one that requires a solid grasp of the fundamentals. Try and not skip any of the stages, and move on to the next once you have a full understanding of the current one. Good luck!
+So you want to *learn* Machine Learning? It will be a long journey - one that requires a solid grasp of the fundamentals. Try and not skip any of the stages, and move on to the next once you have a full understanding of the current one. Good luck!
----
+===== Mathematics and Calculus =====
-[TOC3]
----
-### Mathematics and Calculus
+==== Linear Algebra ====
-#### 1. Linear Algebra
+This is what essentially provides the mathematical framework for understanding and manipulating vectors and matrices, which are the building blocks of almost any ML algorithm. A full grasp of these concepts is **essential**. As always, [Khan Academy]([[https://www.khanacademy.org|https://www.khanacademy.org]]) is a great resource. Below are a list of the essentials, along with the appropriate Khan Academy course materials. You can always choose your own course if you wish to.
-This is what essentially provides the mathematical framework for understanding and manipulating vectors and matrices, which are the building blocks of almost any ML algorithm. A full grasp of these concepts is **essential**. As always, [Khan Academy](https://www.khanacademy.org) is a great resource. Below are a list of the essentials, along with the appropriate Khan Academy course materials. You can always choose your own course if you wish to.
-. [Vectors and Spaces](https://www.khanacademy.org/math/linear-algebra/vectors-and-spaces)
+[Vectors and Spaces]([[https://www.khanacademy.org/math/linear-algebra/vectors-and-spaces|https://www.khanacademy.org/math/linear-algebra/vectors-and-spaces]])
-. [Matrices](https://www.khanacademy.org/math/precalculus/x9e81a4f98389efdf:matrices) and [Matrix Transformations](https://www.khanacademy.org/math/linear-algebra/matrix-transformations)
-#### 2. Calculus
+[Matrices]([[https://www.khanacademy.org/math/precalculus/x9e81a4f98389efdf:matrices|https://www.khanacademy.org/math/precalculus/x9e81a4f98389efdf:matrices]])
-Calculus, and particularly derivatives and gradients, play a key role in optimization algorithms used in ML. You will rely on Calculus for optimization techniques such as [gradient descent](https://en.wikipedia.org/wiki/Gradient_descent), and key components of DL such as [Backpropagation](https://en.wikipedia.org/wiki/Backpropagation) - which heavily relies on the chain rule of calculus. Studying integrals and derivatives are fundamental for understanding the rate of change in functions and how they behave, which is crucial for interpreting and modeling real-world use-cases.
-. [Integrals](https://www.khanacademy.org/math/integral-calculus/ic-integration)
+[Matrix Transformations]([[https://www.khanacademy.org/math/linear-algebra/matrix-transformations|https://www.khanacademy.org/math/linear-algebra/matrix-transformations]])
-. [Differential Equations](https://www.khanacademy.org/math/integral-calculus/ic-diff-eq)
-. [Application of Integrals](https://www.khanacademy.org/math/integral-calculus/ic-int-app)
+==== Calculus ====
-. [Parametric equations, polar coordinates, and vector-valued functions](https://www.khanacademy.org/math/integral-calculus/ic-adv-funcs)
-. [Series](https://www.khanacademy.org/math/integral-calculus/ic-series)
+Calculus, and particularly derivatives and gradients, play a key role in optimization algorithms used in ML. You will rely on Calculus for optimization techniques such as [gradient descent]([[https://en.wikipedia.org/wiki/Gradient_descent|https://en.wikipedia.org/wiki/Gradient_descent]]), and key components of DL such as [Backpropagation]([[https://en.wikipedia.org/wiki/Backpropagation|https://en.wikipedia.org/wiki/Backpropagation]]) - which heavily relies on the chain rule of calculus. Studying integrals and derivatives are fundamental for understanding the rate of change in functions and how they behave, which is crucial for interpreting and modeling real-world use-cases.
-. [Gradients](https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives)
+[Integrals]([[https://www.khanacademy.org/math/integral-calculus/ic-integration|https://www.khanacademy.org/math/integral-calculus/ic-integration]])
+[Differential Equations]([[https://www.khanacademy.org/math/integral-calculus/ic-diff-eq|https://www.khanacademy.org/math/integral-calculus/ic-diff-eq]])
+[Application of Integrals]([[https://www.khanacademy.org/math/integral-calculus/ic-int-app|https://www.khanacademy.org/math/integral-calculus/ic-int-app]])
+[Parametric equations, polar coordinates, and vector-valued functions]([[https://www.khanacademy.org/math/integral-calculus/ic-adv-funcs|https://www.khanacademy.org/math/integral-calculus/ic-adv-funcs]])
+[Series]([[https://www.khanacademy.org/math/integral-calculus/ic-series|https://www.khanacademy.org/math/integral-calculus/ic-series]])
+[Gradients]([[https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives|https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives]])
+==== Probability and Statistics ====
-#### 3. Probability and Statistics
 Another essential building-block. Probability theory provides a math framework for quantifying *uncertainty*. In ML, models often need to make predictions or decisions based on incomplete or noisy data. With probability, we can easily express that uncertainty and reason about it. There's myriad other reasons for learning probability, of course; just keep in mind that Language Models generate text by taking your input and calculating the probability distribution of the next sequence of words that would follow it, and pick the most likely output to complete your input text.
-. [The Entire Khan Academy Statistics and Probability course](https://www.khanacademy.org/math/statistics-probability)
+. [The Entire Khan Academy Statistics and Probability course]([[https://www.khanacademy.org/math/statistics-probability|https://www.khanacademy.org/math/statistics-probability]]) >You can take only the lessons you think might be important and then take the Course Challenge.\\
->You can take only the lessons you think might be important and then take the Course Challenge.
+. Discrete and continuous probability distributions: [binomial]([[https://www.khanacademy.org/math/math3/x5549cc1686316ba5:binomial-prob|https://www.khanacademy.org/math/math3/x5549cc1686316ba5:binomial-prob]]) and [normal (Gaussian)]([[https://www.khanacademy.org/math/math3/x5549cc1686316ba5:normal-dist|https://www.khanacademy.org/math/math3/x5549cc1686316ba5:normal-dist]]). 3. [Bayesian Statistics]([[https://dataorigami.net/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/|https://dataorigami.net/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/]])
-. Discrete and continuous probability distributions: [binomial](https://www.khanacademy.org/math/math3/x5549cc1686316ba5:binomial-prob) and [normal (Gaussian)](https://www.khanacademy.org/math/math3/x5549cc1686316ba5:normal-dist).
-. [Bayesian Statistics](https://dataorigami.net/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/)
 That should probably be enough for Math. I might've missed a few essential things, but I will add more as I come across them.
-###  Programming
+===== Programming =====
 The current programming language dominating the ML community is **Python**. Not surprising, since the ease of use allows you to focus on writing efficient code without needing to spend too much time learning the intricacies of the language's syntax. There's a good chance you already know Python, but we'll go over the basic steps anyway.
-##### Learn Python Basics
-The `roadmap.sh` [Python Developer roadmap](https://roadmap.sh/python) is an incredibly useful resource for this. What it essentially boils down to, however, is:
-. Learn the Basic Syntax and Data Types
+==== Learn Python Basics ====
-You'll need to familiarize yourself with Python's syntax, variables, data types (integers, floats, strings, lists, dicts), and basic operations (arithmetic, string manipulation, indexing, slicing).
-. Control Flow
-Understand conditional statements (`if`, `elif`, `else`), loops (`for`, `while`), and logical operators (`and`, `or`, `not`). Very important for implementing decision-making and repetition in your code.
-. Functions and modules
-Learn how to define and use functions to encapsulate reusable blocks of code. Also, you'll need to understand how to import and utilize modules (libs).
-. Data Structures and Manipulation
-Get yourself acquainted with fundamental data structures like lists, tuples, sets, and dictionaries. Learn how to manipulate and transform data.
-. NumPy
-A fundamental library for scientific computing in Python. You will need to gain proficiency in using NumPy arrays for efficient numerical computations.
-. Pandas
-You will often need Pandas DataFrames to clean, transform, filter, aggregate, and analyze your datasets.
-. Plotting and Data Visualization
-Become familiar with libraries such as [Matplotlib](https://matplotlib.org/) and [Seaborn](https://seaborn.pydata.org/) for creating plots, charts, and visualization. Not strictly necessary, but recommended.
-##### Learn Advanced Python
+`roadmap.sh` [Python Developer roadmap]([[https://roadmap.sh/python|https://roadmap.sh/python]]) is an incredibly useful resource for this. What it essentially boils down to, however, is:
+. Learn the Basic Syntax and Data Types You'll need to familiarize yourself with Python's syntax, variables, data types (integers, floats, strings, lists, dicts), and basic operations (arithmetic, string manipulation, indexing, slicing). 2. Control Flow Understand conditional statements (`if`, `elif`, `else`), loops (`for`, `while`), and logical operators (`and`, `or`, `not`). Very important for implementing decision-making and repetition in your code. 3. Functions and modules Learn how to define and use functions to encapsulate reusable blocks of code. Also, you'll need to understand how to import and utilize modules (libs). 4. Data Structures and Manipulation Get yourself acquainted with fundamental data structures like lists, tuples, sets, and dictionaries. Learn how to manipulate and transform data. 5. NumPy A fundamental library for scientific computing in Python. You will need to gain proficiency in using NumPy arrays for efficient numerical computations. 6. Pandas You will often need Pandas DataFrames to clean, transform, filter, aggregate, and analyze your datasets. 7. Plotting and Data Visualization Become familiar with libraries such as [Matplotlib]([[https://matplotlib.org/|https://matplotlib.org/]]) and [Seaborn]([[https://seaborn.pydata.org/|https://seaborn.pydata.org/]]) for creating plots, charts, and visualization. Not strictly necessary, but recommended.
+==== Learn Advanced Python ====
 At this stage, you'll be sufficiently familiar with Python and ready to tackle the ML aspects of Python. Very exciting.
-. Machine Learning Libraries
+. Machine Learning Libraries Explore the popular ML libraries, such as [PyTorch]([[https://www.youtube.com/watch?v=V_xro1bcAuA&|https://www.youtube.com/watch?v=V_xro1bcAuA&]]), [TensorFlow]([[https://www.tensorflow.org/resources/learn-ml|https://www.tensorflow.org/resources/learn-ml]]), or [scikit-learn]([[https://scikit-learn.org/|https://scikit-learn.org/]]). It's recommended to focus on only one, and as of now, PyTorch is the most popular.
-Explore the popular ML libraries, such as [PyTorch](https://www.youtube.com/watch?v=V_xro1bcAuA&), [TensorFlow](https://www.tensorflow.org/resources/learn-ml), or [scikit-learn](https://scikit-learn.org/). It's recommended to focus on only one, and as of now, PyTorch is the most popular.
-. Object-Oriented Programming (OOP)
+. Object-Oriented Programming (OOP) Get yourself comfortable with the principles of OOP, including classes, objects, inheritance, and encapsulation. Allows for modular and organized code design.
-Get yourself comfortable with the principles of OOP, including classes, objects, inheritance, and encapsulation. Allows for modular and organized code design.
 ### Machine Learning Concepts
-At this point, you can follow whatever ML course you're comfortable with. A popular recommendation is [fastai](https://course.fast.ai). It's a great resource for almost everything you'll need to learn about ML. Otherwise, pick any of the concepts below that interest you and get to learning.
+At this point, you can follow whatever ML course you're comfortable with. A popular recommendation is [fastai]([[https://course.fast.ai|https://course.fast.ai]]). It's a great resource for almost everything you'll need to learn about ML. Otherwise, pick any of the concepts below that interest you and get to learning.
+. **Supervised Learning** This [Coursera]([[https://www.coursera.org/learn/machine-learning|https://www.coursera.org/learn/machine-learning]]) curriculum on Supervised ML will be useful. As of writing this guide, the course is completely free. The main points to learn are:
+  - Classification: Predicting discrete class labels.
+  - Regression: Predicting continuous values.
+. **Unsupervised Learning**  This [course]([[https://www.udacity.com/course/machine-learning-unsupervised-learning--ud741|https://www.udacity.com/course/machine-learning-unsupervised-learning--ud741]]) should provide the adequate amount of knowledge on Unsupervised Learning. The main points to learn are:
+  - Clustering: Grouping similar data points together.
+  - Dimensionality Reduction: Reducing the number of input features while preserving important information.
+  - Anomaly Detection: Identifying rare of abnormal instances in the data.
+. **Reinforcement Learning**  Coursera provides this [course]([[https://www.coursera.org/specializations/reinforcement-learning|https://www.coursera.org/specializations/reinforcement-learning]]) on Reinforcement Learning, which should be a good starting point.
+. **Linear Regression**  This [resource]([[https://machinelearningmastery.com/linear-regression-for-machine-learning/|https://machinelearningmastery.com/linear-regression-for-machine-learning/]]) should be a useful starting point. The main outtakes are:
+  - Understanding linear regression models and assumptions.
+  - Cost functions, including mean squared error.
+  - Gradient descent for parameter optimization.
+  - Evaluation metrics for regression models.
+. **Logistic Regression**  Read through [this resource]([[https://machinelearningmastery.com/logistic-regression-for-machine-learning/|https://machinelearningmastery.com/logistic-regression-for-machine-learning/]]) as a starting point. Main outtakes will be:
+  - Modeling binary classification problems with logistic regression.
+  - Sigmoid function and interpretation of probabilities.
+  - Maximum likelihood estimation and logistic loss.
+  - Regularization techniques for logistic regression.
+. **Decision Trees and Random Forests**  This incredible resource by [Jake VanderPlas]([[https://jakevdp.github.io/PythonDataScienceHandbook/05.08-random-forests.html|https://jakevdp.github.io/PythonDataScienceHandbook/05.08-random-forests.html]]) should be extremely useful. Main outtake are:
+  - Basics of decision tree learning.
+  - Splitting criteria and handling categorical variables.
+  - Ensemble learning with random forests.
+  - Feature importance and tree visualization.
+. **Support Vector Machines (SVM)**  Read through [this resource]([[https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47|https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47]]). The main outtakes are:
+  - Formulation of SVMs for binary classification.
+  - Kernel trick and non-linear decision boundaries.
+  - Soft margin and regularization in SVMs.
+  - SVMs for multi-class classification.
-. **Supervised Learning**
+. **Clustering**  Read through this excellent [Google for Developers]([[https://developers.google.com/machine-learning/clustering|https://developers.google.com/machine-learning/clustering]]) course on Clustering. The main outtakes are:
-This [Coursera](https://www.coursera.org/learn/machine-learning) curriculum on Supervised ML will be useful. As of writing this guide, the course is completely free. The main points to learn are:
-    - Classification: Predicting discrete class labels.
-    - Regression: Predicting continuous values.
-. **Unsupervised Learning**
+  - Overview of unsupervised learning and clustering.
-This [course](https://www.udacity.com/course/machine-learning-unsupervised-learning--ud741) should provide the adequate amount of knowledge on Unsupervised Learning. The main points to learn are:
+  - K-means clustering algorithm and initialization methods.
-    - Clustering: Grouping similar data points together.
+  - Hierarchical clustering and density-based clustering.
-    - Dimensionality Reduction: Reducing the number of input features while preserving important information.
+  - Evaluating clustering performance.
-    - Anomaly Detection: Identifying rare of abnormal instances in the data.
-. **Reinforcement Learning**
+. **Neural Networks and Deep Learning**  The heart of the matter. Read through the papers for each:
-Coursera provides this [course](https://www.coursera.org/specializations/reinforcement-learning) on Reinforcement Learning, which should be a good starting point.
-. **Linear Regression**
+  - [Deep Learning in Neural Networks: an Overview]([[https://arxiv.org/abs/1404.7828|https://arxiv.org/abs/1404.7828]])
-This [resource](https://machinelearningmastery.com/linear-regression-for-machine-learning/) should be a useful starting point. The main outtakes are:
+  - [An Introduction to Convolutional Neural Networks]([[https://arxiv.org/abs/1511.08458|https://arxiv.org/abs/1511.08458]])
-    - Understanding linear regression models and assumptions.
+  - [Recurrent Neural Networks (RNNs): A gentle Introduction and Overview]([[https://arxiv.org/abs/1912.05911|https://arxiv.org/abs/1912.05911]])
-    - Cost functions, including mean squared error.
+  - [Three Mechanisms of Weight Decay Regularization]([[https://arxiv.org/abs/1810.12281|https://arxiv.org/abs/1810.12281]])
-    - Gradient descent for parameter optimization.
+  - [Layer Normalization]([[https://arxiv.org/abs/1607.06450|https://arxiv.org/abs/1607.06450]])
-    - Evaluation metrics for regression models.
+  - [Attention Is All You Need]([[https://arxiv.org/abs/1706.03762|https://arxiv.org/abs/1706.03762]])
-. **Logistic Regression**
+. **Evaluation and Validation**  Read the following papers:
-Read through [this resource](https://machinelearningmastery.com/logistic-regression-for-machine-learning/) as a starting point. Main outtakes will be:
-    - Modeling binary classification problems with logistic regression.
-    - Sigmoid function and interpretation of probabilities.
-    - Maximum likelihood estimation and logistic loss.
-    - Regularization techniques for logistic regression.
-. **Decision Trees and Random Forests**
+  - [Using J-K fold Cross Validation to Reduce Variance When Tuning NLP Models]([[https://arxiv.org/abs/1806.07139|https://arxiv.org/abs/1806.07139]])
-This incredible resource by [Jake VanderPlas](https://jakevdp.github.io/PythonDataScienceHandbook/05.08-random-forests.html) should be extremely useful. Main outtake are:
+  - [Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data]([[https://arxiv.org/abs/2001.00980|https://arxiv.org/abs/2001.00980]])
-    - Basics of decision tree learning.
-    - Splitting criteria and handling categorical variables.
-    - Ensemble learning with random forests.
-    - Feature importance and tree visualization.
-. **Support Vector Machines (SVM)**
+And [this HuggingFace guide]([[https://huggingface.co/docs/evaluate/types_of_evaluations|https://huggingface.co/docs/evaluate/types_of_evaluations]]) on Evaluating ML models.
-Read through [this resource](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47). The main outtakes are:
-    - Formulation of SVMs for binary classification.
-    - Kernel trick and non-linear decision boundaries.
-    - Soft margin and regularization in SVMs.
-    - SVMs for multi-class classification.
-. **Clustering**
+. **Feature Engineering and Dimensionality Reduction**  Take a look at [this article]([[https://towardsdatascience.com/feature-selection-and-dimensionality-reduction-f488d1a035de|https://towardsdatascience.com/feature-selection-and-dimensionality-reduction-f488d1a035de]]) for a general oveeview. Also read these papers:
-Read through this excellent [Google for Developers](https://developers.google.com/machine-learning/clustering) course on Clustering. The main outtakes are:
-    - Overview of unsupervised learning and clustering.
-    - K-means clustering algorithm and initialization methods.
-    - Hierarchical clustering and density-based clustering.
-    - Evaluating clustering performance.
-. **Neural Networks and Deep Learning**
+  - [Beyond One-hot Encoding: lower dimensional target embedding]([[https://arxiv.org/abs/1806.10805|https://arxiv.org/abs/1806.10805]])
-The heart of the matter. Read through the papers for each:
+  - [A Tutorial on Principal Component Analysis]([[https://arxiv.org/abs/1404.1100|https://arxiv.org/abs/1404.1100]])
-    - [Deep Learning in Neural Networks: an Overview](https://arxiv.org/abs/1404.7828)
-    - [An Introduction to Convolutional Neural Networks](https://arxiv.org/abs/1511.08458)
-    - [Recurrent Neural Networks (RNNs): A gentle Introduction and Overview](https://arxiv.org/abs/1912.05911)
-    - [Three Mechanisms of Weight Decay Regularization](https://arxiv.org/abs/1810.12281)
-    - [Layer Normalization](https://arxiv.org/abs/1607.06450)
-    - [Attention Is All You Need](https://arxiv.org/abs/1706.03762)
-. **Evaluation and Validation**
+. **Model Selection and Hyperparameter Tuning**  This is where you're finally dabbling in model training. Good job! You will need to learn:
-Read the following papers:
-    - [Using J-K fold Cross Validation to Reduce Variance When Tuning NLP Models](https://arxiv.org/abs/1806.07139)
-    - [Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data](https://arxiv.org/abs/2001.00980)
-And [this HuggingFace guide](https://huggingface.co/docs/evaluate/types_of_evaluations) on Evaluating ML models.
-. **Feature Engineering and Dimensionality Reduction**
+  - Grid search, random search, and Bayesian optimization for hyperparameter tuning.
-Take a look at [this article](https://towardsdatascience.com/feature-selection-and-dimensionality-reduction-f488d1a035de) for a general oveeview.
+  - Model selection techniques, including nested cross-validation.
-Also read these papers:
+  - Overfitting, underfitting, and bias-variance tradeoff.
-    - [Beyond One-hot Encoding: lower dimensional target embedding](https://arxiv.org/abs/1806.10805)
+  - Performance comparison of different models.
-    - [A Tutorial on Principal Component Analysis](https://arxiv.org/abs/1404.1100)
-. **Model Selection and Hyperparameter Tuning**
+### Train your own model! You're now ready to pre-train your own model, or fine-tune an existing one! For this, you should look into [Transformers]([[https://github.com/huggingface/transformers|https://github.com/huggingface/transformers]]), a framework for developing state-of-the-art Machine Learning models using [PyTorch]([[https://pytorch.org|https://pytorch.org]]).
-This is where you're finally dabbling in model training. Good job! You will need to learn:
-    - Grid search, random search, and Bayesian optimization for hyperparameter tuning.
-    - Model selection techniques, including nested cross-validation.
-    - Overfitting, underfitting, and bias-variance tradeoff.
-    - Performance comparison of different models.
-### Train your own model!
+The [HuggingFace Transformers docs]([[https://huggingface.co/docs/transformers/index|https://huggingface.co/docs/transformers/index]]) go into excruciating detail on how to use Transformers. Make sure to read them, as they might be all you'll need. Good luck on your journey!
-You're now ready to pre-train your own model, or fine-tune an existing one! For this, you should look into [Transformers](https://github.com/huggingface/transformers), a framework for developing state-of-the-art Machine Learning models using [PyTorch](https://pytorch.org).
-The [HuggingFace Transformers docs](https://huggingface.co/docs/transformers/index) go into excruciating detail on how to use Transformers. Make sure to read them, as they might be all you'll need. Good luck on your journey!
+### Stay Updated and Engage in ML community At this point, you know all the essentials. ML is an ever-advancing field, with new innovations emerging everyday. You'll need to stay abreast of the latest developments, hound [arXiv]([[https://arxiv.org|https://arxiv.org]]) for the latest published ML papers, and attend conferences if you're able to.
-### Stay Updated and Engage in ML community
-At this point, you know all the essentials. ML is an ever-advancing field, with new innovations emerging everyday. You'll need to stay abreast of the latest developments, hound [arXiv](https://arxiv.org) for the latest published ML papers, and attend conferences if you're able to.