TK
Home

My Experience Learning Machine Learning & Mathematics

• 7 min read
A photo of the universePhoto by Raphael Nogueira

When reading 'The Art of Doing Science and Engineering' by Richard Hamming last year, I started to follow his ideas through lectures and talks. He used to ask scientists "What are the most important problems in your field?" so he could ask them "Why aren't you working on them?" and also because getting asked this question is useful for focusing their efforts on problems with the potential for significant impact.

With that in mind, I thought deeply about the problems I could solve in my field and how I could leverage my knowledge and technology to make a real difference.

Every possible idea targeted Machine Learning, not because it was, and still is a huge hype, but for its potential to have a real impact on other fields. There are interesting problems in medicine, healthcare, biology, and life sciences that I could leverage AI to have a significant impact.

In 2024 I shifted gears and began my ML learning journey. My idea was to lay down the foundational knowledge, from theory to practice, and learn the field properly. Learning how to use a framework like Pytorch or TensorFlow was not enough for me. I wanted to understand how it works under the hood. From mathematics to code.

The past year, my learning experience looked like this:

  • Mathematics for ML: linear algebra, calculus, probability and statistics
  • ML theory: different ML algorithms, performance and optimization
  • ML practice: building ML pipelines for different problems
  • Deep Learning theory: mathematics and DL concepts
  • Deep Learning practice: building neural nets from scratch

All these are running concurrently and it's working well for me. Because they are foundational knowledge, I feel like I'm building ideas on top of ideas, and they are compounding. For sure, I still have many knowledge gaps and many things to learn but it feels great to make progress on a very hard field. It's intellectually rewarding.

Mathematics

I'm currently reading the books Mathematics for Machine Learning and Why Machines Learn. They are great in content and a bit different. Maths for ML looks like a textbook, rigid, and focuses more on mathematics. Why Machines Learn is also all about mathematics but with a great focus on ML and basic concepts of deep learning.

But my first step was the course on math for ML with Python, covering basic topics of algebra, linear algebra, statistics, and calculus. I liked it because it was very interactive and the usage of Python made it very engaging, mostly because I'm a software engineer myself so it feels like my comfort zone.

In December I decided to make a bigger investment (in time and money) and signed up for Math Academy. I'm putting a whole lot of effort into mathematics now. Starting from basic concepts and climbing to advanced concepts.

The first 1-2 hours of my day always start with math on Math Academy, currently working on the mathematical foundations II. My goal is to finish mathematical foundations I, II, and III soon enough and start the Math for ML course, and be ready to take the Machine Learning course they are cooking.

Studying on Math Academy has been a lot of fun and I'm making lots of progress in my math knowledge, bridging the gap between basic math to advanced concepts in ML. The system just works.

Each topic you learn contains three concepts:

  • The theory
  • Concrete examples
  • Exercises

Each topic is a small bit of the whole portion of the course and you can pretty much do it in like 4-12 minutes each. They are small and fast to do. Basically, you keep motivated and make progress quickly.

You not only learn the concepts but the platform uses a spaced repetition system so from time to time it will suggest topics you did some time ago and you can test how much you can recall the concepts and solve the problems.

They also have quizzes that help a lot and force you to actively recall in a time-pressure manner. If you don't do well on some of the problems, it will suggest you retake the topics you failed, and you can also retake the quiz to make sure you really learned.

They also have links to pre-requisite topics for a given topic so you can review it and strengthen that knowledge before taking the targeted topic.

With more and more practice, you start to pattern-match math concepts and rules and build intuition behind each idea. I also use LLMs to help me build that intuition faster.

Math Academy is helping me work on my knowledge gaps while also building the foundation to learn the math behind machine learning algorithms and neural nets. But it's still not enough, I feel I need to speed up the process to reach that level of expertise. Reading the books in parallel is helping me test my math knowledge and check the gaps I need to fill.

What the future holds in terms of mathematics is: more time investment on Math Academy and testing my knowledge with books like Math for ML and Why Machines Learn and papers. I also thought about doing the Math for ML specialization on Coursera but I'm not sure if I over-optimizing here.

Machine Learning

I didn't want to just use some ML framework. I wanted to deeply understand ML algorithms, how they work, the theory behind them, and all the fun stuff. I started with Introduction to Statistical Learning with Python, both the book and the course. The book is just great with a bunch of detail about each algorithm. I built a good foundation reading that.

I learned scikit-learn from an ML course and I used that tool to practice some interesting challenges on Kaggle. The first one was a regression problem to predict heart failure from clinical features. It was the first time I was using ML models to make predictions. I learned Naive Bayes, SVM, Logistic Regression, KNN, and tree-based models like Decision Tree, Random Forest, and XGBoost.

The second was a classification problem. The idea of the challenge was to predict whether the breast cancer tumors were malignant or benign. I've even written a long-format post about the whole process and the learnings along the way: Training ML Models for Cancer Tumor Classification.

After learning about all this, I decided to take the leap and start learning about deep learning and neural networks, a topic I will cover in the next section.

Currently, I'm reading Designing Machine Learning Systems by Chip Huyen. It's not my top 1 priority but I try to read a bit whenever I have time. I decided to take this book because I want to close the gap between cleaning the dataset, implementing and training the ML model, checking and tuning the performance, and working on the deployment.

I don't consider a knowledge gap, but I know I need to keep honing my skills on a bunch of this. Here is the list of things for the future:

  • The math for ML
  • Improve my skills on Pandas and NumPy
  • Work on more ML projects
  • Learn more about performance optimizations
  • Build intuition on which algorithms to choose and why

Deep Learning

Deep Neural Networks is currently the topic I'm focusing on. I started with the Book Understanding Deep Learning, which is just amazing. I bought the book to help fund this awesome project. The book is so well-written, and if you have an interest in deep learning, you should give it a chance.

I took a break from the book to put the concepts into practice. I learned about the mathematics behind neural nets and implemented it from scratch, only using Python and Math.

I also wrote about this whole experience (still finishing this series):

I'm currently working on the Deep Learning specialization by Andrew Ng, specifically the performance optimization one.

After finishing that, I plan to continue working on mathematics for deep learning (books and Math Academy) and work on advanced deep neural networks (CNN, RNN, and other architectures).

To finish all this, I plan to study a specific domain so I can work on more interesting problems. I'm targeting life science and healthcare challenges but I need to have a better foundational knowledge about these fields and understand the problems to experiment with what I've been learning.

Resources

Hey! You may like this newsletter if you're enjoying this blog. ❤
✖

Twitter · Github