Data Science Prerequisites: calculus (4/5)

Studying the theoretical underpinnings of data science can be a daunting experience, especially for newcomers lacking the required background. To help alleviate this burden, this piece will provide an overview of Calculus, one of the pillars supporting many popular algorithms today, along with a few examples of application in machine learning and resources to get up to speed in the area.

If you are a beginner however and looking to hone your math skills for your future practical or research data science work, I would not recommend diving deep into the theory of Calculus (or any other math area) before you start experimenting in practice. Although this series of articles is called data science prerequisites, it does not necessarily imply that you need to cover everything mentioned herein before you start working on practical projects.
Despite its prevalence in machine learning education, the pure top-down approach is actually harmful for newbies, simply because most people don’t cope well with the frustration of learning disconnected theoretical concepts, which eventually drives them to give up long before reaching the fun parts of data science.
Furthermore, most research on teaching methods also supports the bottom-up approach to learning and education in general. So, when you get tired of the endless math exercises, don’t hesitate to switch to your text editor or programming environment to play around with some code and see how things work in practice. Not only will this reset your frustration meter, but it will also complement your knowledge and understanding.

How to define calculus?

As mentioned before, Calculus is another central topic in machine learning and data science. Calculus is a branch of mathematics that studies continuous change, in the same way that Geometry studies shapes and Algebra studies the generalization of arithmetic operations [Wikipedia].
There are two major sub-branches of calculus: Differential Calculus, which deals with instantaneous rates of changes (represented analytically by derivatives and geometrically by slopes of curves), and Integral Calculus, which is more concerned with accumulated quantities and areas under curves.
So why is Calculus actually required in Data Science?
Most Machine Learning algorithms are framed as an optimization problem where we seek to minimize some Cost Function or maximize some Reward Function. Differential Calculus plays a key role in optimization routines given that derivatives inform about the rates of change, and thus, help identify stationary points. Thus, concepts such as partial derivatives and gradients are of particular importance. As for Integral Calculus, it mostly comes up in probability-based methods dealing with continuous random variables.
The discrete sum you would use in the case of a discrete variable turns into an integral for continuous random variables.
Fortunately, the scope of Calculus needed to carry out most data science workloads is limited, but the few concepts one should know, need to be well understood.

Calculus Concepts

1. Differentiation: Limits, continuity, Derivatives, Partial Derivatives, Chain Rule, Taylor Series, Gradient, Higher-Order Derivatives, Hessian Matrix, Jacobian Matrix.
2. Integrals: Integrals as Riemann sums, Integration Techniques …

For further more

In order to improve your skills, or simply to start learning calculus, we propose you some resources, classified by type of source.

YouTube Playlists

Channel: 3Blue1Brown

Playlist: Essence of Calculus

Description: This channel is about mathematics explained in an easy and digestible way using intuitive visualizations. It is one of the best YouTube channels out there to introduce and explain difficult math concepts to initiated and non-initiated audiences alike. Just like the Essence of Linear Algebra playlist, this one contains 12 videos of ~15 mins each and will introduce calculus concepts through simple explanations and intuitive visualisations.
However, just like its Linear Algebra sibling, the purpose of this playlist is to teach you the intuition motivating mathematical concepts, not the hardcore formulas and equations supporting them. For that, you need other resources.

Books

Author: James Stewart

Title: Calculus Early Transcendentals

Description: This is the standard textbook for calculus courses in Universities. The last edition (8th) is comprised of 1400 pages, so needless to say, this is a very comprehensive book that will teach you more than you need to know about calculus. Obviously, you do not have to cover every chapter, you can just select the ones you require to complement your knowledge. It is detailed, very well written, illustrated with colorful figures, and clearly organized.

Author: Michael Spivak

Title: Calculus

Description: Another good book dedicated calculus. This one is much shorter than the first (around 600 pages) but still covers the required concepts of Calculus (both derivatives and integrals). You may find that it lacks in terms of multivariate calculus which is an important topic in Machine Learning since we usually deal with high dimensional data.

Author: Marc Peter

Title: Mathematics for Machine Learning

Description: If you recall, this same book was mentioned in previous articles. Naturally, it also has a chapter on vector calculus. Combined with some chapters from Calculus by Michael Spivak, this should provide you with a nice overview of the pre-required knowledge in this area. This book can be downloaded from its associated GitHub repository.

MOOC

Institution: Massachusetts Institute of Technology (MIT)

Name: Single Variable Calculus

Description: This course, MIT OCW 18.01, teaches single variable calculus, and covers differentiation and integration of functions of one variable. This is usually referred to in university jargon as Calculus 1 and 2 respectively. The course is designed specifically for self-learners and contains video lectures, lecture notes, assignments and exams including their solutions.

Institution: Massachusetts Institute of Technology (MIT)

Name: Multivariable Calculus

Description: As you might have guessed, this is the second part of the MIT calculus class (MIT OCW 18.02) and covers differentiation and integration of functions of more than one variable. This is usually referred to in university jargon as Calculus 3. This online course combined with the previous one, should be your main stop for learning everything about calculus that you might need to understand the math behind data science techniques and machine learning algorithms.

In a future post, we will propose you to address the fifth pre-requisite: programming.