Why Do AI Engineers Need Calculus

Among many fields and branches of mathematics, Calculus plays a significantly important role in Artificial Intelligence. Today, we’ll cover the basics of what Calculus really is and understand why it is integral for AI Engineers.

Calculus

Calculus is the study of how things change. It constitutes of two branches namely, differential calculus and integral calculus. Differential calculus explains the rate of change, slopes of curves whereas integral calculus talks about the continuous summation of parts i.e. Areas under the curve. Just as summation describes the addition of discrete parts, integral explains the continuous summation of data.

What can we do with the things we learned in the previous article Algebra for Machine Learning?

Following the Algebra from the last Article Algebra for Machine Learning, we learned about the functions and equations. It has built a foundation upon which calculus can further take over and solve more complex problems that constitute real-world scenarios from our microwaves, cell phones, TV, and automobiles to medicine, economy, and national defense.

Topic Covered in this article,

Equation of Functions
Special functions

ln(x)
x^2
cos(x)
1/(x^2-1)
tan(x)

Derivates
Minima and Maxima
Concave or convex functions in a defined interval (saddle points)
Integrals

Equation of Functions,

For an equation,

y= ax + b

Here, x and y are directly proportional to each other.

Ie. y/x = Constant

Fig: In the above graph, the slope of the line y = 2x +3 is constant.

Linear Function

Linear functions are those functions whose graph represents a straight line. A linear function has the following form. y = f(x) = a + bx. The linear function contains one independent variable and one dependent variable. For the above equation, the independent variable is x and the dependent variable is y.

Linear Regression

Linear regression models the relationship between two different variables by fitting a linear equation to observed data. In linear regression, one of the variables is taken as an explanatory variable, and then another is considered as a dependent variable.

A linear regression line has an equation of the form Y = a + bX, where X is the explanatory variable and Y is the dependent variable. The slope of the line is b, and a is the intercept (the value of y when x = 0).

Fig: In the graph above, x and y are inversely proportional such that x*y = constant where y = 2/x

Special functions

Special functions are particular mathematical functions that have more or less established names and notations due to their importance in mathematical analysis, functional analysis, geometry, physics, or other applications. Some of them are listed below,

f(x) = ln(x)

f(x) = x2

f(x) = cos (x)

f(x) = 1 / (x2 – 1)

f(x) = tan (x)

Why are we going across these functions?

We are going to come across functions when we need to tune in to minimize our algorithm. We’ll come across different algorithms while exercising Machine Learning Models and it is vital to know how do these functions react and transform it.

Plotting graph using the function in GeoGebra

GeoGebra is an online web app that provides free tools for learning interactive geometry, algebra, statistics, and calculus application and teaching mathematics and science from primary school to university level.

Fig: y = ln(x)

Fig: y = x2

Fig: y = -x2

Fig: y = x3

Fig: y = cos(x)

Fig: y = tan(x)

Derivatives

Derivatives describe and explain the change i.e.. the slope of a function. It measures the sensitivity of the change in a particular function i.e.. The output value with respect to the change in the input value.

For an analogy of where Derivatives are directly used in ML, while developing Machine Learning Models for Autonomous Driving Vehicles, without Derivative, Instantaneous Velocity and Acceleration would be impossible to calculate for further processing.

Fig: Slope is constant across the line

The difference comes with the different shapes of the function. Let us suppose a curve where there are different slopes at different points. The points where derivatives are 0 like at the top are basically flat points. These are what we call, minima and maxima points. These minima points and maxima points can either be local or global.

Minima Points
A low point in the curve is called a minimum (plural minima).
Maxima Points
A high point in the curve is called a maximum (plural maxima).
Local Minima
The point at which the value of a function is less than or equal to the value at any nearby point is called local minimum.
Local Maxima
A local maximum also called a relative maximum, is a maximum within some neighborhood that need not be (but maybe) a global maximum.
Global Maxima
A global maximum, also known as an absolute maximum, the largest overall value of a set, function, etc., over its entire range. It is impossible to construct an algorithm that will find a global maximum for an arbitrary function.
Global Minima
A global minimum, also known as an absolute minimum, is the smallest overall value of a set, function, etc., over its entire range. It is impossible to construct an algorithm that will find a global minimum for an arbitrary function.

If we make a table of +Ve and –Ve signs for each time the direction of flow changes in the curve, we can understand how the points are changing in the curve above.

If you want to learn more about Calculus, watch this video by AI 42,

Calculating Derivatives

For y = x2, we obtain the following curve.

When, x =0 the derivative becomes 0 ie at the minima at x,y= 0

For +ve values of x, the slope is +ve, and for –ve values of x, the slope is –ve.

Differentiating the equation, f(x) = x2 we get, f’(x) = 2x

Where do we use this sort of calculus in Data Science and Machine Learning?

There will be numerous cases while depicting real-world scenarios throughout machine learning models where we’ll be using similar calculus. One of the examples is as shown below to find loss function for multiple data points and hence calculate Linear Regression.

Loss Function: Loss functions measures the distance between the estimated value and its actual value. It sees how far are the points from the line. It maps decisions to all their respective costs. Loss function varies depending upon the task and goal that needs to be attained.

Tuning algorithm and finding a and b to make the loss function as small as possible using derivate to get as close to the bottom in the parabola.

Integrals

Integrals are concerned with the accumulation of quantities. In mathematics, integrals give numbers to function such that they can explain different quantities such as area, volume, displacement, and multiple other concepts by the continuous summation of the infinitesimal data. The process of finding integrals is known as integration. Along with differentiation, integration is a fundamental, essential operation of calculus, and serves as a tool to solve problems in mathematics.

Calculating Integrals

∫ 2x dx = 2 ∫ x dx = x (x1+1/1+1) + C = 2 * x2 / 2 + C = x2 + C

We have, Derivative of x2 = 2x and Integral of 2x = x2 + C

The calculating area between certain points

The area between certain points can be calculated by performing integral between those points. This would also be the area under the curve in multiple cases. For instance, if A denotes a big curve within a limit, The Area under the bigger curve between the limits would be performed using Integral thus calculating the area of A.

Derivative and Integral are opposite of each other. In calculus, an integral is in fact the space under a graph of an equation that we sometimes call the area under a curve.

An integral is the reverse of a derivative, and integral calculus is the opposite of differential calculus. A derivative is the steepness i.e. slope, as the rate of change, of a curve.

Conclusion

We learned about the different equations of functions, from basic linear functions to representing special functions in graphs. We understood derivatives and integral, minima and maxima, and the processes to solve them. Calculus is extremely important for it represents real-world dynamics which is monumental in Engineering. Since Artificial Intelligence Engineers solve some of the most complex problems in the world with their Engineering aptitude, the lessons of the Calculus we learned today will in fact help you to become a better AI Engineer as you pursue the path.