A line plot in 3D

An Intuitive Guide to Linear Regression

We have described before how supervised learning can help us predict a continuous-valued output or organize the input into discrete categories, commonly known as regression and classification problems, respectively. In this article, we describe linear regression and leave the classification algorithms for a future post. What is Linear Regression? Suppose that you are a young investor living in a region with cold climate. One day an idea flashes in your mind that perhaps the shares in the regional stock market climb linearly with the temperature: the better the weather, the higher the prices. You already know what the temperature is

Continue reading
Regression and classification in supervised learning

What is Supervised Learning?

In a previous article, we had a little introduction to the big picture of machine learning. More than what is it is, we focused on what it is not. Today we explore the category of supervised learning that opens the door to our understanding of advanced machine learning techniques. To see how supervised learning is proliferating in all walks of life, consider a few examples below. Insurance companies can predict the costs of storms damages for the future years due to climate change and adjust their premiums accordingly. During an interview, companies can classify the candidates with high and low

Continue reading
Farm and library

A Beginner’s Guide to Bayesian Methodology

Thomas Bayes was an English statistician and Presbyterian minister who came up with this theorem in 18th century during his investigation on how to update the understanding of a phenomenon as more evidence becomes available. At that time, he did not deem it worthy of publication and never submitted it to any journal. It was discovered in his notes after his death and published by his friend Richard Price. In the past, Bayesian theorem was associated with highly complicated mathematics (and rightly so), and hence it was generally a topic of interest for mathematicians, statisticians and similar professionals. However, as

Continue reading
Buridan's donkey at an equal distance from hay and water

k-Nearest Neighbors (k-NN) – A Man is Known by the Company He Keeps

Aesop was a Greek storyteller who created lots of short stories more than 2500 years ago (such as the hare and the tortoise), now collectively known as Aesop’s Fables. "A man is known by the company he keeps" is a quote also attributed to him that has survived through the ages and has proven to be a timeless truth. The k-Nearest Neighbors (k-NN) algorithm is essentially based on the same idea. It suggests that when a machine needs to classify a new data point in an application, examining its neighboring data points is an effective strategy. In the spirit of

Continue reading
Decision boundary for multivariate logistic regression

Logistic Regression in Machine Learning

A hundred thousand years ago, our ancestors used to roam in the savannas and jungles. It was absolutely necessary for them to judge (or classify) everything they encounter: a movement in the bushes could be due to a harmless rabbit or a dangerous tiger, a fruit on a plant could be nutritious or poisonous, and so on. Then came the wizards who invented language and then people developed agriculture, writing and industry. The advancement in civilization and the quest for scientific knowledge revealed the benefits of a wide spectrum and viewing shades of gray instead of of simple black and

Continue reading