Types of Machine Learning

We now will turn our attention and discuss some learning problems that we will encounter in this book. The most well studied problem in ML is that ofsupervised learning. To explain this, let’s first look at an example. Bob want to learn how to distinguish between bobcats and mountain lions. He types these words into Google Image Search and closely studies all catlike images of bobcats on the one hand and mountain lions on the other. Some months later on a hiking trip in the San Bernardino mountains he sees a big cat....

The data that Bob collected was labelled because Google is supposed to only return pictures of bobcats when you search for the word ”bobcat” (and similarly for mountain lions). Let’s call the imagesX_1,..Xnand the labels_Y_1,...,Yn. Note that_Xi_are much higher dimensional objects because they represent all the information extracted from the image (approximately 1 million pixel color values), while_Yi_is simply−1or1depending on how we choose to label our classes. So, that would be a ratio of about 1 million to 1 in terms of information content! The classification problem can usually be posed as finding (a.k.a. learning) a function_f(x)that approximatesthe correct class labels for any inputx. For instance, we may decide that sign[f(x)]is the predictor for our class label. In the following we will be studying quite a few of these classification algorithms.

There is also a different family of learning problems known asunsupervised learning_problems. In this case there are no labels_Y_involved, just the features_X. Our task is not to classify, but to organize the data, or to discover the structure in the data. This may be very useful for visualization data, compressing data, or organizing data for easy accessibility. Extracting structure in data often leads to the discovery of concepts, topics, abstractions, factors, causes, and more such terms that all reallymean the same thing. These are the underlying semantic