3.1In a Nutshell

Learning is all about generalizing regularities in the training data to new, yet unobserved data. It is not about remembering the training data. Good generalization means that you need to balanceprior knowledge with information from data. Depending on the dataset size, you can entertain more or less complex models. The correct size of model can be determined by playing a compression game. Learning = generalization = abstraction = compression.

16CHAPTER3. LEARNING

Chapter 4