Few important definitions that are useful to understand Machine Learning principles. From “Pattern Recognition and Machine Learning” – C. M. Bishop about Machine Learning.
The result of running the Machine Learning algorithm can be expressed as a function $y(x)$ which takes a sample image $x$ as input and that generates an output vector $y$, encoded in the same way as the target vectors. The precise form of the function $y(x)$ is determined during a training phase, also known as a learning phase, on the basis of the training data. Once the model is trained it can then determine the identity of new sample images. The ability to categorize correctly new examples that differ from those used for training is known as generalization. In practical applications, the variability of the input vectors will be such that the training data can comprise only a tiny fraction of all possible input vectors, and so generalization is a central goal in pattern recognition.
Applications in which the training data comprises examples of the input vectors along with their corresponding target vectors are known as supervised learning problems. Cases such as the digit recognition example, in which the aim is to assign each input vector to one of a finite number of discrete categories, are called classification problems. If the desired output consists of one or more continuous variables, then the task is called regression.
- The book by C. M. Bishop: http://research.microsoft.com/en-us/um/people/cmbishop/prml/