Notes: Few concepts of machine learning




January 20, 2017

Notes on a few ML concepts.

Deduction: Given the rule and the cause, deduce the effect.

Induction: Given a cause and an effect, induce a rule.

Abduction: Given a rule and an effect, abduce a cause.


What? – Parameters, structure, hidden concepts

What from? – Supervised, Unsupervised, Reinforcement

What for? – prediction, diagnostics, summarization

How? – passive, active, online, offline

Outputs? – Classification, Regression

Details? – Generative, Discriminative

Occom’s Razor – Everything else being equal, choose the less complex hypothesis.

The Ultimate goal of Machine Learning is to have data models that can learn and improve overtime.

Evaluation Metrics

Learn from data to make predictions.

Classification is about deciding which categories new instances belong to. Then when we see new objects we can use their features to guess which class they belong to.

In regression, we want to make a prediction on continuous data.

In classification, we want to see how often a model correctly or incorrectly identifies a new example, whereas, in regression we might be more interested to see how far off the model’s prediction is true from true value.

Classification ⇒ Accuracy, precision, recall and F-score.

Regression ⇒ mean absolute error and mean square error.

Short comings of accuracy:

Causes of Error:

Data Types:

Curse of Dimensionality

As the number of features or dimensions grows, the amount of data we need to generalize accurately grows exponentially.