Machine Learning: Class on 1/26/2021

 Predictions will be different based on assumptions. This is called inductive bias. It becomes much easier to set up weights for pixels in the case of pattern recognitions. If there are no assumptions, it's very difficult to learn thins, you have to always have an assumption. In general, all models are wrong but some are useful, and there is no single model that works optimally for all problems. This is the no free lunch theorem. The inductive bias are the assumptions about the data distribution, prediction, and model. 

We can categorize parametric vs non-parametric. Parametric model assume a probability of distribution for data, which can be described using a set of Gaussian distribution. Nonparametric models are assumption free and their parameter number grows with the number of data points. 

For data points, we have elevations, year built, price per square feet, and the number of bedrooms if we want to classify whether a house is in San Francisco and New York.

We look at many dimensions of features, and if we look at more features, and look at the cost per square foot or square meter, and can subsequently make better decisions based on this. We can look for things that are for sure New York, and San Francisco. We can add more dimensions and try to see which fits the best, and look at each 2 dimensions separately to distinguish between the houses in New York and the houses in San Francisco. 

The correct classifications are true positives then wrongly positive classified are false positives. The more levels we add to the tree, the more accurate the results are. 

We can memorize everything with decision tree and this can let us to get to 100% accuracy for decision trees.  We make it a bit more difficult to handle continuous values of decision trees. For continuous data points, I have to assume different thresholds to make the data binary.

Here is the Top-Down induction of decision Trees:

Main Loop:

A <= the "best" decision for each node

Assign A as a decision attribute for node

For each value of A, create a descendant of the node

Sort traning examples

If perfectly classified, stop else iterate through every leaf node. 

A random variable x is a variable whose values depend on the outcoes of random  phenomena. For example, we can flip a coin and the outcome is either head or tail and we can define a random variable that takes 1 if the coin is head and 0 if coin is tail. So P(X = 1) = #head/n and P(X = 0) = #tail / n, if there are n outcomes. The domain of the states is D(X) and the sum of all the possible states in the domain is 1.