Data Science PT 2: Probability Review

Probability spaces consists of the following 3 components: 

Ω sample space, all possible outcomes of the random experiment. F sigma field comprised of events, or subsets of the sample space. Probability P assigned to events. Here's an example. We can represent F as the probability of Heads or Tails. F={{},{H},{T},{H,T}}. The Set of coins is like 

The probability function P satisfies the following axioms: 

1. For every event A∈F, Pr(A)≥0. 

2. Pr(Ω)=1

3, For disjoint A,B∈F (A∩B={}): Pr(A∪B)=Pr(A)+Pr(B)

We can define cardinality as the number of elements in the set or other grouping. For example, the cardinality of a dice roll is 6. Joint probability is the probability that multiple events occur jointly, for example Pr(A, B). The conditional probability is given that an event A happened what's the probability of B. 

Ex: Pr(B|A) = Pr(A,B)/Pr(A) = 0.2/0.5 = 0.4

Now 2 events, we can only describe 2 events as independent if and only if Pr(A|B) = Pr(A) and Pr(B|A) = Pr(B). That means these events don't depend on each other. 

Now we will describe Baye's rule as below. 

The random variable RV is the function of one experiment. For example, {H}, or heads, is mapped to 1, while {T}, or tails, is mapped to -1. An example of this random probability is spinning a wheel and observing the angle it lands on. Ω is from 0 to 2π. That being said, we can assign a random variable as follows: 

X=X(ω), 𝜔 ∈ Ω; X={1 if ω> π, else 0}.

The cumulative distribution function is a function that maps the random variable to its probability space. 

FX(c)=Pr({ω∈Ω, X(ω)≤c})

FX(c)=0 for c<-1, FX(c)=1 for c≥1, FX(c)=0.5 for c∈[-1,1)

The probability density function is the derivative of the cumulative density function. 

There are 2 commonly distributed Random Variables, the Gaussian Random variable (or normal function) and the uniformly distributed random variable, where we take values equally likely in interval [a,b]. This means that for a<c<d<b, Pr(c<X<d) = (d-c)/(b-a).  For a<c<b, FX(c)=(c-a)/(b-a). Taking derivative, fx(c) = 1/(b - a). 

For Normal distribution we can say the mean is  µ and the variance is σ ^ 2. We will calculate the probability as follows: 

Conditional distributions involve the part of a probability space conditioned on some event. We want to characterize random variable conditioned on other event/random variable. Conditional distributions involve the part of a probability space conditioned on some event. For example, the joint cumulative distribution function sees 2 factors. For example, F(x1, x2) = Pr(X1<=x1, X2 <= x2). The joint pdf is a 2 dimensional derivative of the join cdf. 

In the discrete random variable, we weight over all of the probabilities.

The variance is the mean squared error around the expectation, and we can count this to be 

E[(X-E[X])^2]. The standard deviation is the square root of the variance; 

Here, the result is 1/3 - 1/3 + 1/4 = 1/12. Mean square error is usually less the lesser degree polynomial we use to evaluate the function. 

For a review of the norms, here is lp norm, and norm to the power of p, respectively:

And here is the l1 and l2 norm, as well as the l∞ norm. each respective norm is more and more sensitive to outliers. 

The last thing that we will discuss here is the impulse function but this will be in very generalized detail. Here is a function describing x(u(x) - u(x - 1)) + a∂(x) + 2a∂(x + 1). Here is the function and following rules that relates to the impulse function.