-
Permutation Hypothesis TestingMathematics 2020. 12. 29. 14:05
Tutorial about its theory: Computational implementation of the theory: Summary Specify $H_0$ (null hypothesis) and $H_1$. For example, $H_0$ : weight gain is irrelevant to a diet. Choose a test statistic: (absolute) difference in mean or median. Compute a distribution of the test statistic using the permutations of the observed data. Compute $p-value$ which is the probability of getting the perm..
-
Bootstrap / BootstrappingMathematics 2020. 12. 28. 14:00
Tutorial video Summary It is one easy and effective way to estimate the sampling distribution of a statistic by drawing additional samples with replacement from the sample itself. The procedure of the bootstrapping is as follows: Draw a sample value, record, with replacement Repeat $n$ times Record the mean of the $n$ resampled values Repeat the steps 1-3 $R$ times Use the results to a) calculat..
-
Variance, CovarianceMathematics 2020. 11. 3. 15:50
Variance $= \sqrt{\mathrm{std}}$ $ \sigma_{x}^{2} = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})^2 $ Covariance $ \sigma(x, y) = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y}) $ Covariance Matrix Example of a 2d covariance matrix: $ C = \begin{bmatrix} \sigma(x,x) & \sigma(x,y) \\ \sigma(y,x) & \sigma(y,y) \\ \end{bmatrix} $ Diagonal Covariance Matrix Example of a 2d diagonal covar..
-
Intuition behind the dot productMathematics 2020. 11. 3. 14:59
The dot product tells you what amount of one vector goes in the direction of another. This can also be interpreted as a similarity between the two vectors. The dot product of $\vec{a} = $ and $\vec{b} = $ can be represented as follows: $ \vec{a} \cdot \vec{b} = |\vec{a}| |\vec{b}| \mathrm{cos}(\theta) $ $ \quad\quad = a_1 b_1 + a_2 b_2 $ $ \quad\quad = \vec{a} \vec{b}^T $
-
Entropy vs Cross EntropyMathematics 2020. 11. 3. 13:29
Entropy: measures impurity of a dataset Cross Entropy: measures dissimilarity between an actual probabilistic distribution and predicted probabilistic distribution Refer to blog.naver.com/danelee2601/221937318234 Entropy, Cross Entropy Entropy: measures impurity of a datasetCross Entropy: measures dissimilarity between actual pro... blog.naver.com
-
Kullback-Leibler (KL) divergenceMathematics 2020. 11. 3. 13:01
The equation of the KL divergence is expressed as follows: $KL(p||q) = H(p,q) - H(p)$ $H(p,q) = H(p) + \sum{p_i log{\frac{p_i}{q_i}}}$ $KL(p||q) = H(p) + \sum{p_i log{\frac{p_i}{q_i}}} - H(p) = \sum{p_i log{\frac{p_i}{q_i}}} = \mathbb{E}_{p_i}{log\frac{p_i}{q_i}} $ where $H(p,q)$ denotes cross entropy and $H(p)$ denotes entropy. 마지막 KL(.)을 보면 엔트로피가 전부 상쇄되어 사라지고, summation항만 남게되는데, 이 summation항이 ..