Daesoo Lee's Blog

Recent Posts

• Noise Contrastive Estimation and Negative Sampling

  Data/Machine learning 2021. 3. 23. 14:31

  Reference: [C. Dyer, 2014, "Notes on Noise Contrastive Estimation and Negative Sampling"]; Check out my Mendeley. Estimating the parameters of probabilistic models of language such as probabilistic neural models is computationally difficult since it involves evaluating partition functions by summing over an entire vocabulary. Two closely related strategies - noise contrastive estimation and nega..

  Read More
• Odds Ratios and Log(Odds Ratios)

  Mathematics 2021. 3. 23. 09:26

  Odds The chance of my team winning is 0.33, then the chance of my team losing is 0.67. Its odds are computed as $$odds = \frac{0.33}{0.67} = 0.493$$ Odds Ratio "Odds ratio" is different from the odds. It is a ratio of odds. Log(Odds Ratio) If the denominator is larger than the numerator the odds ratio will go from 0 to 1. If the numerator is larger than the denominator, then the odds ratio wil..

  Read More
• Variational Auto Encoder (VAE)

  Data/Machine learning 2021. 3. 19. 09:55

  Reference: www.jeremyjordan.me/variational-autoencoders/ Variational autoencoders. In my introductory post on autoencoders, I discussed various models (undercomplete, sparse, denoising, contractive) which take data as input and discover some latent state representation of that data. More specifically, our input data is converted into an www.jeremyjordan.me Key Concepts We define $x$, $z$ as inpu..

  Read More
• Cosine-similarity Classifier; PyTorch Implementation

  Data/Machine learning 2021. 3. 17. 11:10

  Cosine-similarity Classifier introduced in [S. Gidaris et al., 2018] is implemented here. The cosine-similarity classifier is compared to the linear-softmax classifier. The codes can be found in my Github. [W. Chen et al., 2020] verifies the performance improvement by the cosine-similarity classifier in the few-shot learning regime. Result; In the one-shot learning regime The models are trained ..

  Read More