Daesoo Lee's Blog

There are two kinds of visualization of CNN: 1) visualization of intermediate activation layers, 2) visualization of a representative image or pattern that a certain kernel is highly activated by. 1. Visualization of Intermediate Activation Layers You visualize output $a$ from a certain activation layer, and $a \in \mathbb{R}^{B \times C_{in} \times H \times W}$ where $B$ refers to the batch siz..

.eval() is known to be used during inference of models that usually contain BN and/or Dropout. When .eval() is used, the model with BN uses runinng_mean and running_var instead of mean and var obtained from each mini-batch. Fine-tuning When fine-tuning, it is important to use running_mean and running_var of the trained model and they should be fixed during fine-tuning. This is because usually th..

.gitignore .gitignorefile is a plain text file where each line contains a pattern for files/directories to ignore. You may ignore them for the following reasons: 1) security, 2) size, 3) non-related to a project. .gitignore should be located in the root directory. Syntax [1] www.pluralsight.com/guides/how-to-use-gitignore-file [2] programming119.tistory.com/105 Examples 1-InitialExperiment/devel..

[1] QuickStart: docs.wandb.ai/quickstart [2] wandb.init(...): docs.wandb.ai/library/init [3] wandb.log(...): docs.wandb.ai/library/log [4] PyTorch Integration: docs.wandb.ai/integrations/pytorch Quick Start Set your config with argparse. wandb.init(project='project_name_you_want' , config=args) wandb.watch(model) // automatically log gradients and model parameters Set up a training pipeline and ..

Reference: [C. Dyer, 2014, "Notes on Noise Contrastive Estimation and Negative Sampling"]; Check out my Mendeley. Estimating the parameters of probabilistic models of language such as probabilistic neural models is computationally difficult since it involves evaluating partition functions by summing over an entire vocabulary. Two closely related strategies - noise contrastive estimation and nega..

Reference: www.jeremyjordan.me/variational-autoencoders/ Variational autoencoders. In my introductory post on autoencoders, I discussed various models (undercomplete, sparse, denoising, contractive) which take data as input and discover some latent state representation of that data. More specifically, our input data is converted into an www.jeremyjordan.me Key Concepts We define $x$, $z$ as inpu..

Cosine-similarity Classifier introduced in [S. Gidaris et al., 2018] is implemented here. The cosine-similarity classifier is compared to the linear-softmax classifier. The codes can be found in my Github. [W. Chen et al., 2020] verifies the performance improvement by the cosine-similarity classifier in the few-shot learning regime. Result; In the one-shot learning regime The models are trained ..

Concept It was first proposed from a paper for WaveNet which was developed by Google to generate realistic-sounding speech from text. You can try text2speech of the wavenet here. A comparison between with the dilated causal convolution (DCC) and without it is shown in the following figure: It can be observed that the DCC covers a longer time series, which allows the model to capture the global e..