Daesoo Lee's Blog

There are two kinds of visualization of CNN: 1) visualization of intermediate activation layers, 2) visualization of a representative image or pattern that a certain kernel is highly activated by. 1. Visualization of Intermediate Activation Layers You visualize output $a$ from a certain activation layer, and $a \in \mathbb{R}^{B \times C_{in} \times H \times W}$ where $B$ refers to the batch siz..

J. Zbontar, 2021, "Barlow Twins: Self-Supervised Learning via Redundancy Reduction" This paper proposes an objective function that naturally avoids such collapse by measuring the cross-correlation matrix between the outputs of two identical networks fed with distorted versions of a sample, and making it as close to the identity matrix as possible. This causes the representation vectors of distor..

.eval() is known to be used during inference of models that usually contain BN and/or Dropout. When .eval() is used, the model with BN uses runinng_mean and running_var instead of mean and var obtained from each mini-batch. Fine-tuning When fine-tuning, it is important to use running_mean and running_var of the trained model and they should be fixed during fine-tuning. This is because usually th..

.gitignore .gitignorefile is a plain text file where each line contains a pattern for files/directories to ignore. You may ignore them for the following reasons: 1) security, 2) size, 3) non-related to a project. .gitignore should be located in the root directory. Syntax [1] www.pluralsight.com/guides/how-to-use-gitignore-file [2] programming119.tistory.com/105 Examples 1-InitialExperiment/devel..