-
PyTorch Example of LSTMData/Machine learning 2021. 4. 21. 15:19
Architecture [3] The main components are: 1) hidden and cell states, 2) input gate, forget gate, output gate. $$ i_1 = \sigma ( W_{i_1} \cdot (H_{t-1}, x_t) + b_{i_1} ) $$ $$ i_ = tanh ( W_{i_2} \cdot (H_{t-1}, x_t) + b_{i_2} ) $$ $$ i_{input} = i_1 * i_2 $$ $$ f = \sigma ( W_{forget} \cdot (H_{t-1}, x_t) + b_{forget} ) $$ $$ C_t = C_{t-1} * f + i_{input} $$ $$ O_1 = \sigma ( W_{output_1} \cdot ..
-
Log-bilinear Language ModelData/Machine learning 2021. 4. 20. 17:09
It computes the probability of a next word $w_i$ given the previous words (context) as follows: $$ P(w_i = w | w_{i-1}, ..., w_1) = \frac{ \exp\{ \phi(w)^T c \} }{ \sum_{w^\prime \in V} \exp\{ \phi(w^\prime)^T c \} } $$ Here $\phi (w)$ is a word-vector and $c$ is the context for $w_i$ computed as $$ c = \sum_{n=1}^{i-1} \alpha_n \phi(w_n) $$ Thus, the log-bilinear language model computes a conte..
-
[Idun/HPC] How to open Jupyter Notebook on HPC on your local computerMiscellaneous 2021. 4. 19. 11:13
Run Jupyter Notebook on a terminal in your server terminal by jupyter notebook --no-browser. Run ssh -L :localhost:8888 @ on your local terminal. For example, ssh -L 8000:localhost:8888 daesool@idun-login1.hpc.ntnu.no. Copy http://localhost:8888/?token=... from the server terminal, and change 8888 to 8000, and enter that on your local browser. Reference [1] Lee, D., 2020, "[AWS] how to set up py..