RBM
상수 초기화 (Zeros. Ones, Constant)
class Constant: Initializer that generates tensors with constant values
class Ones: Initializer that generates tensors initialized to 1.
class Zeros: Initializer that generates tensors initialized to 0.
class VarianceScaling: Initializer capable of adapting its scale to the shape of weights tensors.
선형 초기화 (Orthogonal, Identity)
class Orthogonal: Initializer that generates an orthogonal matrix.
class Identity: Initializer that generates the identity matrix.
확률 분포 초기화 (RandomUniform, RandomNormal, TruncatedNormal)
class RandomNormal: Initializer that generates tensors with a normal distribution.
class RandomUniform: Initializer that generates tensors with a uniform distribution.
class TruncatedNormal: Initializer that generates a truncated normal distribution.
자비에르 초기화 (GlorotNormal, GlorotUniform)
Sigmoid, Tanh : Xavier initialization
class GlorotNormal: The Glorot normal initializer, also called Xavier normal initializer.
class GlorotUniform: The Glorot uniform initializer, also called Xavier uniform initializer.
He (Kaming He) 초기화 (HeNormal, HeUniform)
ReLU : He initialization
class HeNormal: He normal initializer.
class HeUniform: He uniform variance scaling initializer.
Lecun 초기화 (LecunNormal, LecunUniform)
class LecunNormal: Lecun normal initializer.
class LecunUniform: Lecun uniform initializer.
# LeCun은 LeNet의 창시자이며, CNN을 고안했다. LeCun은 역전파 논문에서 정규분포를 따르는 방법과 균등분포를 따르는 두가지 방법에 대해서 소개하였다.
'Deep Learning' 카테고리의 다른 글
[Transformer] Self-Attension 셀프 어텐션 (0) (0) | 2021.02.19 |
---|---|
VAE(Variational Autoencoder) (3) MNIST (0) | 2021.02.18 |
VAE(Variational Autoencoder) (2) (0) | 2021.02.18 |
VAE(Variational Autoencoder) (1) (0) | 2021.02.18 |
nvidia-smi 옵션 (0) | 2021.02.16 |