Tensorflow Initializer 초기화 종류

RBM 

 

 

상수 초기화 (Zeros. Ones, Constant)
class Constant: Initializer that generates tensors with constant values
class Ones: Initializer that generates tensors initialized to 1.
class Zeros: Initializer that generates tensors initialized to 0.
class VarianceScaling: Initializer capable of adapting its scale to the shape of weights tensors.

선형 초기화 (Orthogonal, Identity)
class Orthogonal: Initializer that generates an orthogonal matrix.
class Identity: Initializer that generates the identity matrix.

확률 분포  초기화 (RandomUniform, RandomNormal, TruncatedNormal)

class RandomNormal: Initializer that generates tensors with a normal distribution.
class RandomUniform: Initializer that generates tensors with a uniform distribution.
class TruncatedNormal: Initializer that generates a truncated normal distribution.

자비에르 초기화 (GlorotNormal, GlorotUniform)
Sigmoid, Tanh : Xavier initialization

class GlorotNormal: The Glorot normal initializer, also called Xavier normal initializer.
class GlorotUniform: The Glorot uniform initializer, also called Xavier uniform initializer.

He (Kaming He) 초기화 (HeNormal, HeUniform) 
ReLU :  He initialization

class HeNormal: He normal initializer.
class HeUniform: He uniform variance scaling initializer.

Lecun 초기화 (LecunNormal, LecunUniform)

class LecunNormal: Lecun normal initializer.
class LecunUniform: Lecun uniform initializer.

# LeCun은 LeNet의 창시자이며, CNN을 고안했다. LeCun은 역전파 논문에서 정규분포를 따르는 방법과 균등분포를 따르는 두가지 방법에 대해서 소개하였다.