Tensor ๊ฐ€์ง€๊ณ  ๋†€๊ธฐ [Einsum + einops]
ยท
๐Ÿ‘พ Deep Learning
Einstein summation numpy, torch, tensorflow ๋“ฑ ๋‹ค์–‘ํ•œ Tensor Type์˜ ํ˜•ํƒœ๋ฅผ ๋ณ€ํ˜•ํ•  ๋•Œ ์ฃผ๋กœ ์‚ฌ์šฉ๋œ๋‹ค. ํ˜„์žฌ๋Š” einops์™€ ์ฝ”๋“œ๋ฅผ ํ•จ๊ป˜ ์ž‘์„ฑํ•˜๋ฉด ์ •๋ง ๋ชจ๋“  ์ด๊ฐ€ ์•Œ์•„๋ณผ ์ˆ˜ ์žˆ์„ ์„ค๋ช…์„œ ๊ฐ™์€ ๋ชจ๋ธ์„ ์ž‘์„ฑํ•  ์ˆ˜ ์žˆ๋‹ค. Example) Attention์—์„œ energy๋ฅผ ๊ตฌํ• ๋•Œ Q ์™€ K^T์˜ ํ–‰๋ ฌ๊ณฑ์‹œ Key ๋ถ€๋ถ„์˜ Transpose๋ฅผ ์‰ฝ๊ฒŒ ํ‘œํ˜„ ๊ฐ€๋Šฅ quries = torch.tensor(np.array([i for i in range(0,240)]).reshape(1,2,6,20),dtype=torch.float64) keys = torch.tensor(np.array([i*2 for i in range(0,240)]).reshape(1,2,6,20),dtype..
๋‹คํ–ˆ๋‹ค
'pytorch' ํƒœ๊ทธ์˜ ๊ธ€ ๋ชฉ๋ก