- Fast Monte-Carlo Approximation of the Attention Mechanism (고정길 교수님 연구실)
- 게시글 내용
Fast Monte-Carlo Approximation of the Attention Mechanism
AAAI Conference on Artificial Intelligence (AAAI) 2022 (한국연구재단 인정 CS분야 최우수 국제학술대회 - IF 4)
Hyunjun Kim, JeongGil Ko
We introduce Monte-Carlo Attention (MCA), a randomized approximation method for reducing the computational cost of self-attention mechanisms in Transformer architectures. MCA exploits the fact that the importance of each token in an input sequence vary with respect to their attention scores; thus, some degree of error can be tolerable when encoding tokens with low attention. Using approximate matrix multiplication, MCA applies different error bounds to encode input tokens such that those with low attention scores are computed with relaxed precision, whereas errors of salient elements are minimized. MCA can operate in parallel with other attention optimization schemes and does not require model modification. We study the theoretical error bounds and demonstrate that MCA reduces attention complexity (in FLOPS) for various Transformer models by up to 11x in GLUE benchmarks without compromising model accuracy.