The Softmax function is ubiquitous in machine learning, multiple previous works suggested faster alternatives for it. In this paper we propose a way to compute classical Softmax with fewer memory accesses and hypothesize that this reduction in memory accesses should improve Softmax performance on actual hardware. The benchmarks confirm this hypothesis: Softmax accelerates by up to 1.3x and Softmax+TopK combined and fused by up to 5x.
Online normalizer calculation for softmax
A method to compute the Softmax function with reduced memory accesses improves performance by up to 1.3x and the combined Softmax+TopK by up to 5x.
- Year
- 2018
- Venue
- arXiv 2018
- Authors
- 2
- Hosting
- Abstract onlyARXIV-DEFAULT
Cite
Notes
Only stored in your browser.
Attribution
- Abstract & full text
- arxiv.org/abs/1805.02867v2ARXIV-DEFAULT
- TL;DR
- Semantic Scholar