Dense and memory layers. Traditional language models use “dense layers” to encode vast amounts of information in their parameters. In dense layers, all parameters are used at their full ...