I don’t have access to a supercomputer – is this

Question

I don’t have access to a supercomputer – is this

still useful for me?

Though this work has focused on extremely large models, we also find that models with as few as two experts improves performance while easily fitting within memory constraints of commonly available GPUs or TPUs (details in Appendix D). We therefore believe our techniques are useful in small-scale settings.

0

17.01.2021