https://github.com/NVIDIA/apex/issues/429
If you’ve already installed apex, remove it:
pip uninstall apex
rm -rf apex
Reinstall from @ptrblck 's fork of apex apex_no_distributed branch
git clone https://github.com/ptrblck/apex.git
cd apex
git checkout apex_no_distributed
pip install -v --no-cache-dir ./