Attempted to run MXNet 1.1 GPU version on Windows 10. It works well, but the process of installation has a few details that is helpful to know.
1. You can create Anaconda environment based on Python 3.6 and use environment’s pip to install GPU version of Apache MXNet on Windows – pip install mxnet-cu90 at the moment. I believe cu91 is in the works and soon will be available.
2. You still need to download and install CUDA 9.0 as well as CuDNN 7.1 (FOR WINDOWS 10, NOT WINDOWS 7 – very easy to mistake them!) from the nVidia website. If CUDA itself is available for the download without an account – to get CuDNN library you need to have a free account on the nVidia page.
3. You still need to install latest graphics card drivers on the system, even after you installed all the CUDA stuff. For me MXNet was breaking on initialization of GPU context until I’ve rolled GPU drivers.

A few helpful links:
Cuda 9.0 download – https://developer.nvidia.com/cuda-90-download-archive
CuDNN download – https://developer.nvidia.com/cudnn ( I’ve tested 7.1 version and it works well)
nVidia GPU drivers – http://www.nvidia.com/Download/index.aspx