Movies, and engineering.

Menu Close

Tag: python

Upgrading Python binding of Apache MXNet from 0.9 to 0.10 on Windows 10 64-bit

This post is a continuation and somewhat a revision of  “Installing MXNet on Windows 10”

MXNet is evolving and recently there was a release of version 0.10

I want to share the upgrade process that worked for me on my Windows 10 machine. You can use this tutorial as installation tutorial of MXNet 0.10.1.

First, we will need to create a folder somewhere called mxnet-0.10

Then, lets go to https://github.com/yajiedesign/mxnet/releases and download two packages:

  1. VC14 – prebuildbase_win10_x64_vc14.7z
  2. GPU ( or CPU) pre-built package – at the moment of writing it was 20170704_mxnet_x64_vc14_gpu.7z

Unpack VC14 first in the mxnet-0.10 folder and after that do the same with the GPU archive.

If you are using GPU version of MXNet, you will need to add cuda libraries to the folder called 3rd party.

You can register at NVIDIA web-site and get required cuDNN  library for free. The file should be placed into:

Drive:\mxnet-0.10.1\3rdparty\cudnn\bin\

Now in the root of the mxnet-0.10 folder look for a file called setupenv.cmd and run it as Administrator.

This execution of the .cmd file should adjust the PATH, and add MXNET_HOME environment variable.

MXNET_HOME should just point to the location of the mxnet-0.10 folder.

When all above is done, activate python 2.7 64-bit environment of your choice – conda, virtualenv or just system python.

Usually you do that by issuing command source activate [environment name] In my case, it is just source activate mxnet which I’ve created earlier using Anaconda Python 2.7 64bit installation.

Once the environment is active, go to the MXNET_HOME\python and run

python setup.py install

It should upgrade or install your MXNet package. Enjoy!

Python NLTK module and its download() function

I am trying to build a small machine learning service that would use Python NLTK library. As it is a pet project, and a very small one – I’ve decided to use Heroku for the hosting. And as I am using NLTK library I needed to download models and corpora by calling nltk.download() method to parse punctuation and have some other textual tricks.

Heroku doesn’t allow that method to execute, as it requires GUI interaction.

Initially I got a bit lost, as there is a way to bypass GUI when you provide specific list of NLTK corpora to download – but for certain missing models the error message provided a name of a model that was not compatible with nltl.download().

Luckily, I’ve found the page with a list of all available corpora and associated download IDs.

http://www.nltk.org/nltk_data/

Enjoy.

Generators in Python!

Python generators code tutorial:

UUID3 cross compatible in Java and Python

Sometimes, in the world of microservices you want to generate consistent hashes across several languages. I’ve faced a problem of getting the same UUID in Java and Python, and wanted to share how it can be done.

Java

Python

Enjoy!

PyMongo insert_many with overwrite

I’ve recently needed to insert lots of objects into the Mongo collection. The only problem was that some of the objects would have an ‘_id’ key pre-set and would conflict with existing objects in the database.

PyMongo inser_many operation doesn’t support it out of the box, so here is a work-around using bulk api:

© 2017 Movies, and engineering. . All rights reserved.

Theme by Anders Norén.