Generators in Python!

Python generators code tutorial:

import string
import inspect

# Informal tutorial on the generators. We will cover very briefly:
## 1) What are the generators (PEP 255).
## 2) What is yield (PEP268).

# Generator in Python is a fancy object that maintain its state between calls. 
# In simple words generator knows what to return you next on a next function call. 
# And it doesn't need to store it in memory. 
# Think of it as a computation on-demand. Lazy, yeah. 
# Generator uses iterator protocol. 
# Thus it is common to say that every generator is an iterator. 
# It just means we can get a next value from the generator via next() call.
# You get a generator by asking to return one from a function. 

# Theoretically, you can compute something blah blah to 
# infinity using generators, and have very little memory footprint. 

# The 'yield' keywoard in a function will cause the function 
# instead of a value return a generator-iterator object. 

# Lets demonstrate!

# lets define a function that will return us a generator-iterator object 
# when we call it 
def english_alphabet_generator_function():
    for char in string.ascii_lowercase:
        yield char

print english_alphabet_generator_function 
# <function alphabet_generator at 0x1004c7320>, it is still a function

# Lets call english_alphabet_generator to get the actual generator object
alphabet_generator = english_alphabet_generator_function() 
# calling the generator function will return a generator-iterator object. 

print type(alphabet_generator) 
# <type 'generator'>, not a function anymore, we got generator object

# Now lets examine function locals - varibales defined in the function namespace 
print alphabet_generator.gi_frame.f_locals 
# {}, nothing yet 

# Now we can use iterator protocol and get print a few characters by calling generator's next() method. 
print # prints 'a'
print # prints 'b'
print # prints 'c'

print alphabet_generator.gi_frame.f_locals 
# prints {'char': 'c'}

# lets exauhst our generator and see what will happen
# it will print characters from d to y and throw 'StopIteration' exception
empty = False
while not empty:
    except(StopIteration) as exp:
        empty = True
# Another not very useful example:
def _my_gen():
    for x in xrange(200, 400):
        yield 1

gen = _my_gen()
print(sum(gen)) # will print 200! :-) 

# Please, refer to 
# for a in-deep explanation with 
# very interesting examples. Thank you!

UUID3 cross compatible in Java and Python

Sometimes, in the world of microservices you want to generate consistent hashes across several languages. I’ve faced a problem of getting the same UUID in Java and Python, and wanted to share how it can be done.


# b068931c-c450-342b-a3f5-b3d276ea4297


    bytes = b''

uid.uuid3(NULL_NAMESPACE, "name")
# b068931c-c450-342b-a3f5-b3d276ea4297


PyMongo insert_many with overwrite

I’ve recently needed to insert lots of objects into the Mongo collection. The only problem was that some of the objects would have an ‘_id’ key pre-set and would conflict with existing objects in the database.

PyMongo inser_many operation doesn’t support it out of the box, so here is a work-around using bulk api:

    bulk = collection.initialize_unordered_bulk_op() # or ordered
    objects_to_insert = (prepare generator for objects that need to be saved)
    for one in objects_to_insert:
        bulk.find({"_id": one["_id"]}).upsert().replace_one(one)
except BulkWriteError as exc:
    # exc.details available for more information
    pass # do something here