Transforming Code into Beautiful, Idiomatic Python

By: Next Day Video

6207   64   321857

Uploaded on 03/21/2013

Raymond Hettinger
Learn to take better advantage of Python's best features and improve existing code through a series of code transformations, "When you see this, do that instead."

Comments (10):

By clusmore    2017-09-20

Must watch:

I believe that the list comprehension will be faster than filter, but as always, any time you replace readable code with unreadable code for performance reasons, you damn well better time it.

Original Thread

By clusmore    2017-10-30

Check out Raymond Hettinger's talks on YouTube. Here's one to get you started:

Original Thread

By anonymous    2017-09-20

General python considerations :

First of all about time measurement you may use such a snippet:

from time import time, sleep

class Timer(object):
    def __init__(self):
        self.last = time()

    def __call__(self):
        old = self.last
        self.last = time()
        return self.last - old

    def elapsed(self):
        return time() - self.last

timer = Timer()

print timer.elapsed
print timer()
print timer()

Then you could benchmark running code many times, and check for the diff.

About this, i comment inline :

with open(path, 'r') as file_obj:
    line = True
    while line: #iterate on realdines instead.
            line = file_obj.readline()
            match = reg_ex.match(line)
            index +=
            #if match:
            #    index.extend( # or extend

        except AttributeError:

You previous code wat not really pythonic, you may want to try/except. Then try only on do in on the minimum possible lines.

The same notices apply to the second block of code.

If you need to read the same files multiple times. you could store them in RAM using StringIO or easier keep a {path: content} dict that you only read once.

Python regex are known to be slow, your data seems pretty simple, you may consider using split and strip methods on your inputlines.

 striped=[l.split() for l in [c.strip() for c in file_desc.readlines()] if l] 

I recommend you to read this : the correspondig video is here

Original Thread

By anonymous    2017-09-20

This solution is pythonic, elegant and fast: (by the Famous Raymond Hettinger in one of his many conferences).

dict.setdefault is the dictionary method that initialises a key-value if the key is not found in dict as well as performing dict.get for provided key.

l = ['foo','bar','help','this','guy']
d = {}
for e in l:
    key = len(e)
    d.setdefault(key, []).append(name)


{3: ['foo', 'bar', 'guy'], 4: ['help', 'this']}

This solution is the modern way of the solution above:

defaultdict from collection is a subclass of dict that automatically initialises value to any given key that is not in the defaultdict.

from collections import defaultdict
l = ['foo','bar','help','this','guy']
d = defaultdict(list)
for e in l:
    key = len(e)


defaultdict(<class 'list'>, {3: ['foo', 'bar', 'guy'], 4: ['help', 'this']})

Original Thread

By anonymous    2017-09-20

I recently watched a talk by Raymond Hettinger which this post loosely reminded of, so I thought I'd link to it. I think it makes a great argument for the approach you described in you post:

Make the API work for you

First of all, I don't think overriding __getattr__ is as sinful as that post might have you believe.

class MyAPI(object):
  base_args = {'token': 'foobarauthtoekn'}

  def __init__(self, ip):
    self._api = ServerProxy(ip)

  def _execute(self, method='', **kwargs):
    argdict = {'Method': method}

  def __getattr__(self, attr):
    return self.__dict__.get(
      attr, partial(self._execute, method=attr))

api = MyAPI('')

# Execute({'token': ..., 'Method': 'get_users', 'arg1': 'foo' ...}) =
api.get_users(arg1='foo', arg2='bar')

# Execute({'token': ..., 'Method': 'get_data', 'foo': 'stuff' ...}) =
api.get_data(foo='stuff', bar='zxcv')

I like this because it's not a lot of code, and it lets us use Execute in a more convenient way (with keyword args), without relying on support for that from the internal API.

Special behavior via metaclasses

The other approach prevents us having to override __getattr__ in case I've underestimated how much of a mistake it is, and it could also be considered more pythonic, as we explicitly enumerate the methods we'll be providing with our API wrapper:

class APIMeta(type):
  def __new__(cls, clsname, bases, attrs):

    def exec_generic(name):
      base_args = {'token': 'foobarauthtoekn'}
      def exec_wrapper(self, params):
        args = {'Method': name}
        return self._api.Execute(args)
      return exec_wrapper

    new_attrs = {
      name: val if name.startswith('__') else exec_generic(name)
      for name, val in attrs.items() }

    return super(APIMeta, cls).__new__(cls, clsname, bases, new_attrs)

class MyAPI(object, metaclass=APIMeta):
  def __init__(self, ip):
    self._api = ServerProxy(ip)

  def get_users(self):

  def get_data(self):

  # ...

In reality, I don't like this, since it's little more than a very convoluted way to write:

class MyAPI(object):
  _base_args = {'token': 'foobarauthtoken'}

  def __init__(self, ip):
    self._api = ServerProxy(ip)

  def get_users(self, params):
    argdict = {'Method': 'get_users'}
    return self._api.Execute(argdict)

  # ...

It will save you a considerable amount of typing if you have a lot of methods though, and it's also good practice in meta-programming if you're looking to get into that or deepen your understanding of the python language.

Here's a demo with both.

Original Thread

By anonymous    2017-10-22

Raymond Hettinger suggested `nobreak` keyword to be introduced, however the proposal never got through... more [here](

Original Thread

Recommended Books

    Submit Your Video

    If you have some great dev videos to share, please fill out this form.