Ned Batchelder - Facts and Myths about Python names and values - PyCon 2015

By: PyCon 2015

531   3   30752

Uploaded on 04/11/2015

"Speaker: Ned Batchelder

The behavior of names and values in Python can be confusing. Like many parts of Python, it has an underlying simplicity that can be hard to discern, especially if you are used to other programming languages. Here I'll explain how it all works, and present some facts and myths along the way. Call-by-reference? Call-by-value? The answer will be clear!

Slides can be found at: and"

Comments (19):

By anonymous    2017-09-20

Normally I would expect data to stay untouched when passing it to a function as this has its own separate namespace.

x in the function and data at the module level are two names for the same object. Since that object is mutable, any changes made to it will be "seen" regardless of which name is used to refer to the object. Namespaces can't protect you from that.

x /= 10 divides every element of the NumPy array by 10. The original data is gone after this line executes. If you were to run f(data) a few more times, you'd find the contents draw closer to 0.0 each time.

Lists are a more familiar example of the same effect:

l = list(range(4))
# [0, 1, 2, 3]
l += [4]
# [0, 1, 2, 3, 4]

For a good overview of this sort of thing (including related issues) I recommend Ned Batchelder's “Facts and Myths about Python Names and Values” (26 minute video from PyCon US 2015). His example of list "addition" starts about 10 minutes in.

Behind the Scenes

/ and /= (and similar pairs of operators) do different things. Tutorials often claim that these two operations are the same:

x = x / 10
x /= 10

...but they're not. Full details can be found in The Python Language Reference, 3.3.7. Emulating Numeric Types.

/ calls the __truediv__ (or maybe __rtruediv__ --- a topic for another day) method on one of the two objects, feeding the other object as the argument:

# x = x / 10
x = x.__truediv__(10)

Typically, these methods return some new value without altering the old one. This is why data was unchanged by x / 10, but id(x) changed --- x now referred to a new object, and was no longer an alias for data.

/= calls a completely different method, __itruediv__ for the "in-place" operation:

# x /= 10
x = x.__itruediv__(10)

These methods typically modify the object, which then returns self. This explains why id(x) was unchanged and why data's contents had changed --- x and data were still the one and only object. From the docs I linked above:

These methods should attempt to do the operation in-place (modifying self) and return the result (which could be, but does not have to be, self). If a specific method is not defined, the augmented assignment falls back to the normal methods [meaning __add__ and family --- KJC].

If you look at the methods of different data types, you'll find that they don't support all of these.

  • dir(0) shows that integers lack the in-place methods, which shouldn't be surprising, because they're immutable.

  • dir([]) reveals only two in-place methods: __iadd__ and __imul__ --- you can't divide or subtract from a list, but you can in-place add another list, and you can multiply it by an integer. (Again, those methods can do whatever they want with their arguments, including refuse them... list.__iadd__ won't take an integer, while list.__imul__ will reject a list.)

  • dir(np.linspace(0, 1, 5)) shows basically all of the arithmetic, logic, and bitwise methods, with normal and in-place for each. (It could be missing some --- I didn't count them all.)

Finally, to re-reiterate, what namespace these objects are in when their methods get called makes absolutely no difference. In Python, data has no scope... if you have a reference to it, you can call methods on it. (From Ned Batchelder's talk: Variables have a scope, but no type; data has a type, but no scope.)

Original Thread

By anonymous    2017-09-20

Your mental model is pretty much correct. However, you should not worry about whether Python is "pass by value" or "pass by reference". You have to learn how assignment in Python works. This is hands down the best resource to do so. The most important fact to know is that assignment never copies data.

Once you understand assignment, the only thing you have to know is that function parameters are passed by assignment.

The first thing that happens implicitly when you call your function is

NN = unsort_letters

You passed in unsort_letters as the argument. You assign another name (NN) to that argument - and that's it, no data is copied.

Is there ever a circumstance when I cannot change the contents of the passed parameter within the function

Yes, when whatever you pass in is immutable. Integers, for example, have no methods to update their value, so you can't mutate them in the function body (or anywhere else).

It is important to note however that Python does not treat mutable and immutable types differently during assignment and parameter passing. You simply cannot mutate immutable types because they have no mutating methods on them.

Original Thread

By anonymous    2017-09-20

As @DeepSpace mentioned in the comments, Ned Batchelder does a great job demystifying variables (names) and assignments to values in a blog, from which he delivered a talk at PyCon 2015, Facts and Myths about Python names and values. It can be insightful for Pythonistas at any level of mastery.

In addition, take a look at a popular SO documentation related to this topic, Creating variables and assigning values. Here you will find clear and thorough examples in Python you may find helpful.

Original Thread

By anonymous    2017-09-23

The docs carefully avoid those terms, because they are used inconsistently across communities. As far as I know, the same call mechanism is called "call by value" by Java people and "call by reference" by Ruby people.

I think it was Ned Batchelder who proposed the term "call by assignment" - because that's what implicitly happens when you call a function (edit: in Python!).


def foo(x, y):
    return x + y

a = [1, 2, 3]
b = [4, 5, 6]
foo(a, b)

Implicitly, foo does the following before executing the rest of the function's body:

x = a
y = b

which are two assignments. So once you understand assignments - most importantly that assignment never copies data - you understand function calls.

Relevant links:
Facts and Myths about Python Names and Values (video)
Facts and Myths about Python Names and Values (html)

Original Thread

By anonymous    2017-11-20

Assignment never copies data. If you have a function foo that returns a value, then an assignment like result = foo(arg) never copies any data. (You could, of course, have copy-operations in the function's body.) Likewise, return x does not copy the object x.

Your question lacks a specific example, so I can't go into more detail.

edit: You should probably watch the excellent Facts and Myths about Python names and values talk.

Original Thread

By anonymous    2017-12-04

Ned explains everything your need to know [here]( perfectly.

Original Thread

By anonymous    2017-12-04

From Ned Batchelder - Facts and Myths about Python names and values - PyCon 2015:

Functions like x.append("something") and x.remove("something") mutate a value, and change their values. However, using x=x+["something"] or x=x[:-1] rebind a reference, and create a new value which the variable now points to.

Thanks to @timgeb for commenting the video!

Original Thread

By anonymous    2018-01-07

twos.append(ones) does not copy ones.

There is only ever one list ones in memory, which also goes by the following references:

  1. terms[0]
  2. twos[0]

and also terms[-1] and twos[-1] because terms and twos only have one element each, so the first is the last.

Now, when you mutate ones/terms[0]/terms[-1]/twos[0]/twos[-1] you are mutating the same list in memory.

I highly recommend watching Facts and Myths about Python names and values.

Original Thread

By anonymous    2018-03-26

What's the proper dupe target for [Facts and Myths about Python Names and Values](

Original Thread

By anonymous    2018-05-01

Please watch:, it will be worth the time.

Original Thread

By anonymous    2018-05-09

I strongly encourage you to watch [**THIS**](

Original Thread

By anonymous    2018-05-09

Remember in python, strings are immutable. So you can't change them. (I am guessing by 'I can't keep the change to a variable', you were expecting the global test_variable to change as well.

You are essentially assigning names. This talk is a good place to start to understand some core python concepts:

Original Thread

By anonymous    2018-05-09

It's extremely hard for me to comprehend your code because all variable names are meaningless letters (edit: some of which you *reuse*!). Also, could you come up with a shorter example that shows the same behavior that irritates you? In the meantime, I have a hunch that [this]( will help you in any case.

Original Thread

By anonymous    2018-05-29

If this behavior surprises you, you **must** watch [facts and myths about Python names and values](

Original Thread

By anonymous    2018-06-22

Looks like you want to add method xtradata_rm to str objects self.artist and self.track.

One thing that you misunderstand about Python is that can't change your object by assigning something to the variable self (or any other variable). self = 123 doesn't change the object, that is behind the name self to 123, it makes the name self point to object 123 (and do it only inside current scope).

To really get this distinction you should watch the talk Facts and Myths about Python names and values by Ned Batchelder.

The other thing is that str objects are immutable, so even if names worked as you expected, you simply could not modify str. For example, bytearray is mutable, and str is not, see the difference:

In [1]: b = bytearray(b'My example string')

In [2]: id(b)
Out[2]: 4584776792

In [3]: b[3:10] = b'modified'

In [4]: b
Out[4]: bytearray(b'My modified string')

In [5]: id(b) # same object
Out[5]: 4584776792

In [6]: s = 'My example string'

In [7]: s[3:10] = 'modified'
TypeError                                 Traceback (most recent call last)
<ipython-input-7-22fe89ae82a3> in <module>()
----> 1 s[3:10] = 'modified'

TypeError: 'str' object does not support item assignment

In [8]: new_s = s.replace('example', 'modified')

In [9]: id(new_s) # different object
Out[9]: 4584725936

In [10]: id(s)
Out[10]: 4584762296

In [11]: s # original string unmodified
Out[11]: 'My example string'

So to implement your method we need to create wrapper for str object that looks like str and acts like str, but also implements your method. This can be rather hard, for many complicated reasons proxying objects in python is a really involved ordeal.

But fear not! In the dephs of standard library lives a class (144 lines of boring code) just for you: collections.UserString.

All we need to do is to subclass it and implement your method on it:

class SongAttribute(collections.UserString):
    def example_mutate(self):
        """Works UNLIKE other string methods, mutates SongAttribute object,
        but I think this is how you want your code to work. Jugging just a bit ;) 
        Note: actual str object still is immutable and wasn't mutated, now just references another immutable str object.

        P.S.: is the object being proxied by UserString class
        """ =' ', '_')
        return self

    def example_return_new(self):
        """Works like all other string metods, returns new string"""
        return self.replace(' ', '_')

song = SongAttribute('My Song Name') # creating new song attribute (artist or track)
print(song, type(song)) # it looks like str, but isn't
print(song.upper(), type(song.upper())) # it has all of the str methods, but they return SongAttribute objects, not str objects.

# Return new
new_song = song.example_return_new()
print(new_song, type(new_song)) # we got underscored SongAttribute

# Mutate
print(song, type(song))
print(song.example_mutate(), type(song.example_mutate())) # this method changed song object internally
print(song, type(song)) # and now we still see the changes


My Song Name <class '__main__.SongAttribute'>
MY SONG NAME <class '__main__.SongAttribute'>

My_Song_Name <class '__main__.SongAttribute'>

My Song Name <class '__main__.SongAttribute'>
My_Song_Name <class '__main__.SongAttribute'>
My_Song_Name <class '__main__.SongAttribute'>

Now you can implement your method on SongAttribute, and change SongData constructor to:

def __init__(self, datapoint):
    self.artist = SongAttribute(datapoint['artist'])
    self.track = SongAttribute(datapoint['name'])

Original Thread

By anonymous    2018-06-22

I'll add the talk [Facts and Myths about Python names and values]( Really recommend it for anyone struggling with "pass by name/value/reference" in python

Original Thread

Submit Your Video

If you have some great dev videos to share, please fill out this form.