Ned Batchelder - Facts and Myths about Python names and values - PyCon 2015
531 3 30752
"Speaker: Ned Batchelder
The behavior of names and values in Python can be confusing. Like many parts of Python, it has an underlying simplicity that can be hard to discern, especially if you are used to other programming languages. Here I'll explain how it all works, and present some facts and myths along the way. Call-by-reference? Call-by-value? The answer will be clear!
Slides can be found at: https://speakerdeck.com/pycon2015 and https://github.com/PyCon/2015-slides"
By anonymous 2017-09-20
Normally I would expect data to stay untouched when passing it to a function as this has its own separate namespace.
x in the function and
data at the module level are two names for the same object. Since that object is mutable, any changes made to it will be "seen" regardless of which name is used to refer to the object. Namespaces can't protect you from that.
x /= 10 divides every element of the NumPy array by 10. The original data is gone after this line executes. If you were to run
f(data) a few more times, you'd find the contents draw closer to 0.0 each time.
Lists are a more familiar example of the same effect:
l = list(range(4)) print(l) # [0, 1, 2, 3] l +=  print(l) # [0, 1, 2, 3, 4]
For a good overview of this sort of thing (including related issues) I recommend Ned Batchelder's “Facts and Myths about Python Names and Values” (26 minute video from PyCon US 2015). His example of list "addition" starts about 10 minutes in.
Behind the Scenes
/= (and similar pairs of operators) do different things. Tutorials often claim that these two operations are the same:
x = x / 10 x /= 10
/ calls the
__truediv__ (or maybe
__rtruediv__ --- a topic for another day) method on one of the two objects, feeding the other object as the argument:
# x = x / 10 x = x.__truediv__(10)
Typically, these methods return some new value without altering the old one. This is why
data was unchanged by
x / 10, but
id(x) changed ---
x now referred to a new object, and was no longer an alias for
/= calls a completely different method,
__itruediv__ for the "in-place" operation:
# x /= 10 x = x.__itruediv__(10)
These methods typically modify the object, which then returns
self. This explains why
id(x) was unchanged and why
data's contents had changed ---
data were still the one and only object. From the docs I linked above:
These methods should attempt to do the operation in-place (modifying
self) and return the result (which could be, but does not have to be,
self). If a specific method is not defined, the augmented assignment falls back to the normal methods [meaning
__add__and family --- KJC].
If you look at the methods of different data types, you'll find that they don't support all of these.
dir(0)shows that integers lack the in-place methods, which shouldn't be surprising, because they're immutable.
dir()reveals only two in-place methods:
__imul__--- you can't divide or subtract from a list, but you can in-place add another list, and you can multiply it by an integer. (Again, those methods can do whatever they want with their arguments, including refuse them...
list.__iadd__won't take an integer, while
list.__imul__will reject a list.)
dir(np.linspace(0, 1, 5))shows basically all of the arithmetic, logic, and bitwise methods, with normal and in-place for each. (It could be missing some --- I didn't count them all.)
Finally, to re-reiterate, what namespace these objects are in when their methods get called makes absolutely no difference. In Python, data has no scope... if you have a reference to it, you can call methods on it. (From Ned Batchelder's talk: Variables have a scope, but no type; data has a type, but no scope.)
By anonymous 2017-09-20
Your mental model is pretty much correct. However, you should not worry about whether Python is "pass by value" or "pass by reference". You have to learn how assignment in Python works. This is hands down the best resource to do so. The most important fact to know is that assignment never copies data.
Once you understand assignment, the only thing you have to know is that function parameters are passed by assignment.
The first thing that happens implicitly when you call your function is
NN = unsort_letters
You passed in
unsort_letters as the argument. You assign another name (
NN) to that argument - and that's it, no data is copied.
Is there ever a circumstance when I cannot change the contents of the passed parameter within the function
Yes, when whatever you pass in is immutable. Integers, for example, have no methods to update their value, so you can't mutate them in the function body (or anywhere else).
It is important to note however that Python does not treat mutable and immutable types differently during assignment and parameter passing. You simply cannot mutate immutable types because they have no mutating methods on them.
By anonymous 2017-09-20
As @DeepSpace mentioned in the comments, Ned Batchelder does a great job demystifying variables (names) and assignments to values in a blog, from which he delivered a talk at PyCon 2015, Facts and Myths about Python names and values. It can be insightful for Pythonistas at any level of mastery.
In addition, take a look at a popular SO documentation related to this topic, Creating variables and assigning values. Here you will find clear and thorough examples in Python you may find helpful.
By anonymous 2017-09-23
The docs carefully avoid those terms, because they are used inconsistently across communities. As far as I know, the same call mechanism is called "call by value" by Java people and "call by reference" by Ruby people.
I think it was Ned Batchelder who proposed the term "call by assignment" - because that's what implicitly happens when you call a function (edit: in Python!).
def foo(x, y): return x + y a = [1, 2, 3] b = [4, 5, 6] foo(a, b)
foo does the following before executing the rest of the function's body:
x = a y = b
which are two assignments. So once you understand assignments - most importantly that assignment never copies data - you understand function calls.
By anonymous 2017-11-20
Assignment never copies data. If you have a function
foo that returns a value, then an assignment like
result = foo(arg) never copies any data. (You could, of course, have copy-operations in the function's body.) Likewise,
return x does not copy the object
Your question lacks a specific example, so I can't go into more detail.
edit: You should probably watch the excellent Facts and Myths about Python names and values talk.
By anonymous 2017-12-04
Ned explains everything your need to know [here](https://www.youtube.com/watch?v=_AEJHKGk9ns) perfectly.
By anonymous 2017-12-04
x.remove("something") mutate a value, and change their values. However, using
x=x[:-1] rebind a reference, and create a new value which the variable now points to.
Thanks to @timgeb for commenting the video!
By anonymous 2018-01-07
twos.append(ones) does not copy
There is only ever one list
ones in memory, which also goes by the following references:
twos only have one element each, so the first is the last.
Now, when you mutate
twos[-1] you are mutating the same list in memory.
I highly recommend watching Facts and Myths about Python names and values.