An exercise to help build the right mental model for Python data.
- Solution: https://memory-graph.com/#codeurl=https%3A%2F%2Fraw.githubusercontent.com%2Fbterwijn%2Fmemory_graph_videos%2Frefs%2Fheads%2Fmain%2Fexercises%2Fexercise9.py&play=
- Explanation: https://github.com/bterwijn/memory_graph?tab=readme-ov-file#python-data-model
The “Solution” link visualizes execution and reveals what’s actually happening using 𝗺𝗲𝗺𝗼𝗿𝘆_𝗴𝗿𝗮𝗽𝗵: https://github.com/bterwijn/memory_graph
Hot take: if
a += bis not the same asa = a + b, you done fucked upIt’s definitely not the same. Similarly for a class you can define the
__add__dunder method fora + band separately the__iadd__dunder method fora += b. The first creates a new object, the latter changes/mutates the existing objecta. For immutable types it is the same though.For immutable types it is the same though.
The most twisted thing I learned is that all
ints below a fixed limit share the sameid()result, so>>> x = 1 >>> id(x) 135993914337648 >>> y = 1 >>> id(y) 135993914337648But then suddenly:
>>> x = 1000000 >>> id(x) 135993893250992 >>> y = 1000000 >>> id(y) 135993893251056Using
id()as a key indict()may get you into trouble.Using id() as a key in dict() may get you into trouble.
IMO, using id() as a key would never be a good idea under any circumstance.
Two different (and even unequal) objects can have the same id():
>>> x = [1] >>> id(x) 4527263424 >>> del x >>> x = [2] >>> id(x) 4527263424 >>> del x >>> y = [3] >>> id(y) 4527263424Note - a dictionary lookup already looks up the key by id() first as a shortcut (under-the-hood), so there’s no need to try doing this as an optimization.
Edit: in case it wasn’t clear above, the object with the same id()s don’t all exist at the same time; but if you store their ids as a key, you’d have to ensure the object lifetimes are identical to be sure the ids could identify the same stored value. The dictionary does this for you when you use the key object, but it’s not automatic when using the id of the key.
Other Note - Since you phrased it as “all ints below a fixed limit share the same id() result”, I’d suggest a better way to semantically think of it is that certain constant objects are pre-allocated, and thus are kinda like singletons. There is usually only one
int(1)object, and the language keeps a pre-allocated pool of these common small ints (since they are used so often for indexing anyway).Similarly, many short string constants are ‘interned’ in a similar way; they aren’t pre-created at startup, but once they are created by the user declaring a string constant when the code is run, it saves memory to check and only keep one copy of those string objects, as the string constants can be checked at byte-compile time. But if you construct the string with code, it doesn’t bother to check all the strings to see if there exists an identical one. So for example:
>>> x = 'ab' >>> y = 'ab' >>> id(x) == id(y) True >>> s = 'a' >>> s = s + 'b' >>> id(s) == id(x) False >>> s == x == y TrueBut you can force it to make this check; it’s something they made more tedious to do in Python3 since it’s really an implementation detail:
>>> import sys >>> s = sys.intern(s) >>> id(s) == id(x) TrueSorry for the verbose reply; hope it helped.
Oh absolutely, I understand that the language allows implementations to violate my proposed equivalence — I’m saying that’s a bad implementation (some might say a bad language, for allowing bad implementations, but I don’t necessarily agree)
That’s what you get because you’re afraid of pointers 😁! /j
What the fuck
I expect A if “b” is a clone, or E if it’s a reference. But I also wouldnt combine array operations like this.
The answer being C feels like a bug.
In my limited understanding, the 5th step
b = b + [4]would cause problems, infinite execution or alike, if it kept being a reference.Coming from MATLAB, anything but A feels like a bug. I don’t want my script to use references when initializing a variable unless I tell it to.
Eh, I get it. The equal operator creates a reference but the plus operator isn’t destructive so it creates a new list and overwrites the variable b with a new list, when assigned.
Of course, this would all be avoided if creating copies was the norm; which is why I stick with functional languages.
Copying a list with a million elements every time you make a small change is not fun. Sure, you can optimize a bit behind the scenes, but that still gives a lot of overhead.
And we can create data structures and algorithms that fit a more functional style without relying on imperative assumptions of how data should be handled. Data structures like vlists could be applicable, for example.
Tell me again how python is easy to learn for beginner programmers.
Other languages that have similar behavior include Java and JavaScript, and yes you have to be careful with list / array operations in those languages as well, lest you operate on the wrong list inadvertently. Happened to me. It will happen to you.
You don’t have to compile. You don’t need semicolons.
Python was my first programming language, and those two things alone honestly are really nice. Doesn’t mean there aren’t a million other issues and difficulties, though, lol.
Let’s look at the suspects.
b.append(), you add to the existing list. This mutates the current list. You can not be the culprit.
b = b + [], you join up lists, making a new list in the process, that then gets stored in variable b, without changing the original list that’s still stored in variable a. You are not the culprit either.
No, the culprit must be someone that ambiguously looks both like a mutation and an instantiation.
Isn’t that right, b += []? Because the culprit… IS YOU!
