My biggest complaint about Python is that it somehow doesn’t get flak for having the same (if not worse) scoping as JS, which gets endless hate for its function-scoped variables. (So much so that block scoped variables are the new normal in JS, but not in Py!)
Take for instance:
>>> powers_of_x = [lambda x: x^i for i in range(10)]
To me this is more absurd than any of the JS “wat”s I’ve seen.
In JS:
powersOfX = Array.from({length: 10}).map((_,i) => x => x^i)
powersOfX.map(f => f(2))
Not a particular fan of the Array.from over list comprehensions in terms of syntax, but I much prefer consistency of semantics JS provides by not adding new syntax.
Here Python has special syntax for both list comprehensions and for anonymous single line functions, and they interact in a highly unexpected way.
Funny that Py doesn’t need semicolons, as I’m reminded of GJS telling me of adding special forms to languages: “beware the allure of syntactic sugar, lest you bring upon yourself the curse of the semicolon” (paraphrased of course, he said it much better). It seems Python has fallen into the trap of syntactic sugar, with the curse manifesting itself not in the form of a semicolon, but in confusing and unexpected interactions between its various special forms. Another example: the walrus operator and all its oddities listed in TFA.
You can achieve the same thing in python by using two lambdas (which is actually what you're doing in JS with map):
>>> powers_of_x = [(lambda i: (lambda x: x**i))(i) for i in range(10)] >>> [f(2) for f in powers_of_x] [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
Or you can use default parameter values to use a single lambda (though this means it can be overridden, it's not semantically equivalent to the js implementation)
>>> powers_of_x = [lambda x, i=i: x**i for i in range(10)] >>> [f(2) for f in powers_of_x] [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
your python snippet is equivalent to the following JS:
And Python's comprehensions are actually a shorthand for writing the above generator with this syntax: yieldExpression for i in iterable. The semantics are consistent with the regular for..in
>>> def comprehension(iterable): >>> for i in iterable: >>> yield lambda x: x**i >>> [*comprehension(range(10))]
is equivalent to
>>> [lambda x: x**i for i in range(10)]
This is all a consequence of python's late binding.
That's a different issue around default arguments that are mutable, rather than default arguments generally.
The lambda case here is more like a default argument of a constant integer (immutable) than a default that is a list, which can be altered directly by the function it's a default of.
The underlying feature that both rely on is that default arguments are set on function declaration instead of function call. Mutable/immutable only affects whether or not you hit the bug.
Parenthetical comprehensions (generator expressions) are equivalent to your expanded function, but list comprehensions are not. `(x for x in range(10))` creates a generator, but using square brackets converts it to a list, including allocating the necessary memory. If you’re trying to do yield and streaming, the parens will serve you better.
I agree totally with that, it's an ugly trap in python, although mitigated by the fact the language doesn't encourage this kind of paradigm so you rarely encounter it.
But more absurd than creating a global variable if you forget "var/let/const" ? More absurd than "this" being schizophrenic ? More absurd than having no namespace for 20 years ?
In my opinion, if standard tooling is able to identify the foot-gun, it’s not too terrible of a foot-gun. I don’t think standard python tooling would catch the above issues, but standard JS tooling would identify a missing variable declaration. As for the other things you mention (this, namespaces), I never said JS was perfect! Just that JS got a lot of shit for its scoping, fixed the problem, and now has sane scopes. Py has scoping with a lot of the same problems JS used to have, but for some reason didn’t fix them, and thus to this day is riddled with these foot-guns.
> I don’t think standard python tooling would catch the above issues,
Of course it does.
$ echo " > powers_of_x = [lambda x: x^i for i in range(10)] > [f(2) for f in powers_of_x] > " > test.py $ pylint test.py ... test.py:2:27: W0640: Cell variable i defined in loop (cell-var-from-loop) ...
Pylint is the most powerful linter for Python is well integrated in VSCode or PyCharm.
When concerns are raised about JS, people are quick to answer you are not supposed to use it without modern tooling (Webpack + babel + typescript + eslint + lowdash...), although many devs still do.
Now, I use and teach a lot of JS, and I enjoy many of its modern features. I especially like the object destructuring, it's even better than Python's unpacking.
But objectively, you can get a very decent experience in Python without any kind of tooling. And yet, there is a lot of it if you want to up your game: pylint, black, mypy, jedi, pytest, poetry, etc.
Are they really foot-guns? The lambda variable reference thing has appeared more in forums comparing languages than I’ve seen it used at all in python and I’ve spent a lot of time working in large open source python projects with thousands of contributors.
Even if you got the binding right, your powers of 2 would be shot down for being unpythonic anyway due to lambdas being shunned when there are cleaner ways to write things.
powers_of_x makes no sense as a variable name for a list of functions that you have to call in order with the same number to actually get the powers of that number. I get that it’s just a contrived example, but most can be turned into someone more apparent to the reader.
def powers_of_x(x, limit=10): for i in range(limit): yield x^i
Less memory, no anonymous functions, more flexibility, testable...
Modules avoid the variable issue because ‘use strict’ is applied by default for them.
IMO JS as a language is in a much better place than Python currently. I end up frustrated a fair amount by arcane errors in Python due to less expressive constructs or some wtf issues like unexpected variable scoping within Python modules.
But even without that, I'm not saying they are comparable. I'm saying the same API will use explicit anonymous callbacks in JS and something else in Python (decorators, subclassing, protocols, generators...). I'm saying that the same API will use __iter__ in Python and something else in JS (type conversion, proxy object, explicit method call...).
E.G, this is a Python pattern you'll find in contextlib or in pytest fixtures:
@somekindofregistration def foo(): print('code that runs before') try: yield except Stuff: print('Error handling') print('code that runs after')
This uses Python iteration mechanism to run code at 3 different times in a life cycle.
Map, filter, and fold are higher order functions, functions which take functions as parameters.
You are misusing the word callback. A callback is a function passed to another thread that will maybe be invoked later as a response (like it calls you back).
In contrast, Lodash goes out of its way not to use the term "callback", presumably to avoid confusion since it's a place where many functions-that-take-a-function-argument do not call the provided function, e.g. https://lodash.com/docs/#curry
Unfortunately, this is not true. Although the equality operators are a favorite bikeshedding topic, the largest source of errors in JS code bases are type errors. Using a "non-sane" equality check, such as those with type coercion would actually mask or alleviate these source of bugs you mention.
I was thinking more of the case where the types match but objects and arrays with the same contents are considered different. I’ve watched every member of my team get stung by it again and again - and then have to create workarounds to get past it.
I don’t agree and this behavior shouldn’t be surprising. The alternative would be to walk the container and compare the value of each element, which could be horrible.
It's very explicit, practical, and you can set the scale of practicality vs performances where you want. Plus: no implicit weird type conversion, only one equality comparison operator, and no hidden rules.
I’m not sure if you’re for or against here. Walking the container is exactly what you have to do, and it is horrible. More importantly, if it’s not your library, you don’t get any choice on how the equality check is implemented.
> I agree totally with that, it's an ugly trap in python, although mitigated by the fact the language doesn't encourage this kind of paradigm so you rarely encounter it.
This. I've made this mistake exactly once in the last 7 years, but that one time almost caused a security issue.
"This happens because i is not local to the lambdas, but is defined in the outer scope, and it is accessed when the lambda is called — not when it is defined. At the end of the loop, the value of i is 4, so all the functions now return 42, i.e. 16."
These sort of scoping gotchas are pretty common across all programming languages and are a great argument for unit testing. As usual the answer is be more explicit what you're asking the language to do:
>>> powers_of_x = [lambda x, i=i: x*i for i in range(10)]
There is nothing bad in general with list comprehension there. Problem is lambda which uses variable from outer scope. In Python, comparing to JS, using variables from outer scope is not so common. I prefer more obvious constructions and in case of problems with scope I can even create class to protect value that I need to manage.
In a decade of python use I can probably count on one hand the number of times lambda has been a good solution to a problem I had. Many times I end up re-writing them as regular functions for clarity sake.
Imperative loop, maybe. But list comprehensions are a sublanguage imported from Haskell and looking like Haskell, so it's somewhat natural to be surprised by mutable variables here.
>My biggest complaint about Python is that it somehow doesn’t get flak for having the same (if not worse) scoping as JS, which gets endless hate for its function-scoped variables. (So much so that block scoped variables are the new normal in JS, but not in Py!)
It's less of a problem in Python because it has built-in module support. A global variable is only global in the module, unless you import it with "from foo import *", which lazy programmers do but is discouraged by style guides.
Maybe I’m misunderstanding, but I’d argue that arrow syntax in js is equally special syntax as lambda function. They’re both native ways of making anonymous functions. Python’s version is more clunky, but no more special
Interesting, I don’t use python day-to-day, so I had assumed that the delayed capturing of the iteration variable was a lambda thing. Looks like this is still broken:
for i in range(10):
def mul(x): return i^x powers_of_x.append(mul)
That to me coming from a JS background is totally wild
By definition, a closures closes over its definition context, it doesn't capture the values at definition time but instead keeps referring to said definition context. If the definition context is mutable and modified, the closure reflects that change when it's finally invoked.
It’s actually due to python late binding. The variables are looked up at call time, rather than function define time. One could argue that your example shouldn’t work anymore in python3 since variables in list comprehensions go out of scope when they finish now. I haven’t tried it myself. I fill it under, “things I never need to do” :-)
It's not "python late binding", closures behave this way in any language with mutable bindings[0]. The entire point of a closure is to close over its lexical context, if the lexical context is mutable and mutated before the invocation of the closure, the closure is going to reflect that change.
will behave the same way because it does the same thing, so will e.g.
for i := 0; i<10; i++ { powers_of_x = append(powers_of_x, func(x int) int { return x ^ i }) }
in Go.
One of the confounding factors is that many languages have block-scoped for loops especially when using iterators instead of low-level C-style loops; that's also the case in Javascript when using `let` bindings (and is in fact one of the major reason to use `let` instead of `var` if that's possible).
The underlying concern is still there[1], but this very common failure case is avoided: rather than update a single binding, each iteration creates a brand new binding for the closure to close over.
Alternatively, a common mitigation technique is to emulate that using e.g. immediately invoked function expressions.
In Python you can also use the "default parameter" trick to shadow the closed-over binding (though most of the time this is used for performance reasons) without the overhead (both syntactic and runtime) of lambdas in lambdas in lambdas:
for i in range(10): powers_of_x.append(lambda x, i=i: x^i)
[0] excluding the special case of low-level languages with capture clauses, as well as languages like Java where closing over mutable bindings is specifically forbidden (a lambda or anonymous class can only close over `final` bindings)
[1] and remains a regular issue in async code fighting over closed-over context
You're right. I remembered the reason after reading your sibling comment and almost deleted my comment to avoid the confusion. Thanks for going into the details.
std::vector<std::function<int(int)>> powers_of_x; for (int i = 0; i < 10; ++1) power_of_x.push_back([=](int x) { return std::pow(x, i); });
works as one would intuitively expect (a different i is captured for each iteration). The issue is with those languages that conflate values with references.
> excluding the special case of low-level languages with capture clauses
Of course it works, you're specifically capturing `i` by value. It's not exactly surprising that doing things completely differently yields a different result.
To be fair I missed the note at the end of your commend on my first read. Still, in the specific C++ example, no other capture clause is valid.
Also, python could have chosen a slightly different closure semantics and preserved sanity: instead of closing over the binding itself, it could close over each object reference separately (exactly in the same way the default parameter hack works).
package main import ( "fmt" ) // return a^n func Power(a, n int) int { var i, result int result = 1 for i = 0; i < n; i++ { result *= a } return result } func main() { fmt.Println("Hello, playground") x := 2 var powers_of_x []int for i := 0; i < 10; i++ { powers_of_x = append(powers_of_x, func(x int) int { return Power(x, i) }(x)) } fmt.Println(powers_of_x) } //Output: //Hello, playground //[1 2 4 8 16 32 64 128 256 512]
I know nothing about go, but it seems to me in your example powers_of_x is a list of n integers, while in the example being discussed is an list of n functions (which can be used to compute the n'th power).
The real WTF here is that you are getting a power output from thee ^ operator, when it actually does xor. Indeed, I've never seen JavaScript mix up operators :)
Even more fundamental is the nonlocal/global stuff required in order to avoid declaring your variables. Many people are surprised by what this snippet does:
Well it's f that has the problem, rather obviously, and the other three lines are unrelated. So I don't see how this is some especially simple way of triggering that error.
The other three lines are not unrelated. If you just take the first three lies (and the last one), it's fine:
a = 0 def f(): print(a) f()
Maybe this is all obvious to you, but I'll bet you can't name any other language which behaves this way. Compare the original Python to JavaScript (or Lua, Perl, Tcl, Ruby, C, Scheme, Rust, Clojure, or ...):
a = 0 function f() { console.log(a) a = 1 } f() f()
This one behaves how I think most people would initially expect the Python snippet to behave. First it prints 0, then it prints 1. (Of course JavaScript has it's flaws too...) If you don't believe me, ask your coworkers and friends what they think it does before running it.
My only real point is that Python conflates variable declaration and variable assignment in a way which initially seems like a friendly time-saver, but which ends up being pretty subtle and confusing until you've learned its quirks. All of that just to avoid declaring your variables (in some fictional version of Python):
var a = 0 def f(): print(a) a = 1 f() f()
Here it would be clear which are declarations + initializations, and which are only assignments. And for just the cost of typing the word "var", the compiler could tell you when you've made typos in your variable names. As an added bonus, you could get rid of the "global" and "nonlocal" keywords.
Of course I'm not a good judge, I have been programming Python since 1.4 and my colleagues also all have Python experience.
But at least I like the idea that it gives an error message when confronted with ambiguity. That way it doesn't do something you didn't expect silently. Sadly it only gives it at runtime, not at compile time.
I'm not sure what your point is about Python 1.4. Maybe it's that you're past the point where these things trip you up.
However, there are plenty of cases where Python won't give an error message too. Make a typo somewhere in the middle of your function, and it'll quietly introduce a new variable instead of letting you know. A rarely used branch of an if-statement could hide this indefinitely. This could also be avoided (or at least mitigated) by requiring variables to be declared explicitly.
You’re sidestepping the problem. The problem is that Python uses the same syntax for variable declaration and variable assignment, and this can lead to unexpected behavior. That’s far from normal in the programming world, making it more unexpected.
Yes! So much that! I can't believe that it's okay in python to declare/initialize a variable within a loop or if statement, and that it goes to live on afterwards. JS fixed that issue years ago with let and const.
It's a good repo, but remember a lot of those are "A Good Thing™".
The first snippet is a very good example:
a := "wtf_walrus"
doesn't work while:
(a := "wtf_walrus")
works.
It's a fantastic design decision.
Python took a long time before getting this operator, because it's a language that favors being readable, easy to use, and above all, to learn.
But in many other languages, the very same operator is often misused as, or confused for, the operator for equality or assignation.
We, as a community, didn't want people to wonder why there are several ways to do assignation (we have this problem with string templating already). Instead, we wanted to be sure that people could ignore the existence of ":=" for some times during their learning process.
And so the decision has been taken to make is very easy to distinguish it from "=" and "==", forcing parenthesis when necessary to make it clear this is a completely different use case. Also to hint people at using it only when necessary.
If you like scripting in Python because it's so easy to go from an idea to code, it's not random luck. It's because the language is a collections of thousands of such decisions.
Why should people only use it when necessary? Why not say “the walrus operator is the exact same as the assignment operator, but it can be used in expressions and thus is spelled a bit differently in order to better distinguish it from the equality operator. Feel free to use it in all places you’d previously use the assignment operator.”. Simple, easy to understand, easy to learn, and most importantly (though python folks might disagree, given 2 => 3), easy to adopt. (Find and replace all single equals with colon equals and you’re done).
Instead, they’ve chosen the narrative: “the walrus operator is a lot like the assignment operator, but it is able to be used in expressions, cannot be used as a statement, and it sits below the comma operator in precedence instead of above it. We understand that this is super confusing, so use it sparingly”.
One way you have two operators for people to learn, with the understanding that there is a third legacy operator that works pretty much the same as one of them. The way they chose you have three operators for people to learn, with two of them behaving pretty similarly but not interchangeably, and with subtle yet important differences.
Because we have experience with things such as easy of learning, cognitive load, writing simple to read code, making it hard to introduce bugs...
This comment show how little experience one can have with it, and yet make a quick judgment to offer to "fix" things. E.G:
> eel free to use it in all places you’d previously use the assignment operator.”. Simple, easy to understand, easy to learn, and most importantly (though python folks might disagree, given 2 => 3), easy to adopt.
Well because in many languages, people do this:
while (foo = bar):
While they mean this:
while (foo == bar):
Even experienced devs do this by mistake from time to time. It's hard for tooling to find out if you are doing something stupid or smart. Beside, you don't use tooling when you learn the language or do a quick script.
Designing a language is not a 15 days job in a rush. Usually.
I don’t think you’ve actually understood my proposal, and the snide remarks about completely unrelated languages don’t help your argument.
To reiterate, my proposal is that the walrus operator has the exact same semantics as the assignment operator, but creates an expression rather than a statement. A linter rule could then be added prohibits the assignment operator, and converts all instances of it to the walrus operator. The codebase now has only the two operators. Language reference books need only explain the two operators, and simply note that the legacy assignment operator behaves the same as the walrus operator but cannot be used in expressions. This is easy to learn.
What the language creators have done instead is add a whole new operator with all new semantics, leading to all new sources of confusion. (See TFA). This is not easy to learn.
Side note: It’s funny that you appeal to the authority of the language creators with phrases like: Python is easy because “the language is a collections of thousands of such decisions”, yet ignore the fact that the actual creator of the language was so against the addition of this operator that he saw the community’s insistence on it as reason to step down as BDFL.
> and the snide remarks about completely unrelated languages don’t help your argument.
Granted, I apologize. It was childish.
> but creates an expression rather than a statement.
Expressions are limited in Python for the same reasons. It's the same rational that got us tone done lambdas. People code with a certain style using it, which is not the style we want to promote for Python.
> A linter rule could then be added prohibits the assignment operator, and converts all instances of it to the walrus operator.
Every time you delegate something to a linter, you fail. A good language must be a good language without 3rd party tooling. Tooling should only be a bonus. This is why we have forced indentation in Python. This is why we have namespaces and don't rely on a bundler.
> What the language creators have done instead is add a whole new operator with all new semantics, leading to all new sources of confusion. (See TFA). This is not easy to learn.
You don't learn this operator when you learn Python. Just like people don't learn about list comprehensions at first, don't use yield and code without creating a decorators for years.
Python scales down: you are productive with it without all its features. It is designed to be partially learned and still be useful. Thousands of Python devs are not coders, but scientists, geographers, finance people, etc. They know very little Python and yet, can work all day with it.
Walrus is made so that it doesn't interfere with the original design. It's something separate that you get to later.
> yet ignore the fact that the actual creator of the language was so against the addition of this operator that he saw the community’s insistence on it as reason to step down as BDFL.
That's not what happened. The bad quality of the debate is what made Guido step down. We had a huge number of new comers in the later years because of Python popularity. These people didn't learn internet from the age of mailing list, IRC, etc. They addressed the topic like others talk on Twitter: lot of noise, little content. I understand that after 30 years of doing what is basically free awesome work for thousands of people, he didn't feel like being disrespected by a horde of juniors that though they knew better.
> yet ignore the fact that the actual creator of the language was so against the addition of this operator that he saw the community’s insistence on it as reason to step down as BDFL.
Regarding this part, Guido specifically addressed the toxic debate as the reason for stepping down as BDFL, and not the operator per se.
(Yes, I'm reinforcing BiteCode_dev's reply on this point)
This isn’t delegating anything to the linter, it’s saying that a linter rule would help some people enforce particular styles (only walrus), if that’s what they want. I.e. the point of a linter.
> the actual creator of the language was so against the addition of this operator that he saw the community’s insistence on it as reason to step down as BDFL.
That's backwards. The community was generally opposed to the walrus operator, and Guido stepped down because of (among other things) the community's reaction when he insisted on adding it.
I'm on Guidos side. When it's well used, it's a good thing -- problem is getting people to use it right. But the fact that many people didn't consider the use cases that were being proposed, immediately took strong and toxic positions about it, and were more interested in calling names rather than having a civilized discussion is what wore down the (then) BDFL and caused him to step down.
As a relative newcomer to Python (compared to most, but I do use it professionally every working day for a year, and some non-working days for personal stuff), I find this to be the case. At work I have to use Django, and it's almost a chore to find anything explicit or obvious in it. Its level of abstraction is "Too Damn High" as the meme goes, IMO.
It feels like saving boilerplate for the sake of it which just makes obvious, readable code into magic incantations where shit happens and you don't know why or how or where to look.
This is the danger with high level frameworks, everything is abstracted away, and with a dynamic language like python the black magic can go very deep.
If you've ever taken a look at something like Flask or Cherrypi, the difference is very apparent.
That said, I've been doing some Rails lately, after years of Django, and I'm finding the experience to be worse from a readability point of view. Metaprogramming everywhere, modifications of language constructs ... It gets very confusing very quickly. Could be simply a lack of experience of course.
In parallel, I've also been using Go and the experience could not be any different. So much easier to understand what is going on, but so much boilerplate code!
It doesn't look like it's possible to have both ease of use and explicitness...
Thanks; I've glanced at Flask and it makes a lot more sense to me, but this is a project of immense size and age, and not under my control, so I just have to live with it. Overabstraction for the sake of it seems to be Django's mantra. Then there's Django Rest Framework, and the 3-4 different filtering mechanisms.
> there should be one-- and preferably only one --obvious way to do it.
Yes, this has been one of the hardest balance to find. And believe me when I tell you the community tries very, very hard. But it's a difficult problem: making the language evolves fast enough, but keeping it solid and stable. Very difficult indeed.
> Python became a language that can be really hard to read now.
Not in my experience.
My job involve going a lot from company to company. I see a lot of code, from a lot of different people.
The way people write Python hasn't change much during the last 15 years.
In fact, I'd say because of Python 3, there is less cruft all in all.
You do have a few heavy stuff. E.G: asyncio or the type hints. But how many code out there uses that ? Very little. Mostly, the code that needs it.
The new walrus operator now makes indentation obsolete. You can write averything within one line as an array and it is hard to read. This in combination with syntactic sugar, operator overloading and unicode variables can make the language very hard to read.
e.g.
# compute pi 1000000 >> ψ( ψ(χ>>op("(x**2+y**2)**0.5<1")@rlµ<<χ)>>Σ*4>> _/_)
or this:
# 10 fibonacci numbers [x:=[1,1]] + [x := [x[1], sum(x)] for i in range(10)]
is valid python code. (for the first see my github jamitzky/iverson). Not that I wouldnt like it, but python3 has changed and the zen of python is not valid for some time now.
As for the unicode caracters as variables... Remember you could do that in Python 2 ?
# -*- coding: rot13 -*- cevag "Relax"
But people don't do it. Just like they don't use "import *" everywhere, or monkey patch methods like it's going out of style the way Ruby loved it 15 years ago.
Because such capabilities are restricted (one line lambda, parenthesis for walrus, not all unicode is allowed as var names or everywhere...), introduced slowly, and the community culture is to value readability, Python stays Python.
I was against the walrus personally, for the reasons you mention. But while I do see Raymond Hettinger trolling twitter regularly with his latest crazy walrus magic, in production we see no such thing.
Not to say it never happens. It does, I've seen monstrosities in the field. Like with every tech. I mean, you can start a fire with a water hose if you try hard enough.
I think Python’s direction changed when Guido moved to Dropbox. Suddenly he was working on a million-line Python codebase, and started working to make the language more suitable for programming in the large.
I understand why people are willing to defend it or make apologies for it, but there really are some lessons to be learned from the mistakes it made, and dismissing legitimate criticism just ignores the opportunity.
I agree, there is a long list of things I would change in Python. And believe me when I say people involved in the community listen to this very carefully.
This is why we had Python 3 in the first place. Because text handling was such a cause of pain.
This is why we have type hints in the first place. Because big projects using Python felt let down.
And Python 3 and type hints are also a huge source of criticism (I was the first one to do so).
It's never ending. People will complain. People will be unhappy. Things, also, will be imperfect, as we are limited in resources and are submitted to many constraints. Not to mention taste and opinion, that are used to request changes.
But I'm not answering to dismiss valid concerns. I'm answering because as a very experienced Python dev, I now have a good grasp of what is due to culture, lack of experience, or a real deal.
E.G:
- := forcing parenthesis. Has not been a real problem in my experience, at least as of today.
I suspect it's healthier to judge a few misfeatures in a programming language than to judge all developers, or people as a whole. I'm really very optimistic about the future of humanity, but less so about the future of Python :-)
I bet neither will go down, but the human race will improve! Seriously, things are getting better - it's just that paying attention to what needs fixing is how engineers contribute to the progress.
Despite what many apparently think, this line very specifically does not say "there should be one way to do it", it is much more restrictive.
And as far as I'm concerned, the walrus operator breaks it in no way whatsoever: an assignment statement is the one obvious way to perform an assignment, and it's a syntax error to use a walrus as an assignment statement.
The many, many ways to print variables inside of a string are testament to that. I remember feeling kind of a chill when the number of methods hit three. Two is an acceptable number of methods if you are transitioning from one form to another as kind of a nod to the growing pains of a language, but at three you have hit some kind of watershed.
1. manual string concatenation 2. the old, printf-style. Still heavily used in e.g. stdlib logging module. 3. .format() 4. f-strings 5. string.Template in the stdlib
Yup, I definitely see this as one of the bigger failures in Python's design.
It's a problem, but not a failure in design. The language is old. It started with features from that times and evolved to add more modern ones. Removing the old ones would break the world. Why do you thing you can still use == and declare vars without keywords in JS ?
Even in the p3 transition, the community screamed to not remove them. Did you know we removed % for a few versiond ? We had to put it back because of all the complaints.
in practice it's 3. and 4. though. we outright ban 2. through a linter. it not only looks ugly, but is inconsistent. e.g. it has the tuple pitfall with a (language?) hack to make it work with one variable. even for logging, it's possible to use the `.format` style - although this isn't default. i've never seen `string.Template` actually used, except for niche use-cases with untrusted input, where a full-blown templating engine was overkill.
> The very same operator is often misused as, or confused for, the operator for equality or assignation.
It will still be confused, I'm afraid. It's just a little less likely that that confusion will be silently accepted by the interpreter.
> Also to hint people at using it only when necessary.
Let's be clear: the walrus operator was never, ever, necessary. It's syntactic sugar to avoid extra lines of code. The hope is that it will be convenient and obvious enough that folks will prefer it to both extra lines of code and repeated sub-expressions, but the very advice that beginners ought to "ignore the existence of := for some time" (_plus_ the intentional deviation from other languages) suggests that this is not going to be nearly as obvious or convenient as its proponents might have wished.
Did you get far past the first example? Because a lot of these seem just plain unacceptable..
How about this from the second example:
>>> a, b = "wtf!", "wtf!" >>> a is b # All versions except 3.7.x True >>> a = "wtf!"; b = "wtf!" >>> a is b # This will print True or False depending on where you're invoking it (python shell / ipython / as a script) False
Or this from further down
>>> a = 256 >>> b = 256 >>> a is b True >>> a = 257 >>> b = 257 >>> a is b False
I agree. I wonder how confusing it would be to make `is` not allowed on integers or strings. Or define it to be equivalent to `==` for those types. Using `is` in these situations is almost always a big, if for whatever reason you really need it you can do `objectid(x) == objectid(y)` which is usefully explicitly.
I thought this walrus thing looked esoteric and ridiculous. Two words which also describe Python. I think it's prime time for the scripting languages to be disrupted. Twould be nice to have one as well designed and consistent as Rust.
Probably the best collection/explanation of edge cases which are not a bug and supposed to work that way by design that I ever have seen for Python. (it's my pramary language for almost 10 years).
All the id() equivalencies only exist for optimization, and not out of some promise to the user.
The walrus operator can be confusing, but any language can be made to look confusing. It's an advanced feature that should be used very scarcely (if at all).
It does point out a few real weak points in Python, but most of them require non-trivial usage of the language.
"Effect and exception" is bad, but its a special case of a more general bad design - the fact that `a += b` is sometimes equivalent to `a = a + b` and sometimes changes in-place.
So the question is if += and sort are both "in-place", why does one throw and the other doesn't? Clearly "in-place" is not the full explanation here.
I think the real explanation is that with += there are two steps:
1- The existing list gets modified. This is fine since lists are mutable.
2- The tuple's reference to the list gets updated (even though this update is unnecessary since the list object's identity is the same).
The exception occurs at step 2, but this step is otherwise a no-op. Whether mutating a reference to the same value it already had is an actual mutation is an interesting semantics debate.
Tuples are immutable, but that doesn't mean they can't contain references to mutable objects.
When you want to use a tuple as a key in a dictionary, Python checks if all members of the tuple are immutable. If one or more of them aren't, you'll get an 'unhashable type' error.
Yeah, typing KINDA works until you try to use a library that has no annotations and uses reflections, such as - say - plumbum, boto3 or some other pre-typing shit. It's a bit like with async code - once you decide to use it, you can keep the language but basically need a whole new ecosystem. I tried to write an actual company project with `mypy --strict` passing as a requirement and you quickly end up having abstractions only to bypass mypy, as well as surpress comments as uses of Any. And this is where it pays off to just switch to a statically typedk, compiled language.
mypy --strict is only useful for projects for which everything is defined.
I recently did a project with tornado, which is typed. Typing helped me immensely. My editor would give me type hints, errors and show me other edge cases.
For an existing Django project I'm adding types when I touch functions. It helps a bit, but way less. That's also because typing almost forces you to change all the dicts that get's passed around in dataclasses. Which is a lot of work after in an existing project.
It's interesting to see the walrus operator in there. I wonder, is this another example of unnecessary features being added just to claim a bigger change list, or someone really needs this operator? Why would anyone prefer a walrus instead of adding just one more line to their code, which also helps with readability?
Coming from Mathematica to Python, the lack of a assignment expressions yet inclusion of list comprehensions felt like a severe crippling of the language feature. Sure you “can still do the same through other means”, just like if, for and while are all unnecessary syntactic sugar in every language. But the thing is that list comprehensions provide a nice obvious concise way to frame your solution, but if you use it without assignment expressions you often end up repeating computations unnecessarily.
A while ago, I was also sceptical about the walrus operator, although I use assignments in expressions all the time in C++. But when I was perusing a library written Python the other day, I found five spots in which the walrus operator would make sense - eliminating one line of source code without readability suffering from it.
Python's principle is not an has never been "one way to do it". Python's principle is:
> There should be one — and preferably only one — obvious way to do it.
Which is a very, very different assertion. And there remains one — and exactly only one — obvious way to do an assignment, because the walrus operator is literally invalid syntax as a statement:
That principle was bullshit anyway. There have always been multiple ways to do things, the one way was just whatever the elder Pythonista deemed to be pythonic that day.
Not true. The language optimizes for the way it is intended to be used, and changes remain sensitive to those optimizations.
"Pythonic" means intended usage, and "unPythonic" is shorthand for "you found another way to do it that kinda does what you want but (is ten times as slow/takes up ten times as much memory/doesn't work for edge-cases/has more unintentional side-effects) because it wasn't the intended usage, which is fine for your own personal projects, but please don't bring that into my code base, and pretty please don't teach other people to do it that way..."
In my work we have code in many places along the lines of:
data = expensive_function(blah, blah_blah) if data: # many lines of processing data = expensive_function(blah, blah_bah)
And seen a lot of times where newcomers forget the assingment at the end that makes everything move. So yeah, the walrus version would be a lot simpler:
while data := expensive_function(blah, blah_blah): # process data
This is just one of the "edge cases" where the walrus makes sense.
In my experience Python is not very nice to work with. I do like the ecosystem for data science though, it's just amazing how much there is. Hopefully the language will grow into something better now that the dictator stepped down.
Second this. I can’t understand finding Python beautiful or even elegant. It just took a hodgepodge of features from languages like Haskell and C++, and repackaged them clumsily. It’s good for writing short scripts and throw-away code, but it shouldn’t be used as a serious programming language.
Fixings a lot of the WTFs requires breaking backwards compatibility.
Given how long it took to transition to Python 3 and how painful the transition was, I'm not sure people will have the appetite or patience for this anytime soon.
Yeah, I'm relatively new to Python as a JVM dev, but am finding myself (accidentally) in the realm of Data Science/Engineering and am looking to Python as opposed to Scala simply due to better Libraries. I do like that Python is really easy to be productive in, but the challenges of scale I guess will be learned as our team gains experience.
I try to keep my data science work in Python limited to short scripts for reading a csv and then making some matplotlib figures. I personally find it unsuitable for anything beyond that
If only it had curly braces instead of just indention. God, that kills me. And used unicode instead of ascii as the default. And there wasn't python 2.7 vs 3. Someone help me stop this list.
I've coded in python for >10 years and C# for ~half of that. I prefer indentation-based control flow purely because it condenses code vertically and creates consistent indentation with actual purpose (many of the devs on C# projects I've dealt with have had inconsistent tab/space settings and no linting).
If anything, the one thing I hate about Python3 is dynamic typing - inferring datatypes in function bodies is great but strongly typed (and _enforced_) parameters and return values would clean up a lot of the problems I have seen, and created in the past.
I don't disagree with anything you've said, but in my opinion strongly typed function definitions/returns are about communication rather than dictation - I'd rather the compiler tell me that I'm passing an unexpected argument type rather than the runtime executor.
But to contradict your point, there's also a fair amount of Python which is a dictation - Guido even called his job role 'Benevolent dictator for life'. I really hated some of the guidelines as a new programmer but after dealing with them for some time I understand that consistency is far more important than preference in many/most/all cases.
Hmm, I'm not really a fan of indentation based control flow, but I see the appeal of condensing code vertically.
I primarily work with C#, but I've gradually come to like the "K&R"/Javascript style (where the 1st brace is at the end of the first line), precisely because it helps condense code vertically.
I have worked at places where they insisted on 2 space indention, which made it much much harder to read the code and follow indention levels. It was just stupid of course. That's what really turned me off of python. I know there are bad formatting options for c style languages, and the editor should make it clear. but 2 spaces is too little, too late.
But it doesn't work this way. You don't work strictly with the language itself, you work with the whole ecosystem. Libraries, tools, snippets, SO questions, etc. And many of those are still in 2.7 or at least need to specify different solutions for 2.7... It is still a pain.
> Libraries, tools, snippets, SO questions, etc. And many of those are still in 2.7 or at least need to specify different solutions for 2.7... It is still a pain.
Admittedly there were some libraries that took a long time to switch, but at this point there is no serious library left, that did not make the switch.
Exactly. I've gone farther even, and this is my motto now:
Python 3.6+ or burn it to the ground
I mself still have to port a few projects yet, but those that got stuck in 2.7-3.5 land are en route to dying a fiery death, and being reborn in 3.6+ (3.8 where applicable, 3.6 as the lowest I'm willing to accept).
That is evil and wonderful. another reason to love/hate python. I'm working on a project for fun over the holidays and of course I wrote it in python 2.7, and it needs to process unicode so I inflicted double punishment on myself ;-)
My work project just happens to use python 2.7, causing pain for everyone.
Braces in Python would have issues, e.g. ambiguity with set/dict literals. Most similar languages (except JavaScript) seem to use begin/end instead. Would that be more acceptable than significant indentation?
Encoding strings internally as UTF-8 is a fine idea, usually you don't need constant-time access to individual code points. E.g. PyPy (faster Python, with JIT) does so. Many other languages also save strings as UTF-8.
Sure it does. Where CPython does byte inflation to make sure random access works, PyPy makes you use an extra indexing structure for cases where you can't just advance one char at a time.
I must say it is a rather elegant way, but it still fits firmly within "a new way or inefficient code".
Take for instance:
>>> powers_of_x = [lambda x: x^i for i in range(10)]
>>> [f(2) for f in powers_of_x]
[512, 512, 512, 512, 512, 512, 512, 512, 512, 512]
To me this is more absurd than any of the JS “wat”s I’ve seen.
In JS:
powersOfX = Array.from({length: 10}).map((_,i) => x => x^i)
powersOfX.map(f => f(2))
Not a particular fan of the Array.from over list comprehensions in terms of syntax, but I much prefer consistency of semantics JS provides by not adding new syntax.
Here Python has special syntax for both list comprehensions and for anonymous single line functions, and they interact in a highly unexpected way.
Funny that Py doesn’t need semicolons, as I’m reminded of GJS telling me of adding special forms to languages: “beware the allure of syntactic sugar, lest you bring upon yourself the curse of the semicolon” (paraphrased of course, he said it much better). It seems Python has fallen into the trap of syntactic sugar, with the curse manifesting itself not in the form of a semicolon, but in confusing and unexpected interactions between its various special forms. Another example: the walrus operator and all its oddities listed in TFA.