Blog by Sumana Harihareswara, Changeset founder
Changes Coming To Pip In October 2020
I'm working on improving the Python packaging toolchain, foundational work that will (in the long run) make the whole Python experience way less confusing. In the short term this may mess with some people's workflows, so we want lots of people to hear about it now.
We are also doing user experience studies, and want you to sign up if you ever do anything with Python (whatever your level of skill/experience).
Computers need to know the right order to install pieces of software ("to install x, you need to install y first"). So, when Python programmers share software, like when they publish packages on the Python Package Index or internally in large companies, they have to precisely describe those installation prerequisites. And then pip needs to navigate tricky situations when it gets conflicting instructions.
Up until now, pip's been very inconsistent in handling this stuff, which makes it easy for your Python environment to get messed up. That's why we successfully applied for $407K in funding from Mozilla and the Chan Zuckerberg Initiative to finish and roll out a proper dependency resolver for pip. The goal is that pip will get better at handling that tricky logic, and easier for you to use and troubleshoot.
You can test the new behavior (in beta) right now by using an optional flag in pip 20.2. And in pip 20.3, coming in October, the new behavior will be the default.
Once you're using the new resolver, pip is going to be stricter and more consistent. So things won't mysteriously break as much, and we can add more features that lots of people want.
But! Right now, a ton of people unknowingly have Jenga towers of wobbly dependencies in their environments and will run into pain when we make the resolver stricter and more consistent. And this may lead to you getting stuck in troubleshooting, assuming that pip caused the problem, when actually the deeper cause is conflicts among how your upstreams specify requirements (TensorFlow just fixed a related thing, for example).
So: We're trying to get Python users to try out the beta of the new resolver that's available in the current stable release of pip (20.2), fix your own environments, report bugs in your upstreams in advance, and report bugs to us so we can fix them in the next couple weeks. We started spreading the word about this a few months ago. And now: video! People watch videos, I hear? I hope this helps.