You start with::
pip install django pip install boto...
Now lets deploy. SSH to server, copy dist files, copy source, talk done!
O, missed dependencies. requirements.txt to the rescue
Everything breaks again, because you did not pin your packages yet.
Also pin dependencies of your dependencies:
pip freeze > requirements.txt
pip does not check that your dependencies are consistent to each other.
pip-tools helps with that.
pip-compile recursively compiles dependencies together and checks for consistency
Create a requirement.txt.in with top level requirements.
Run pip-compile, and the requirements.txt contains all pinned requirements.
Every time you run pip-compile, the second level dependencies get updated again. If after that, tests fail, you must update your requirements.txt.in to document the known incompatibility.
pip compile takes time and can be platform specfic. pip install takes time to resolve distributions, resolves depdencies and install the distributions.
Our solution: wheelsproxy a pypi proxy, builds wheels, knows abouts platform. http://github.com/divio/ac-wheelsproxy
Uses docker to build wheels for the different platforms.
pip install went down from 180 seconds to 15 seconds.
Demo time: reqs.in file with 15 requirements. uploads reqs.in to wheelsproxy, then pip compile runs on the server with cached data. pip installed django with 120 requirements in 40 seconds.
What about tools to see if there are newer versions.
Answer: you committed the reqs.txt, so run git-compile again and check the diff.