upgrade all outdated pip packages discarding failures [duplicate] - pip

This question already has answers here:
How to upgrade all Python packages with pip
(56 answers)
Closed 4 years ago.
I have a bash command to upgrade all pip packages that I installed.
The command is:
pip3 list --outdated | cut -d' ' -f1 | tail -n +3 | xargs pip3 install --upgrade
The problem is that if one of the packages fails to upgrade, it rolls back deleting the upgrades of the ones that were successful upgraded.
Is there a way to upgrade all outdated packages with a single command discarding the failures of some packages?

I slightly modified the command posted in the duplicate of link.
pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U --user

Related

how to use grep with logical operator AND

when searching for packages, i'm using grep, if not installed, i have to install it using shell script as mentioned below:
if
list packages installed | grep rap | grep rap-devel
## only if both are installed then exit with status 0
else
install the missing packages
How should I do this?
I tried this but its very lengthy
yum installed packages | grep rap
if $? = 1
yum install rap
yum installed packages | grep rap-devel
if $? = 1
yum install rap-devel
And I dont think its a good practice to do like this. Can anyone help me to shorten this? what if there still more to search and install 🥲
use && and || operate:
(yum installed packages | grep rap || yum install rap) && (yum installed packages | grep rap-devel || yum install rap-devel)
for more search packages in the list:
pack_list=(rap rap-devel)
for i in "${pack_list[#]}"; do yum installed packages | grep rap || yum install rap; done
or if it could, you could use which rap instead of yum installed packages | grep rap.

grep output to pip install

How to take a subset of python dependencies in requirements.txt and using grep send them to pip install?
Let's say I want to install redis and gunicorn only: with
cat requirements.txt | grep "redis\|gunicorn"
I get only the dependencies I want,
redis>=3.5.3
gunicorn>=20.1.0
but I would like to pass it as requirement file to pip install.
I guess I should create a temp file with the output of grep and do something like pip install -r tempfile, but I don't understand how to do it.
Could anyone help me?
You can use xargs to pass the outputs to pip:
grep "redis\|gunicorn" requirements.txt | xargs pip install

Automatically update outdated pip3 packages with bash scripts

I have a bash script to automatically update pip3 packages:
It is one line script:
pip3 list --outdated | cut -d' ' -f1 | xargs pip3 install --upgrade
and it has always worked.
Now (maybe after the upgrade to Ubuntu 18.04) it is not working anymore apparently because of a wrong usage of pip3 install.
What is wrong with the script?
Looks like headers of the pip3 list are causing failure in installation of package.
You can trim those header lines by using tail.
pip3 list --outdated | cut -d' ' -f1 | tail -n+3 | xargs pip3 install --upgrade
tail -n+3 removes the header and boundary line which only gives package names to xargs.

Get size of each installed formula in homebrew? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
This post was edited and submitted for review 1 year ago and failed to reopen the post:
Original close reason(s) were not resolved
Improve this question
I would like to find the size of my installed brew packages so I can see how much space each is taking up. How do I go about doing this from the terminal?
Note:
some of the answers below also include brew cask packages which is also very helpful to this answer.
riffing off of PTao's solution, here's an ugly improved version which will include a human-readable sum of the combined sized of all package versions
for pkg in `brew list --formula -1 | egrep -v '\.|\.\.'`
do echo $pkg `brew info $pkg | egrep '[0-9]* files, ' | sed 's/^.*[0-9]* files, \(.*\)).*$/\1/' | awk '{print $1;}/[0-9]$/{s+=$1};/[mM][bB]$/{s+=$1*(1024*1024);next};/[kK][bB]$/{s+=$1*1024;next} END { suffix=" KMGT"; for(i=1; s>1024 && i < length(suffix); i++) s/=1024; printf "\t(all versions: %0.1f%s)",s,substr(suffix, i, 1), $3; }'`
done
example output:
ack 173.5KB 178.4KB 182.7KB 190.5KB (all versions: 725.1K)
afl-fuzz 556.5KB 561.3KB (all versions: 1.1M)
aircrack-ng 934.2KB 953KB (all versions: 1.8M)
autoconf 1.9MB (all versions: 1.9M)
autojump 325.4KB (all versions: 325.4K)
automake 2.9MB 3.0MB 3MB (all versions: 8.9M)
bash-completion 608.6KB (all versions: 608.6K)
boost 414.6MB 398.7MB (all versions: 813.3M)
cairo 5.9MB 5.9MB (all versions: 11.8M)
cask 166.6KB (all versions: 166.6K)
cmake 31.4MB (all versions: 31.4M)
coreutils 8.5MB 7.9MB 9MB (all versions: 25.4M)
curl 3MB (all versions: 3.0M)
dos2unix 344.4KB 360.5KB (all versions: 704.9K)
ebook-tools 69.6KB 70.5KB (all versions: 140.1K)
eigen 3.5MB 6.5MB (all versions: 10.0M)
brew list --formula | xargs -n1 -P8 -I {} \
sh -c "brew info {} | egrep '[0-9]* files, ' | sed 's/^.*[0-9]* files, \(.*\)).*$/{} \1/'" | \
sort -h -r -k2 - | column -t
It display brews like below in sorted form (by size):
It's not exceedingly pretty, but you can do
$ brew list --formula | xargs brew info
And it will output something along the lines of
...
/usr/local/Cellar/ant/1.9.6 (1,611 files, 34.8M)
Poured from bottle on 2016-03-31 at 09:35:41
/usr/local/Cellar/ant/1.9.7 (1,611 files, 34.9M) *
Poured from bottle on 2016-12-15 at 09:58:56
..
for each package you have installed. I'm sure some wizard with grep could make this give you a nice table if you have many taps installed.
Colored list of all brew packages with size (and overall size of all versions)
for pkg in `brew list -f1 | egrep -v '\.|\.\.'`
do echo $pkg `brew info $pkg | egrep '[0-9]* files, ' | sed 's/^.*[0-9]* files, \(.*\)).*$/\1/' | awk '{print "\033[1;32m"$1;}/[0-9]$/{s+=$1};/[mM][bB]$/{s+=$1*(1024*1024);next};/[kK][bB]$/{s+=$1*1024;next} END { suffix=" KBMBGBTB"; for(i=1; s>1024 && i < length(suffix); i+=2) s/=1024; printf "\033[0m\t(all versions: \033[33m%0.1f %s\033[0m)",s,substr(suffix, i, 2), $3; }'`
done
Output looks like this:
This shell snippet from https://gist.github.com/eguven/23d8c9fc78856bd20f65f8bcf03e691b will help you
for pkg in `brew list -f1 | egrep -v '\.|\.\.'`
do echo $pkg `brew info $pkg | egrep '[0-9]* files, ' | sed 's/^.*[0-9]* files, \(.*\)).*$/\1/'`
done
On version Homebrew 2.7.5
Following command throw an error
brew list | xargs brew info | egrep --color '\d*\.\d*(KB|MB|GB)'
Error: Calling brew list to only list formulae is disabled! Use brew list --formula instead.
After adding --formula to brew list command it worked.
brew list --formula | xargs brew info | egrep --color '\d*\.\d*(KB|MB|GB)'

Hide "Requirement already satisfied" warning

We have a shell script that automatically prepares virtual environment and then runs tests in it. Part of the script installs requirements:
pip install -r requirements.txt
When the script is run multiple times it prints a warning for each requirement:
Requirement already satisfied (use --upgrade to upgrade): discover==0.4.0
...
I need to run the installation step every time in case that someone adds a new requirement. I understand why the warning is displayed. The problem is that it clutters the test output.
Is there a way how to disable/hide this warning?
It worked for me:
pip install -r requirements.txt | grep -v 'already satisfied'
Assuming the errors go to stderr, this bash code should do it:
pip install -r requirements.txt 2> >(grep -v 'Requirement already satisfied' 1>&2)
For more recent versions of 'pip', where errors go to stdout, the above can be simplified to:
pip install -r requirements.txt | grep -v 'already satisfied'
For Windows users landing here this can be resolved in a virtually similar manner using Find
pip install -r requirements.txt | find /V "already satisfied"
and as answered in a Windows specific question the other way is
pip install -r requirements.txt | findstr /V /C:"Requirement already satisfied"

Resources