I want to check if updates are available, and if so, perform some steps before I install the updates. I run yum check-updates to see a list of updates available for installed packages, but I'd like to grep this and get a count that I can use for some logic in a bash script. So ideally, I would like to grep that output of check-updates and return 0 if there are no updates, or if five updates are available then I would like the grep to return 5.
How can I grep this to return the count?
I like a simpler approach.
"yum -q" reduces the output from yum so it only displays a list of packages. Combine this with "wc -l" to count the number of lines output.
So to get a count of packages requiring updates I would run
sudo yum -q check-update | wc -l
Are you aware of grep -c? I've just created some nonsense file, giving following result:
Prompt> grep "AA" test.txt
1A01 TCCTTGAAAG
TCAACAAGAA
TCGCAAA
TTTAAAGTCGT
GGCGGAATCAATAC
GATGGAATATGCGCC
If I use grep -c, this is the result:
Prompt> grep -c "AA" test.txt
6
In case this does not answer your question completely, please edit your question and add some more information, just to show what you are looking for.
Also, please be aware that adding | wc -l behind every UNIX command reads the amount of results of that command.
This combination of awk and grep gives the count of available updates for installed packages:
yum check-updates | awk 'p;/^$/{p=1}' | grep -c "\."
This was based on the info in How to get just a list of yum updates
The -q for quiet is great but you may also want to grep out any blank lines and also the trailing Loaded plugins message. This works nicely for an accurate count:
yum check-update -q|egrep -v "Loaded plugins: langpacks, product-id, subscription-manager|^$"|wc -l
It works ok as a single tool:
curl "someURL"
curl -o - "someURL"
but it doesn't work in a pipeline:
curl "someURL" | tr -d '\n'
curl -o - "someURL" | tr -d '\n'
it returns:
(23) Failed writing body
What is the problem with piping the cURL output? How to buffer the whole cURL output and then handle it?
This happens when a piped program (e.g. grep) closes the read pipe before the previous program is finished writing the whole page.
In curl "url" | grep -qs foo, as soon as grep has what it wants it will close the read stream from curl. cURL doesn't expect this and emits the "Failed writing body" error.
A workaround is to pipe the stream through an intermediary program that always reads the whole page before feeding it to the next program.
E.g.
curl "url" | tac | tac | grep -qs foo
tac is a simple Unix program that reads the entire input page and reverses the line order (hence we run it twice). Because it has to read the whole input to find the last line, it will not output anything to grep until cURL is finished. Grep will still close the read stream when it has what it's looking for, but it will only affect tac, which doesn't emit an error.
For completeness and future searches:
It's a matter of how cURL manages the buffer, the buffer disables the output stream with the -N option.
Example:
curl -s -N "URL" | grep -q Welcome
Another possibility, if using the -o (output file) option - the destination directory does not exist.
eg. if you have -o /tmp/download/abc.txt and /tmp/download does not exist.
Hence, ensure any required directories are created/exist beforehand, use the --create-dirs option as well as -o if necessary
The server ran out of disk space, in my case.
Check for it with df -k .
I was alerted to the lack of disk space when I tried piping through tac twice, as described in one of the other answers: https://stackoverflow.com/a/28879552/336694. It showed me the error message write error: No space left on device.
You can do this instead of using -o option:
curl [url] > [file]
So it was a problem of encoding. Iconv solves the problem
curl 'http://www.multitran.ru/c/m.exe?CL=1&s=hello&l1=1' | iconv -f windows-1251 | tr -dc '[:print:]' | ...
If you are trying something similar like source <( curl -sS $url ) and getting the (23) Failed writing body error, it is because sourcing a process substitution doesn't work in bash 3.2 (the default for macOS).
Instead, you can use this workaround.
source /dev/stdin <<<"$( curl -sS $url )"
Trying the command with sudo worked for me. For example:
sudo curl -O -k 'https url here'
note: -O (this is capital o, not zero) & -k for https url.
I had the same error but from different reason. In my case I had (tmpfs) partition with only 1GB space and I was downloading big file which finally filled all memory on that partition and I got the same error as you.
I encountered the same problem when doing:
curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | apt-key add -
The above query needs to be executed using root privileges.
Writing it in following way solved the issue for me:
curl -L https://packagecloud.io/golang-migrate/migrate/gpgkey | sudo apt-key add -
If you write sudo before curl, you will get the Failed writing body error.
For me, it was permission issue. Docker run is called with a user profile but root is the user inside the container. The solution was to make curl write to /tmp since that has write permission for all users , not just root.
I used the -o option.
-o /tmp/file_to_download
In my case, I was doing:
curl <blabla> | jq | grep <blibli>
With jq . it worked: curl <blabla> | jq . | grep <blibli>
I encountered this error message while trying to install varnish cache on ubuntu. The google search landed me here for the error (23) Failed writing body, hence posting a solution that worked for me.
The bug is encountered while running the command as root curl -L https://packagecloud.io/varnishcache/varnish5/gpgkey | apt-key add -
the solution is to run apt-key add as non root
curl -L https://packagecloud.io/varnishcache/varnish5/gpgkey | apt-key add -
The explanation here by #Kaworu is great: https://stackoverflow.com/a/28879552/198219
This happens when a piped program (e.g. grep) closes the read pipe before the previous program is finished writing the whole page. cURL doesn't expect this and emits the "Failed writing body" error.
A workaround is to pipe the stream through an intermediary program that always reads the whole page before feeding it to the next program.
I believe the more correct implementation would be to use sponge, as already suggested by #nisetama in the comments:
curl "url" | sponge | grep -qs foo
I got this error trying to use jq when I didn't have jq installed. So... make sure jq is installed if you're trying to use it.
In Bash and zsh (and perhaps other shells), you can use process substitution (Bash/zsh) to create a file on the fly, and then use that as input to the next process in the pipeline chain.
For example, I was trying to parse JSON output from cURL using jq and less, but was getting the Failed writing body error.
# Note: this does NOT work
curl https://gitlab.com/api/v4/projects/ | jq | less
When I rewrote it using process substitution, it worked!
# this works!
jq "" <(curl https://gitlab.com/api/v4/projects/) | less
Note: jq uses its 2nd argument to specify an input file
Bonus: If you're using jq like me and want to keep the colorized output in less, use the following command line instead:
jq -C "" <(curl https://gitlab.com/api/v4/projects/) | less -r
(Thanks to Kowaru for their explanation of why Failed writing body was occurring. However, their solution of using tac twice didn't work for me. I also wanted to find a solution that would scale better for large files and tries to avoid the other issues noted as comments to that answer.)
I was getting curl: (23) Failed writing body . Later I noticed that I did not had sufficient space for downloading an rpm package via curl and thats the reason I was getting issue. I freed up some space and issue for resolved.
I had the same question because of my own typo mistake:
# fails because of reasons mentioned above
curl -I -fail https://www.google.com | echo $?
curl: (23) Failed writing body
# success
curl -I -fail https://www.google.com || echo $?
I added flag -s and it did the job. eg: curl -o- -s https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
I am writing a shell script where i want to ssh to a server and get the cpu and memory details data of that displayed as a result. I’m using the help of top command here.
Script line:
ssh -q user#host -n “cd; top -n 1 | egrep ‘Cpu|Mem|Swap’”
But the result is
TERM environment variable is not set.
I had checked the same in the server by entering set | grep TERM and got result as TERM=xterm
Please someone help me on this. Many thanks.
Try using the top -b flag:
ssh -q user#host -n "cd; top -bn 1 | egrep 'Cpu|Mem|Swap'"
This tells top to run non-interactively, and is intended for this sort of use.
top need an environment. You have to add the parameter -t to get the result:
ssh -t user#host -n "top -n 1 | egrep 'Cpu|Mem|Swap'"
Got it..!! Need to make a small modification for the below script line.
ssh -t user#host -n "top -n 1 | egrep 'Cpu|Mem|Swap'"
Instead of -t we need to give -tt. It worked for me.
To execute command top after ssh’ing. It requires a tty to run. Using -tt it will enable a force pseudo-tty allocation.
Thanks stony for providing me a close enough answer!! :)
I use putty to connnect to a cubietruck board which has armbian debian jessie software on it. I want to see coloured live log of an app. I followed the following example using watch , tail and ccze together.
When I use the command :
tail -f app.log | ccze
It worked great. Also when I use the command :
watch `tail -f app.log`
It also worked great. However when I gave :
watch --color 'tail -f app.log | ccze'
or
watch -c 'tail -f app.log | ccze'
I get a lot of
(B
charachter and in the text in most of the cases no new lines are recognized and looks as seamless text. I assume that the color related ASCII characters are not decoded correctly.
I also changed the putty keyboard from ESC to VT400 and Linux but the same problem occured.
Does anyone has an idea what am I doing wrong?
watch -c -n5 'tail app.log | ccze -A'
Leaving out the -f parameter for tail, to stop tail watching for changes in the log file (because watch should do that)
Adding the -A parameter to ccze to enable raw ANSI colors
I'm not quite sure what the issue is. I'm on Kali Linux 2.0 right now, fresh install. The following worked on Ubuntu 14.04 but it's not working anymore (maybe I accidentally changed it?). It looks correct to me, but every time it runs it blocks.
backup_folder=$(ssh -i /home/dexter/.ssh/id_rsa $server 'ls -t '$dir' | head -1')
This is part of a larger script. $server and $dir are set. When I run the command alone, I get the correct output, but it doesn't end the connection.
I don't know if this may help to solve the question but your command doesn't handle dirs with space in the filename. Add double quotes inside the single quote section like this:
SERVER='remoteServer' && REMOTE_DIR='remoteDir' && backup_folder=$(ssh -i /home/dexter/.ssh/id_rsa "${SERVER}" 'ls -t "'${REMOTE_DIR}'" | head -n1'); echo "${backup_folder}"
If it doesn't help try to add increasing number of -v switch to ssh to debug eventually reaching:
SERVER='remoteServer' && REMOTE_DIR='remoteDir' && backup_folder=$(ssh -vvv -i /home/dexter/.ssh/id_rsa "${SERVER}" 'ls -t "'${REMOTE_DIR}'" | head -n1'); echo "${backup_folder}"
If the verbose output does not help may be an MTU problem (these kind of problems are not of binary type, acts strangely).
You can try lowering MTU (usually 1500) on your side to solve:
sudo ifconfig eth0 mtu 1048 up
eth0 is obviously an example interface, use your own.