I am using this command to capture time taken by cURL command:
time curl -v -k http://10.164.128.232:8011/oam/server/HeartBeat >> abc.txt
This leaves abc.txt blank. I further tried this:
time curl -v -k http://10.164.128.232:8011/oam/server/HeartBeat 2>> bcde.txt
I was expecting this command to write complete console output on my text file, but it din't capture time in bcde.txt.
I am unable to find a way using which I can capture cURL's output alongside time taken by it.
Please assist me on this.
The time command may think that the redirection is part of the command being timed. In that case, you can get past it with grouping:
(time curl -v -k http://10.164.128.232:8011/oam/server/HeartBeat) >> abc.txt
(time curl -v -k http://10.164.128.232:8011/oam/server/HeartBeat) 2>> abc.txt
This worked for me!
Related
I want to make a loop to run over multiple input files and produce one output file per input file.
I can use this command to make 1 output bam files, from 1 input sam file:
samtools view -S -b -h $input_file > $output_file
where:
input_file="/scratch/RNAseq/hisat2_alignment/456.sam"
output_file="/scratch/RNAseq/BAM_files/raw_BAM/456.bam"
When making this command into a loop I am unsure of what to do with the $output_file equivalent. Because I don't know how to make the file path and file extension change required for the $many_output_file variable:
many_input_files="/scratch/RNAseq/hisat2_alignment/*.sam"
for i in $many_input_files
do
samtools view -S -b -h $i > $many_output_file
done
Can anyone help please? I am new to Bash, I usually use R. I have tried using sed and tr but they seem to come up with errors when I try to make the file list of many_output_file from many_input_files
This is how I made the loop work, thanks to the help in the comments:
for i in $input_files
do
tmp=${i/hisat2_alignment/BAM_files/raw_BAM}
samtools view -S -b -h $i > ${tmp/.sam/.bam}
done
I am currently attempting to make a script that when i enter the name of a vulnerability it will return to me the CVSS3 scores from tenable.
So far my plan is:
Curl the page
Grep the content i want
output the grepped CVSS3 score
when running myscript however grep is throwing the following error:
~/Documents/Tools/Scripts ❯ ./CVSS3-Grabber.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 30964 0 30964 0 0 28355 0 --:--:-- 0:00:01 --:--:-- 28355
grep: unrecognized option '-->Nessus<!--'
Usage: grep [OPTION]... PATTERNS [FILE]...
Try 'grep --help' for more information.
This has me very confused as when i run this in the command line i curl the content to sample.txt and then using the exact same grep syntax:
grep $pagetext -e CVSS:3.0/E:./RL:./RC:.
it returns to me the content i need, however when i run it via my script below...
#! /bin/bash
pagetext=$(curl https://www.tenable.com/plugins/nessus/64784)
cvss3_temporal=$(grep $pagetext -e CVSS:3.0/E:./RL:./RC:.)
echo $cvss3_temporal
i receive the errors above!
I believe this is because the '--' are causing grep to think the text inside the file that it is an instruction which grep doesnt know hence the error. I have tried copying the output of the curl to a text file and then grepping that rather than straight from the curl but still no joy. Does anyone know of a method to get grep to ignore '--' or any flags when reading text? Or alternatively if i can configure curl so that it only brings back text and no symbols?
Thanks in advance!
You don't need to store curl response in a variable, just pipe grep after curl like this:
cvss3_temporal=$(curl -s https://www.tenable.com/plugins/nessus/64784 |
grep -F 'CVSS:3.0/E:./RL:./RC:.')
Note use of -s in curl to suppress progress and -F in grep to make sure you are searching for a fixed string.
Grep filters a given file or standard input if none was given. In bash, you can use the <<< here-word syntax to send the variable content to grep's input:
grep -e 'CVSS:3.0/E:./RL:./RC:.' <<< "$pagetext"
Or, if you don't need the page anywhere else, you can pipe the output from curl directly to grep:
curl https://www.tenable.com/plugins/nessus/64784 | grep -e 'CVSS:3.0/E:./RL:./RC:.'
I have something like that (it replaces a special sequence in request and then it sends it with curl).
SPECIAL_SEQUENCE=My_value
sed -i -e "s|SPECIAL_SEQUENCE|$SPECIAL_SEQUENCE|g" file.txt
curl http://127.0.0.1:1478/ -X POST -d #file.txt
It does work ok. However, the problem for me is that it leaves changed the file file.txt. I could undo sed at the end of this script, but I wouldn't like do this (becase I often interrupt this script with Ctrl+C).
Can you give me some other ideas to deal with this ? In other words the final form of request can be known only during executing mentioned script.
Instead of modifying the file, modify it in the memory and pipe it to curl:
SPECIAL_SEQUENCE=My_value
sed -e "s|SPECIAL_SEQUENCE|$SPECIAL_SEQUENCE|g" file.txt | curl http://127.0.0.1:1478/ -X POST
I am using below curl command to download file from url. But the output file has new line and extra characters due to which Tiff file getting corrupt.
curl -k -u Username:Password URL >/Test.Tiff
Sample Test.Tiff has below data
1.
2.
3.IDCFILE87918
4.II*ÿûÞ©¥zKÿJÛï_]ÿÿÿ÷ÿÞï¹×ëÿ¤ÿO]
5¿ûÕÿÿ¯zê¿ß£0•¿þÛ¯kÚÿ¹5Éöûé_u_éwÕzkJï·_¯¯ßþýuw]í~þžmúºßÿzÈfçúîC7½õëÿÛ¯ô¿Z[6.ý®Úö·4ýý ~«v×ÿº^Ÿ¿í¾Ýÿzuýëÿ÷×]}ûÿõé‰ÿ¿m/KûÿµÛ_ý¾×Oín½}+wýzíýö¿õÿî—7.ékñN¿ûSߦ=ºì%±N—í¯i_Û¶¬:×·m{
8.ÿ¶ÿím¿í/ívÒ®ÒP¯Õ¥¶¿}SÛúì%Ú_kûimú«i·V½»
9..Âýt•¿ßoÛ]¦Òý´»KßØaPaa…å87M…VÂúý?ÿa„˜ei
First three lines where line no 1 and 2 is newlines which is coming as ^M through VI editor are extra which should not be there.When i delete first 3 lines and save the file then i am able to open the file.
Let me know how first three lines are getting appended.
Update: Try greping the Curl output to remove blank lines, like this:
curl -k -u <username>:<password> <url> | grep -v '^$' > /Test.Tiff
Curl also has the --output <name> option to redirect output to a file. You may first output the response to a file and then use it as grep input:
curl -k -u <username>:<password> <url> > curl_out.txt
grep -v '^$' curl_out.txt > Test.Tiff
Looking at the man page for cURL:
-w, --write-out <format>
Make curl display information on stdout after a completed transfer.
Where it is possible to use this flag and append a string to the output of cURL. However I can only get this to append to the end of the output of cURL, because as the man page suggests, the -w flag appends after a completed transfer.
so doing:
curl -sS "http:/somewebsite" -w "hello_world"
will produce:
$
contentfromcurl
hello_world
....well how do you get the output to be
$
hello_worldcontentfromcurl
i.e. is it possible to get -w to prepend rather than append?
thanks to #Adrian, this is the final answer -
curl -sS "http:/somewebsite" | xargs echo "mystring"
cheers!
If you're really desperate you can make a code block and include an echo. The following will have the output you're looking for:
{ echo -n "hello_world"; curl -sS "http:/somewebsite"; }
As for getting the -w option to prepend, the answer is no:
-w, --write-out
Make curl display information on stdout after a completed transfer. The format is a string ...
Is this what you are after?
$ printf "bar\nquux\n"
bar
quux
$ printf "bar\nquux\n" | sed 's#^#foo#g'
foobar
fooquux
Obviously, you would replace printf with your curl invocation.
But this seems a bit like an XY-problem - what are you trying to accomplish?