i am having a problem with (STDIN)
i have written the following bash script to calculate the sha1 hash of a string and output it in upper-case formatted ready for copying into a plist file for example.
here is the script :-
#!/bin/bash
echo 'Enter String :-';
read temp_var1; #Read string
hash=$(echo -n $temp_var1 | openssl sha1); #Copy hash into variable $hash
uphash=$(echo $hash | tr '[:lower:]' '[:upper:]';); #Convert hash to Upper-case
echo 'Below is the UPPERCASE SHA1 Hash of your string';
echo ' </yourHash>'$uphash'</yourHash>'; #Echo formatted string
This whole script works well apart for one key problem.
the output is not what i want it to be.
for example with an input of
Hello, World!
i want to get the following output:-
Below is the UPPERCASE SHA1 Hash of your string
</yourHash>0A0A9F2A6772942557AB5355D76AF442F8F65E01</yourHash>
instead what i am getting is this :-
Below is the UPPERCASE SHA1 Hash of your string
</yourHash>(STDIN)= 0A0A9F2A6772942557AB5355D76AF442F8F65E01</yourHash>
my question is. How do i remove this (STDIN)= lable and the space that follows it?
many thanks in advance for any help you can offer.
MiRAGE
You can strip it off the front using parameter substitution like this:
a="(STDIN)= 0A0A9F2A6772942557AB5355D76AF442F8F65E01"
b=${a#*= } # Strip up to "=" sign
echo $b
0A0A9F2A6772942557AB5355D76AF442F8F65E01
I suspect that there is an option to openssl to stop it generating the "(STDIN)" part, but mine doesn't do that.
If you want to learn more about this sort of thing, have a look here.
Probably not the best solution, but you can use awk to select 2nd column. Like this:
echo -n $temp_var1 | openssl sha1 | awk '{print $2}'
I was successful with this approach (using sed) as well:
hash="(STDIN)= 0A0A9F2A6772942557AB5355D76AF442F8F65E01"
echo "before"
echo $hash
hash=$(echo $hash | sed 's/(stdin)= //g') #remove identified pattern globally
echo "after"
echo $hash
Result
before
(STDIN)= 0A0A9F2A6772942557AB5355D76AF442F8F65E01
after
0A0A9F2A6772942557AB5355D76AF442F8F65E01
Related
I am trying to write a shell script that reads the system configuration status of linux and saves the result as CSV.
The result to be saved is the hostname, setting criterion ID, and actual setting value.
The most problematic part when saving as csv is the newline.
For example,
#!/bin/bash
hostname='linux01'
id='SRV-01'
result="*result*\n"
result=$result`cat /etc/passwd | head -n 5`
echo "$hostname, $id, $value" >> test.csv
When the above script exists, the CSV format is broken due to newline.
When script result is as below
linux01, SRV-01, *result*
root:x:0:0:root:/root:/bin/zsh
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
I want to save to csv file this change.
linux01, SRV-01, *result*\nroot:x:0:0:root:/root:/bin/zsh\ndaemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\nbin:x:2:2:bin: /bin:/usr/sbin/nologin\nsys:x:3:3:sys:/dev:/usr/sbin/nologin\nsync:x:4:65534:sync:/bin:/bin/sync
First, I used sed as a test to change \n to \\n, but when outputting with echo, the \n string is changed to newline.
echo `cat /etc/passwd | head -n 5 | sed -z 's/\n/\\\n/g'`
It is saved as a \n string in the variable, but it is judged that "\n" is changed to 0x0a in echo.
Ask if there is a way to change the echo output to the string "\n" instead of newline.
You can replace the the newlines.
echo "$hostname, $id, ${result//$'\n'/\\n}" >> test.csv
I have two strings saved in a bash variable delimited by :. I want to get extract the second string, prepend that with THIS_VAR= and append it to a file named saved.txt
For example if myVar="abc:pqr", THIS_VAR=pqr should be appended to saved.txt.
This is what I have so far,
myVar="abc:pqr"
echo $myVar | cut -d ':' -f 2 >> saved.txt
How do I prepend THIS_VAR=?
printf 'THIS_VAR=%q\n' "${myVar#*:}"
See Shell Parameter Expansion and run help printf.
The more general solution in addition to #konsolebox's answer is piping into a compound statement, where you can perform arbitrary operations:
echo This is in the middle | {
echo This is first
cat
echo This is last
}
I strongly doubt about the grep best use in my code and would like to find a better and cleaner coding style for extracting the session ID and security level from my cookie file :
cat mycookie
# Netscape HTTP Cookie File
# https://curl.haxx.se/docs/http-cookies.html
# This file was generated by libcurl! Edit at your own risk.
#HttpOnly_127.0.0.1 FALSE / FALSE 0 PHPSESSID 1hjs18icittvqvpa4tm2lv9b12
#HttpOnly_127.0.0.1 FALSE /mydir/ FALSE 0 security medium
The expected output is the SSID hash :
1hjs18icittvqvpa4tm2lv9b12
Piping grep with tr '\n' '\0' works like a charm in the command line, but generates warnings (warning: command substitution: ignored null byte in input”) at the bash code execution. Here is the related code (with warnings):
ssid=$(grep -Po 'PHPSESSID.*' path/sessionFile | grep -Po '[a-z]|[0-9]' | tr '\n' '\0')
I am using bash 4.4.12 (x86_64-pc-linux-gnu) and could read here this crystal clear explanation :
Bash variables are stored as C strings. C strings are NUL-terminated.
They thus cannot store NULs by definition.
I could see here and there in both cases a coding solution using read:
# read content from stdin into array variable and a scalar variable "suffix"
array=( )
while IFS= read -r -d '' line; do
array+=( "$line" )
done < <(process that generates NUL stream here)
suffix=$line # content after last NUL, if any
# emit recorded content
printf '%s\0' "${array[#]}"; printf '%s' "$suffix"
I don't want to use arrays nor a while loop for this specific case, or others. I found this workaround using sed:
ssid=$(grep -Po 'PHPSESSID.*' path/sessionFile | grep -Po '[a-z]|[0-9]' | tr '\n' '_' | sed -e 's/_//g')
My two questions are :
1) Would it be a better way to substitute tr '\n' '\0', without using read into a while loop ?
2) Would it be a better way to extract properly the SSID and security level ?
Thx
It looks like you're trying to get rid of the newlines in the output from grep, but turning them into nulls doesn't do this. Nulls aren't visible in your terminal, but are still there and (like many other nonprinting characters) will wreak havoc if they get treated as part of your actual data. If you want to get rid of the newlines, just tell tr to delete them for you with ... | tr -d '\n'. But if you're trying to get the PHPSESSID value from a Netscape-format cookie file, there's a much much better way:
ssid=$(awk '($6 == "PHPSESSID") {print $7}' path/sessionFile)
This looks for "PHPSESSID" only in the sixth field (not in e.g. the path or cookie values -- both places it could legally appear), and specifically prints the seventh field of matching lines (not just anything after "PHPSESSID" that happens to be a digit or lowercase letter).
You could also try this, if you don't want to use awk:
ssid=$(grep -P '\bPHPSESSID\b' you_cookies_file)
echo $ssid # for debug only
which outputs something like
#HttpOnly_127.0.0.1 FALSE / FALSE 0 PHPSESSID 1hjs18icittvqvpa4tm2lv9b12
Then with cut(1) extract the relevant field:
echo $ssid |cut -d" " -f7
which outputs
1hjs18icittvqvpa4tm2lv9b12
Of course you should capture the last echo.
UPDATE
If you don't want to use cut, it is possible to emulate it with:
echo $ssid | (read a1 b2 c3 d4 e5 f6 g7; echo $g7)
Demonstration to capture in a variable:
$ field=$(echo $ssid | (read a1 b2 c3 d4 e5 f6 g7; echo $g7))
$ echo $field
1hjs18icittvqvpa4tm2lv9b12
$
Another way is to use positional parameters passing the string to a function which then refers to $7. Perhaps cleaner. Otherwise, you can use an array:
array=($(echo $ssid))
echo ${array[6]} # outputs the 7th field
It should also be possible to use regular expressions and/or string manipulation is bash, but they seem a little more difficult to me.
Say I have a string like
s1="sxfn://xfn.oxbr.ac.uk:8843/xfn/mech2?XFN=/castor/
xf.oxbr.ac.uk/prod/oxbr.ac.uk/disk/xf20.m.ac.uk/prod/v1.8/pienug_ib-2/reco_c21_dr3809_r35057.dst"
or
s2="sxfn://xfn.gla.ac.uk:8841/xfn/mech2?XFN=/castor/
xf.gla.ac.uk/space/disk1/prod/v1.8/pienug_ib-2/reco_c21_dr3809_r35057.dst"
and I want in my script to extract the last part starting from prod/ i.e. "prod/v1.8/pienug_ib-2/reco_c21_dr3809_r35057.dst". Note that $s1 contains two occurrences of "prod/".
What is the most elegant way to do this in bash?
Using BASH string manipulations you can do:
echo "prod/${s1##*prod/}"
prod/v1.8/pienug_ib-2/reco_c21_dr3809_r35057.dst
echo "prod/${s2##*prod/}"
prod/v1.8/pienug_ib-2/reco_c21_dr3809_r35057.dst
With awk (which is a little overpowered for this, but it may be helpful if you have a file full of these strings you need to parse:
echo "sxfn://xfn.gla.ac.uk:8841/xfn/mech2?XFN=/castor/xf.gla.ac.uk/space/disk1/prod/v1.8/pienug_ib-2/reco_c21_dr3809_r35057.dst" | awk -F"\/prod" '{print "/prod"$NF}'
That's splitting the string by '/prod' then printing out the '/prod' delimiter and the last token in the string ($NF)
sed can do it nicely:
s1="sxfn://xfn.oxbr.ac.uk:8843/xfn/mech2?XFN=/castor/xf.oxbr.ac.uk/prod/oxbr.ac.uk/disk/xf20.m.ac.uk/prod/v1.8/pienug_ib-2/reco_c21_dr3809_r35057.dst"
echo "$s1" | sed 's/.*\/prod/\/prod/'
this relies on the earger matching of the .* part up front.
What I am trying to achieve is pass the Base64 encoded value captured in the sed regex to the base64 and have it decoded.
But the problem is, even though it seems like the correct value is being passed to the function using backreference, base64 complains that the input is invalid.
Following is my script -
#!/bin/bash
decodeBaseSixtyFour() {
echo "$1 is decoded to `echo $1 | base64 -d`"
}
echo Passing direct value ...
echo SGVsbG8gQmFzZTY0Cg== | sed -r "s/(.+)$/$(decodeBaseSixtyFour SGVsbG8gQmFzZTY0Cg==)/"
echo Passing captured value ...
echo SGVsbG8gQmFzZTY0Cg== | sed -r "s/(.+)$/$(decodeBaseSixtyFour \\1)/"
And when ran it produces the following output -
Passing direct value ...
SGVsbG8gQmFzZTY0Cg== is decoded to Hello Base64
Passing captured value ...
base64: invalid input
SGVsbG8gQmFzZTY0Cg== is decoded to
I think the output explains what I mean.
Is it possible to do what I am trying to do? If not, why?
Perl s/// can do what you want, but I don't think what you're asking for is what you need.
$ echo SGVsbG8gQmFzZTY0Cg== | perl -MMIME::Base64 -pe 's/(.+)/decode_base64($1)/e'
Hello Base64
What's actually happening:
echo SGVsbG8gQmFzZTY0Cg== | sed -r "s/(.+)$/$(decodeBaseSixtyFour \\1)/"
Before sed starts reading input, the shell notices the process substitution in the double quoted string
the decodeBaseSixtyFour function is called with the string "\\1"
base64 chokes on the input \1 and emits the error message
the function returns the string "\1 is decoded to "
now the sed script is 's/(.+)$/\1 is decoded to /' which is how you get the last line.
As I commented sed cannot do an equivalent of replace_callback which is esentially what you're trying to do.
Following awk does close to what you're trying to do:
s="My string is SGVsbG8gQmFzZTY0Cg== something"
awk '{for(i=1; i<=NF; i++) if ($i~/==$/) "base64 -D<<<"$i|getline $i}1'<<<"$s"
My string is Hello Base64 something