i would like to create a bash script that creates the md5-hash of a string.
BUT the string can contain special characters and spaces. How can i get it that such a string is readable for eg. md5sum?
I have written the following script as "md5.sh":
#!/bin/bash
echo -n $1 | md5sum | awk '{print $1}'
But if i use "./md5.sh " (with a space at the end) this will not be recognized.
Also quoted cannot be read from the script. And a single " will end in an prompt...
Hope someone can help me :)
First, you need to quote the parameter in the script (and use printf; the world would be a better place if people forgot echo existed):
printf '%s' "$1" | md5sum | awk '{print $1}'
Second, to pass an actual space as the argument, it must be quoted as well so that the shell doesn't discard it:
./md5.sh " "
Related
I found this thread with two solutions for trimming whitespace: piping to xargs and defining a trim() function:
trim() {
local var="$*"
# remove leading whitespace characters
var="${var#"${var%%[![:space:]]*}"}"
# remove trailing whitespace characters
var="${var%"${var##*[![:space:]]}"}"
echo -n "$var"
}
I prefer the second because of one comment:
This is overwhelmingly the ideal solution. Forking one or more external processes merely to trim whitespace from a single string is fundamentally insane – particularly when most shells (including bash) already provide native string munging facilities out-of-the-box.
I am getting, for example, the wifi SSID on macOS by piping to awk (when I get comfortable with regular expressions in bash, I won't fork an awk process), which includes a leading space:
$ /System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -I | awk -F: '/ SSID/{print $2}'
<some-ssid>
$ /System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -I | awk -F: '/ SSID/{print $2}' | xargs
<some-ssid>
$ /System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -I | awk -F: '/ SSID/{print $2}' | trim
$ wifi=$(/System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -I | awk -F: '/ SSID/{print $2}')
$ trim "$wifi"
<some-ssid>
Why does piping to the trim function fail and giving it an argument work?
It is because your trim() function is expecting a positional argument list to process. The $* is the argument list passed to your function. For the case that you report as not working, you are connecting the read end of a pipe to the function inside which you need to fetch from the standard input file descriptor.
In such a case you need to read from standard input using read command and process the argument list, i.e. as
trim() {
# convert the input received over pipe to a a single string
IFS= read -r var
# remove leading whitespace characters
var="${var#"${var%%[![:space:]]*}"}"
# remove trailing whitespace characters
var="${var%"${var##*[![:space:]]}"}"
echo -n "$var"
}
for which you can now do
$ echo " abc " | trim
abc
or using a command substitution syntax to run the command that fetches the string, that you want to pass to trim() with your older definition.
trim "$(/System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport -I | awk -F: '/ SSID/{print $2}')"
In this case, the shell expands the $(..) by running the command inside and replaces it with output of the commands run. So now the function sees trim <args> which it interprets as a positional argument and runs the string replacement functions directly on it.
I have a variable called $folder_name which contains the string
Release_2019_Config_V6_Standalone_PJ6678_Test
which is the name of a folder.
I'm trying to extract PJ6678 from the folder name.
I know the folder name will put the user id (the text I need) between the 5th and 6th underscore, I don't know what text/symbols will be present after the 6th underscore.
I'm using Bash script, i'd really appreciate the help if someone could help with this functionality as i'm completely lost trying to use sed (after reading for hours i'm assuming this is the correct tool for the job?
Here is a Bash only solution:
#!/bin/bash
INPUT="Release_2019_Config_V6_Standalone_PJ6678_Test"
IFS='_' read -ra IN <<< "$INPUT"
echo ${IN[5]}
Or use cut:
cut -d '_' -f 6 <<< "Release_2019_Config_V6_Standalone_PJ6678_Test"
Or use awk:
awk -F "_" '{ print $6 }' <<< "Release_2019_Config_V6_Standalone_PJ6678_Test"
If you want pure-bash solution, you can use tokenize the file name, and pick up the 5th element
IFS=_ read -a token <<< "$folder_name"
id=${token[5]}
Eliminating dependency and performance hit from launching additional programs per folder name.
Try this command:
echo $a | awk -F'_' '{print $6}'
Here, _ is the delimiter and $a is a variable that holds the value.
For completeness, here's a pure-shell solution that doesn't rely on bash extensions like arrays.
$ folder_name=Release_2019_Config_V6_Standalone_PJ6678_Test
$ tmp=${folder_name#*_*_*_*_*_} # Because we know how many _ to strip
$ echo ${tmp%_*}
PJ6678
Because the # operator strips the shortest prefix, this won't allow * to match any _ itself; if it did, we could shorten the prefix by making the underscore match one of the literal _ in the pattern instead.
Hi I'm using GNU awk version 3.1.5 and I've specified 2 variables in a KSH script
PKNAME= ls -lt /var/db/pkg | tr -s " " | cut -d" " -f9
PKDATE= ls -lt /var/db/pkg/$PKNAME/ | tr -s " " | cut -d" " -f6-8
I'm trying to prove that I'm getting the correct output, by running a test using
echo bar
awk -F, -v pkname="$PKNAME" -v pkdate="$PKDATE" 'BEGIN{print pkname, pkdate, "foo"; exit}'
echo baz
The output from this results in 2 blank spaces and foo, like so
bar
foo
baz
I have tried, double quoting the variables, single quotes and back ticks. Also tried double quotes with back ticks.
Any ideas why the variables are not being executed? I'm fairly new to awk and appreciate any help! Thanks
I suppose it is possible that it is not possible to run a sub shell comand within an awk statement. Is this true?
This has nothing to do with awk. The problem is in the way you're assigning your variables. Your lines should be like this:
PKNAME=$(ls -lt /var/db/pkg | tr -s " " | cut -d" " -f9)
There can be no spaces around either side of an assignment in the shell.
At the moment, you're running the command ls -lt ... with a variable PKNAME temporarily assigned to an empty string. In subsequent commands the variable remains unset.
Your awk command should remain unchanged, i.e. the shell variables should be passed like -v pkname="$PKNAME". As an aside, it's generally considered bad practice to use uppercase variable names, as these should be reserved for internal use by the shell.
Suppose echo $PATH yields /first/dir:/second/dir:/third/dir.
Question: How does one echo the contents of $PATH one directory at a time as in:
$ newcommand $PATH
/first/dir
/second/dir
/third/dir
Preferably, I'm trying to figure out how to do this with a for loop that issues one instance of echo per instance of a directory in $PATH.
echo "$PATH" | tr ':' '\n'
Should do the trick. This will simply take the output of echo "$PATH" and replaces any colon with a newline delimiter.
Note that the quotation marks around $PATH prevents the collapsing of multiple successive spaces in the output of $PATH while still outputting the content of the variable.
As an additional option (and in case you need the entries in an array for some other purpose) you can do this with a custom IFS and read -a:
IFS=: read -r -a patharr <<<"$PATH"
printf %s\\n "${patharr[#]}"
Or since the question asks for a version with a for loop:
for dir in "${patharr[#]}"; do
echo "$dir"
done
How about this:
echo "$PATH" | sed -e 's/:/\n/g'
(See sed's s command; sed -e 'y/:/\n/' will also work, and is equivalent to the tr ":" "\n" from some other answers.)
It's preferable not to complicate things unless absolutely necessary: a for loop is not needed here. There are other ways to execute a command for each entry in the list, more in line with the Unix Philosophy:
This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.
such as:
echo "$PATH" | sed -e 's/:/\n/g' | xargs -n 1 echo
This is functionally equivalent to a for-loop iterating over the PATH elements, executing that last echo command for each element. The -n 1 tells xargs to supply only 1 argument to it's command; without it we would get the same output as echo "$PATH" | sed -e 'y/:/ /'.
Since this uses xargs, which has built-in support to split the input, and echoes the input if no command is given, we can write that as:
echo -n "$PATH" | xargs -d ':' -n 1
The -d ':' tells xargs to use : to separate it's input rather than a newline, and the -n tells /bin/echo to not write a newline, otherwise we end up with a blank trailing line.
here is another shorter one:
echo -e ${PATH//:/\\n}
You can use tr (translate) to replace the colons (:) with newlines (\n), and then iterate over that in a for loop.
directories=$(echo $PATH | tr ":" "\n")
for directory in $directories
do
echo $directory
done
My idea is to use echo and awk.
echo $PATH | awk 'BEGIN {FS=":"} {for (i=0; i<=NF; i++) print $i}'
EDIT
This command is better than my former idea.
echo "$PATH" | awk 'BEGIN {FS=":"; OFS="\n"} {$1=$1; print $0}'
If you can guarantee that PATH does not contain embedded spaces, you can:
for dir in ${PATH//:/ }; do
echo $dir
done
If there are embedded spaces, this will fail badly.
# preserve the existing internal field separator
OLD_IFS=${IFS}
# define the internal field separator to be a colon
IFS=":"
# do what you need to do with $PATH
for DIRECTORY in ${PATH}
do
echo ${DIRECTORY}
done
# restore the original internal field separator
IFS=${OLD_IFS}
I have two strings and I want to split with space and use them two by two:
namespaces="Calc Fs"
files="calc.hpp fs.hpp"
for example, I want to use like this: command -q namespace[i] -l files[j]
I'm a noob in Bourne Shell.
Put them into an array like so:
#!/bin/bash
namespaces="Calc Fs"
files="calc.hpp fs.hpp"
i=1
j=0
name_arr=( $namespaces )
file_arr=( $files )
command -q "${name_arr[i]}" -l "${file_arr[j]}"
echo "hello world" | awk '{split($0, array, " ")} END{print array[2]}'
is how you would split a simple string.
if what you want to do is loop through combinations of the two split strings, then you want something like this:
for namespace in $namespaces
do
for file in $files
do
command -q $namespace -l $file
done
done
EDIT:
or to expand on the awk solution that was posted, you could also just do:
echo $foo | awk '{print $'$i'}'
EDIT 2:
Disclaimer: I don not profess to be any kind of expert in awk at all, so there may be small errors in this explanation.
Basically what the snippet above does is pipe the contents of $foo into the standard input of awk. Awk reads from it's standard in line by line, separating each line into fields based on a field separator, which is any number of spaces by default. Awk executes the program that it is given as an argument. In this case, the shell expands '{ print $'$1' }' into { print $1 } which simply tells awk to print field number 1 of each line of its input.
If you want to learn more I think that this blog post does a pretty good job of describing the basics (as well as the basics of sed and grep) if you skip past the more theoretical stuff at the start (unless you're into that kind of thing).
I wanted to find a way to do it without arrays, here it is:
paste -d " " <(tr " " "\n" <<< $namespaces) <(tr " " "\n" <<< $files) |
while read namespace file; do
command -q $namespace -l $file
done
Two special usage here: process substitution (<(...)) and here strings (<<<). Here strings are a shortcut for echo $namespaces | tr " " "\n". Process substitution is a shortcut for fifo creation, it allows paste to be run using the output of commands instead of files.
If you are using zsh this could be very easy:
files="calc.hpp fs.hpp"
# all elements
print -l ${(s/ /)files}
# just the first one
echo ${${(s/ /)files}[1]} # just the first one