I have a variable equal to a string, which is a series of key/value pairs separated by newlines.
I want to then replace these newline characters with spaces, and set a new variable equal to the result
From various answers on the internet I've arrived at the following:
#test.txt has the content:
#test=example
#what=s0omething
vars="$(cat ./test.txt)"
formattedVars= $("$vars" | tr '\n' ' ')
echo "$taliskerEnvVars"
Problem is when I try to set formattedVars it tries to execute the second line:
script.sh: line 7: test=example
what=s0omething: command not found
I just want formattedVars to equal test=example what=s0omething
What trick am I missing?
Change your line to:
formattedVars=$(tr '\n' ' ' <<< "$secretsContent")
Notice the space of = in your code, which is not permitted in assignment statements.
I see that you are not setting secretsContent in your code, you are setting vars instead.
If possible, use an array to hold contents of the file:
readarray -t vars < ./test.txt # bash 4
or
# bash 3.x
declare -a vars
while IFS= read -r line; do
vars+=( "$line" )
done < ./test.txt
Then you can do what you need with the array. You can make your space-separated list with
formattedVars="${vars[*]}"
, but consider whether you need to. If the goal is to use them as a pre-command modifier, use, for instance,
"${vars[#]}" my_command arg1 arg2
Related
this is probably a very simple question. I looked at other answers but couldn't come up with a solution. I have a 365 line date file. file as below,
01-01-2000
02-01-2000
I need to read this file line by line and assign each day to a separate variable. like this,
d001=01-01-2000
d002=02-01-2000
I tried while read commands but couldn't get them to work.It takes a lot of time to shoot one by one. How can I do it quickly?
Trying to create named variable out of an associative array, is time waste and not supported de-facto. Better use this, using an associative array:
#!/bin/bash
declare -A array
while read -r line; do
printf -v key 'd%03d' $((++c))
array[$key]=$line
done < file
Output
for i in "${!array[#]}"; do echo "key=$i value=${array[$i]}"; done
key=d001 value=01-01-2000
key=d002 value=02-01-2000
Assumptions:
an array is acceptable
array index should start with 1
Sample input:
$ cat sample.dat
01-01-2000
02-01-2000
03-01-2000
04-01-2000
05-01-2000
One bash/mapfile option:
unset d # make sure variable is not currently in use
mapfile -t -O1 d < sample.dat # load each line from file into separate array location
This generates:
$ typeset -p d
declare -a d=([1]="01-01-2000" [2]="02-01-2000" [3]="03-01-2000" [4]="04-01-2000" [5]="05-01-2000")
$ for i in "${!d[#]}"; do echo "d[$i] = ${d[i]}"; done
d[1] = 01-01-2000
d[2] = 02-01-2000
d[3] = 03-01-2000
d[4] = 04-01-2000
d[5] = 05-01-2000
In OP's code, references to $d001 now become ${d[1]}.
A quick one-liner would be:
eval $(awk 'BEGIN{cnt=0}{printf "d%3.3d=\"%s\"\n",cnt,$0; cnt++}' your_file)
eval makes the shell variables known inside your script or shell. Use echo $d000 to show the first one of the newly defined variables. There should be no shell special characters (like * and $) inside your_file. Remove eval $() to see the result of the awk command. The \" quoted %s is to allow spaces in the variable values. If you don't have any spaces in your_file you can remove the \" before and after %s.
I have done some processing in perl, and got the result in perl's hash data structure. Usually in bash, when I try to retrieve result from other script like
output=$(perl -E '...')
I got the output in string. Is it possible to save the result in bash array?
Assuming a perl variable hash is an associative array, please try:
declare -A "output=($(perl -e '
$hash{"foo"} = "xx"; # just an example
$hash{"bar"} = "yy"; # ditto
for (keys %hash) {print "[\"$_\"]=\"$hash{$_}\"\n"}'))"
for i in "${!output[#]}"; do
echo "$i => ${output[$i]}" # see the result
done
The outermost double quotes around output=.. is required to tell declare
to evaluate the argument.
[Update]
Considering tripleee's comment, here is a robust version against special characters:
mapfile -d "" -t a < <(perl -e '
$hash{"baz"} = "boo"; # example
$hash{"foo"} = "x\"x"; # example with a double quote
$hash{"bar"} = "y\ny"; # example with a newline
print join("\0", %hash), "\0"') # use a nul byte as a delimiter
declare -A output # bash associative array
for ((i = 0; i < ${#a[#]}; i+=2 )); do
output[${a[i]}]=${a[i+1]} # key and value pair
done
for i in "${!output[#]}"; do
echo "$i => ${output[$i]}" # see the result
done
The conversion from perl variables to bash variables works only if they are free of null bytes (\n), as perl can store null bytes in strings, but bash cannot.
At least, we can use that limitation to print the hash in perl with null delimiters and safely parse it in bash again:
declare -A "array=($(
perl -e 'print join("\0", %hash), "\0"' |
xargs -0 printf '[%q]=%q '
))"
Please note that neither %q nor -0 are specified by posix. For a more portable solution see tshiono's answer.
If the hash is very big such that ARG_MAX might be exceeded you should ensure that xargs does not split a key value pair across two calls to printf. To do so, add the option -n2 (or any other number 2n where you are sure that n key value pairs never exceed ARG_MAX).
I wrote a bash script to read multiple inputs from the user
Here is the command:
read -a choice
In this way, I can put all the inputs in the choice variable as an array so that I can extract them using an index.
The problem is that when one of the inputs, which is a string has space in it, like
user1 google.com "login: myLogin\npassword: myPassword"
the read command will split the quoted string into 3 words. How can I stop this from happening?
bash doesn't process quotes in user input. The only thing I can think of is to use eval to execute an array assignment.
IFS= read -r input
eval "choice=($input)"
Unfortunately this is dangerous -- if the input contains executable code, it will be executed by eval.
You can use a tab instead of space as a field delimiter. For instance :
$ IFS=$'\t' read -a choice
value1 value2 a value with many words ## This is typed
$ echo ${choice[2]}
a value with many words
Regards!
Given risk of using eval, and the fact the input seems to have only two types of tokens: unquoted, and quoted, consider using scripting engine that will put all text into proper format that will be easy to read.
It's not clear from the example what other quoting rules are used. Example assume 'standard' escaped that can be processed with bash #E processor.
The following uses Perl one liner to generate TAB delimited tokens (hopefully, raw tabs can not be part of the input, but other character can be used instead).
input='user1 google.com "login: myLogin\npassword: myPassword"'
tsv_input=$(perl -e '$_ = " $ARGV[0]" ; print $2 // $3, "\t" while ( /\s+("([^"]*)"|(\S*))/g) ;' "$input")
IFS=$'\t' read -d '' id domain values <<< $(echo -e "${tsv_input#E}")
Or using a function to get more readable code
function data_to_tsv {
# Translate to TSV
local tsv_input=$(perl -e '$_ = " $ARGV[0]" ; print $2 // $3, "\t" while ( /\s+("([^"]*)"|(\S*))/g) ;' "$1")
# Process escapes
echo -n "${tsv_input#E}"
}
input='user1 google.com "login: myLogin\npassword: myPassword"'
IFS=$'\t' read -d '' id domain values <<< $(data_to_tsv "$input")
I'm writing a for loop in bash to run a command and I need to add a comma after one of my variables. I can't seem to do this without an extra space added. When I move "," right next to $bams then it outputs *.sorted,
#!/bin/bash
bams=*.sorted
for i in $bams
do echo $bams ","
done;
Output should be this:
'file1.sorted','file2.sorted','file3.sorted'
The eventual end goal is to be able to insert a list of files into a --flag in the format above. Not sure how to do that either.
First, a literal answer (if your goal were to generate a string of the form 'foo','bar','baz', rather than to run a program with a command line equivalent to somecommand --flag='foo','bar','baz', which is quite different):
shopt -s nullglob # generate a null result if no matches exist
printf -v var "'%s'," *.sorted # put list of files, each w/ a comma, in var
echo "${var%,}" # echo contents of var, with last comma removed
Or, if you don't need the literal single quotes (and if you're passing your result to another program on its command line with the single quotes being syntactic rather than literal, you absolutely don't want them):
files=( *.sorted ) # put *.sorted in an array
IFS=, # set the comma character as the field separator
somecommand --flag "${files[*]}" # run your program with the comma-separated list
try this -
lst=$( echo *.sorted | sed 's/ /,/g' ) # stack filenames with commas
echo $lst
if you really need the single-ticks around each filename, then
lst="'$( echo *.sorted | sed "s/ /','/g" )'" # commas AND quotes
#!/bin/bash
bams=*.sorted
for i in $bams
do flag+="${flag:+,}'$i'"
done
echo $flag
In Shell script I want to achieve something like below:
str="india,uk,us,uae"
I want to split it and concatenate each item as below and assign to some variable
newstr = '-myParam="india" -myParam="uk" -myParam="us" -myParam="uae"'
so that I can use above concatenated string in my next command as below
curl "admin/admin" "localhost" $newstr.
I found a way using local IFS and for loop but the variable updated inside loop is not retaining value outside of loop because it runs in a separate bash.
str="india,uk,us,uae"
var=-myparam=\"${str//,/\" -myparam=\"}\"
echo $var
Read the params into an array:
IFS=, read -a params <<< "$str"
And then loop through them and store the command in an array:
for i in "${params[#]}"; do
command+=(-myparam=\"$i\")
done
Now you can expand it using printf "${command[#]}":
$ printf "%s " "${command[#]}"
-myparam="india" -myparam="uk" -myparam="us" -myparam="uae"
That is, now you have to say:
curl "admin/admin" "localhost" "${command[#]}"
This is based on this answer by chepner: command line arguments fed from an array.
Below code would do :
$ str="india,uk,us,uae"
$ newstr=$(awk 'BEGIN{RS=","}{printf "-myParam=\"%s\" ",$1}' <<<"$str")
$ echo "$newstr"
-myParam="india" -myParam="uk" -myParam="us" -myParam="uae"
Also when you pass new string as parameter to curl, double quote it to prevent word splitting and globbing, so do :
curl "admin/admin" "localhost" "$newstr"
Note: <<< or herestring is only supported in a few shells (Bash, ksh, or zsh) if I recall correctly. If your shell does not support it use echo,pipe combination.
IFS=',' read -ra a <<< "${str//,/\",}";
curl "admin/admin" "localhost" "${a[#]/#/ -myParam=\"}\""
Explanation:
Starting with:
str="india,uk,us,uae";
Next, split the string into an array, using parameter substitution to insert " before each comma:
IFS=',' read -ra a <<< "${str//,/\",}";
Finally, we can get newstr through parameter substitution (while also appending the final "):
newstr="${a[#]/#/ -myParam=\"}\"";
newstr is now set to '-myParam="india" -myParam="uk" -myParam="us" -myParam="uae"'. We can skip the previous step and go straight to:
curl "admin/admin" "localhost" "${a[#]/#/ -myParam=\"}\""