I'm learning Bash and am looking at the read command. I thought the difference between the -N and the -n option was that -N would overwrite the IFS while -n wouldn't. In the following example I expected var6 to take the value of "ijfz", because I thought the space would act as field separator.
But it seems to have value "ijfz e". The space wasn't used as field separator
printf "%s\n" "ijfz eszev enacht" | {
read -n 6 var6
printf "%s\n" "$var6"
}
I wanted to see what was $IFS, but the following printf command doesn't learn me too much:
printf ":%s:\n" "$IFS"
gives the following output
:
:
What am I not understanding...?
By delimiter it means using option -d. See this difference:
printf "%s\n" "ijfz eszev enacht" | { read -d ' ' -N 6 var6; printf "[%s]\n" "$var6"; }
[ijfz e]
printf "%s\n" "ijfz eszev enacht" | { read -d ' ' -n 6 var6; printf "[%s]\n" "$var6"; }
[ijfz]
In first case when using -N it ignores -d ' ' and reads exactly 6 characters.
In second case when using -n it respects -d ' ' and reads until a space is read.
Related
I have a file with tab-separated-values and also with blank spaces like this:
! (desambiguación) http://es.dbpedia.org/resource/!_(desambiguación) 5
! (álbum) http://es.dbpedia.org/resource/!_(álbum_de_Trippie_Redd) 2
!! http://es.dbpedia.org/resource/!! 4
$9.99 http://es.dbpedia.org/resource/$9.99 6
Tomlinson http://es.dbpedia.org/resource/(10108)_Tomlinson 20
102 Miriam http://es.dbpedia.org/resource/(102)_Miriam 2
2003 QQ47 http://es.dbpedia.org/resource/(143649)_2003_QQ47 2
I want to extract the last number of every line:
5
2
4
6
20
2
2
For that, I have done this:
while read line;
do
NUMBER=$(echo $line | cut -f 3 -d ' ')
echo $NUMBER
done < $PAIRCOUNTS_FILE
The main problem is that some lines have more spaces than others and cut doesn't work for me with default delimiter (tab). I dont' know why, maybe because I am using WSL.
I have tried cut with several options but it doesn't work in anyway:
NUMBER=$(echo $line | cut -f 3 -d ' ')
NUMBER=$(echo $line | cut -f 4 -d ' ')
NUMBER=$(echo $line | cut -f 2)
NUMBER=$(echo $line | cut -f 3)
Hope you can help me with this. Thanks in advance.
I want to extract the last number of every line:
You could use grep
grep -Eo '[[:digit:]]+$' file
Or mapfile aka readarray which is a bash4+ feature.
mapfile -t array < file
printf '%s\n' "${array[#]##* }"
You can use awk:
awk '{print $NF}' file
With cut (if it is truly TAB separated and 3 fields per line):
cat file | cut -f3
If you have some variable number of fields per line, use rev|cut|rev to get the last field:
cat file | rev | cut -f1 | rev
Or with pure Bash and parameter expansion:
while IFS= read -r line; do
last=${line##* } # that is a literal TAB in the parameter expansion
printf "%s\n" "$last";
done <file
Or, read into a bash array and echo the last field:
while IFS=$'\t' read -r -a arr; do
echo "${arr[${#arr[#]}-1]}"
done <file
If you have a mixture of tabs and spaces you can do what usually is a mistake and break a Bash variable on white spaces in general (tabs and spaces) into an array:
while IFS= read -r line; do
arr=($line) # break on either tab or space without quotes
echo "${arr[${#arr[#]}-1]}"
done <file
I have a file (file.env) similar to this:
kw_var1='string1'
kw_var2='string 2'
kw_var3='this is string 3'
kw_var4='does not matter'
kw_var5='maybe'
w_var1=qwert_1
w_var2=qwert_2
w_var3=qwert_3
w_var4=qwert_4
and I need to create a string list_of_values which contains the values of all variables that start with kw_, i.e.
$ echo -e $list_of_values
should output:
'string1' 'string 2' 'this is string 3' 'does not matter' 'maybe'
I tried to iterate over them, but cannot get this to work. My code:
list_of_values=$(for param in $(cat $file.env | grep "kw\_"); do echo $(echo -e '$'$param | cut -s -d '=' -f1); done)
but this is what I get:
$kw_var1 $kw_var2 $kw_var3 $kw_var4 $kw_var5
Note that:
the variable values will contain spaces;
list_of_values will be used as an argument to another function
Any ideas with what is wrong?
UPDATE:
When doing the final echo I used:
$ echo -e $list_of_values | tr '\n' ' '
to get everything in one line.
Trying your code
I tried your command and get this as output :
$kw_var1
$kw_var2
$kw_var3
$kw_var4
$kw_var5
You had the wrong output because you chose the first field when you used cut instead of the second.
Fixing cut command
for param in $(cat test.txt | grep "kw\_"); do echo $(echo '$'$param | cut -s -d '=' -f2); done
Returns :
'string1'
'string
'this
'does
'maybe'
Fixing IFS
You used a for in loop but it does not iterate over newlines, it iterates over spaces. You need to change the IFS (Internal Field Separator) variable first :
IFS=$'\n'; for param in $(cat <file> | grep "kw\_"); do echo $(echo $param | cut -s -d '=' -f2); done
Output :
'string1'
'string 2'
'this is string 3'
'does not matter'
'maybe'
Using printf
To get the output on one line, you can use printf instead of echo :
for param in $(cat <file> | grep "kw\_"); do printf "$(echo $param | cut -s -d '=' -f2) "; done; printf "\n"
Output :
'string1' 'string 2' 'this is string 3' 'does not matter' 'maybe'
Using while
You could simplify the command and use a while read statement that iterates directly over lines :
cat <file> | grep "kw\_" | cut -d"=" -f2 | while read line; do printf "${line} "; done; printf "\n"
Using awk
Last but not least, you can use awk which radically simplifies your code:
awk -F"=" '/kw_/{printf "%s ", $2}END{print ""}' <file>
Output :
'string1' 'string 2' 'this is string 3' 'does not matter' 'maybe'
If the extra space at the end of the line is annoying, you can do this :
awk -F"=" '/kw_/{printf "%s%s", delim, $2; delim=" "}END{print ""}' <file>
Awk explained :
# Using = as delimiter
awk -F"=" '
# If line contains kw_
/kw_/{
# Prints second field
printf "%s%s", delim, $2;
delim=" "
}
END{
# Prints newline
print ""
}' <file>
Final code
list_of_values=$(awk -F"=" '/kw_/{printf "%s%s", delim, $2; delim=" "}END{print ""}' $file.env)
$ cat tst.awk
/^kw_/ {
sub(/[^=]+=/,"")
str = str sep $0
sep = " "
}
END {
print str
}
e.g. note that it handles this=that in your desired output string correctly:
$ cat file
kw_var1='string1'
kw_var2='string 2'
kw_var3='this is string 3'
kw_var4='does not matter'
kw_var5='maybe'
kw_var6='this=that'
w_var1=qwert_1
w_var2=qwert_2
w_var3=qwert_3
w_var4=qwert_4
$ awk -f tst.awk file
'string1' 'string 2' 'this is string 3' 'does not matter' 'maybe' 'this=that'
Updated: given what you've now told us in comments, here's how I'd do it assuming you need to access individual values by their tags sometimes, otherwise you could use a regular array instead of associative:
$ cat tst.sh
#!/bin/env bash
declare -A kw
declare -A w
while IFS= read -r line; do
tag="${line%%=*}"
val="${line#*=}"
case "$tag" in
kw* ) kw["$tag"]="$val" ;;
w* ) w["$tag"]="$val" ;;
?* ) printf 'Error: unexpected contents: "%s"\n' "$line"; exit 1;;
esac
done < file.env
printf '\nAll kw indices => values:\n'
for idx in "${!kw[#]}"; do
printf '\t%s => %s\n' "$idx" "${kw[$idx]}"
done
printf '\nAll kw values passed to a function (printf) at once:\n'
printf '\t%s\n' "${kw[#]}"
printf '\nAll w indices => values:\n'
for idx in "${!w[#]}"; do
printf '\t%s => %s\n' "$idx" "${w[$idx]}"
done
printf '\nAll w values passed to a function (printf) at once:\n'
printf '\t%s\n' "${w[#]}"
.
$ ./tst.sh
All kw indices => values:
kw_var4 => does not matter
kw_var5 => maybe
kw_var6 => this=that
kw_var1 => string1
kw_var2 => string 2
kw_var3 => this is string 3
All kw values passed to a function (printf) at once:
does not matter
maybe
this=that
string1
string 2
this is string 3
All w indices => values:
w_var3 => qwert_3
w_var2 => qwert_2
w_var1 => qwert_1
w_var4 => qwert_4
All w values passed to a function (printf) at once:
qwert_3
qwert_2
qwert_1
qwert_4
The above was run on this file.env without the redundant single quotes around the values, otherwise you'd just remove them in the script:
$ cat file.env
kw_var1=string1
kw_var2=string 2
kw_var3=this is string 3
kw_var4=does not matter
kw_var5=maybe
kw_var6=this=that
w_var1=qwert_1
w_var2=qwert_2
w_var3=qwert_3
w_var4=qwert_4
wrt our discussion in the comments and using printf '<%s>\n' in place of fitsort which I don't know and don't have:
$ list[0]='foo bar'; list[1]='etc'
$ printf '<%s>\n' "${list[#]}"
<foo bar>
<etc>
$ printf '<%s>\n' $(printf '%s\n' "${list[#]}")
<foo>
<bar>
<etc>
$ printf '<%s>\n' "$(printf '%s\n' "${list[#]}")"
<foo bar
etc>
See how the first version correctly simply passes the contents of list[] to the fitsort-replacement command while the others pass the strings output by printf to it instead?
Make two arrays of your bunch of variables, then you can easily iterate over them like this
#!/bin/bash
kw=(
'string1'
'string 2'
'this is string 3'
'does not matter'
'maybe'
)
w=(
'qwert_1'
'qwert_2'
'qwert_3'
'qwert_4'
)
for i in {1..5}
do
echo -n "\"${kw[$i]}\" "
done
echo
for i in {1..4}
do
echo -n "\"${w[$i]}\" "
done
echo
I used dynamic references.
$: out="$( . file.env; for r in ${!kw_*}; do printf "'%s' " "${!r}"; done; echo )"
$: echo "$out"
'string1' 'string 2' 'this is string 3' 'does not matter' 'maybe'
Your attempt uses a few practices that aren't recommended, and contains a few syntax errors:
You use $file.env, but it should be just file.env
Don't use for to read lines from a file (see this article)
echo $(cmd) is the same as just cmd plus globbing and word splitting, which often isn't what you want
echo -e '$'$param is going to print a literal $ sign
cut -f1 is selecting the first field, but you want the second one
This is a solution "in the spirit" of what you tried, but using just Bash:
list=$(
while IFS='=' read -r key val; do
[[ $key == kw_* ]] && printf '%s ' "$val"
done < file.env
)
list=${list% } # Remove trailing blank
If you deal with strings containing spaces, though, it's generally advisable to use an array instead. Since file.env is valid Bash, we can source the lines we're interested in and then build an array with the values of all the kw_ variables:
source <(grep '^kw_' file.env)
declare -n var
for var in "${!kw_#}"; do list+=("$var"); done
The array now contains one string per element, without the literal single quotes:
$ printf '%s\n' "${list[#]}"
string1
string 2
this is string 3
does not matter
maybe
declare -n sets var to a nameref: it is treated as if it actually were the variable whose name it holds. This requires Bash 4.3 or newer.
I have the below text file
1
2
3
4
5
I want the file contents to be in the below way
1,2,3,4,5
How can I go with this?
I have the data file below:
DATE: 2015-11-30
TIME: 13:15:00
Charge Remaining (mAh): 4828
Full Charge Capacity (mAh): 5634
I want the data arranged in this order-
Date,Time,'Charge Remaining','Full Charge Capacity'
So, the output should be:
2015-11-30,13:15:00,4828,5634
tr '\n' ',' < text-file | sed 's/,$//'
Replace all newlines by commas; replace the final comma by 'nothing' (actually, a newline).
Dealing with the more complex data, you want the last field on each line as a field in the CSV-style output. That's easy with awk plus the script from before:
awk '{ print $NF }' text-file | tr '\n' ',' | sed 's/,$//'
You can use a short shell script to read the data file and output the comma separated values. You can redirect the output to a new file if you choose:
#!/bin/bash
declare -i cnt=0;
while read -r line; do
[ "$cnt" -eq 0 ] && printf "%s" "${line##*: }" || printf ",%s" "${line##*: }"
((cnt++))
done < "$1"
printf "\n"
Input
$ cat dat/charge.txt
DATE: 2015-11-30
TIME: 13:15:00
Charge Remaining (mAh): 4828
Full Charge Capacity (mAh): 5634
Output
$ bash rfmtcharge.sh dat/charge.txt
2015-11-30,13:15:00,4828,5634
If you wanted it in a one-line script, you could use:
( declare -i cnt=0; while read -r line; do [ "$cnt" -eq 0 ] && printf "%s" "${line##*: }" || printf ",%s" "${line##*: }"; ((cnt++)); done < dat/charge.txt; printf "\n" )
note: the one-line script is wrapped in ( ... ) to execute in a subshell.
This is probably pretty basic, I want to read in a occurrence file.
Then the program should find all occurrences of "CallTilEdb" in the file Hendelse.logg:
CallTilEdb 8
CallCustomer 9
CallTilEdb 4
CustomerChk 10
CustomerChk 15
CallTilEdb 16
and sum up then right column. For this case it would be 8 + 4 + 16, so the output I would want would be 28.
I'm not sure how to do this, and this is as far as I have gotten with vistid.sh:
#!/bin/bash
declare -t filename=hendelse.logg
declare -t occurance="$1"
declare -i sumTime=0
while read -r line
do
if [ "$occurance" = $(cut -f1 line) ] #line 10
then
sumTime+=$(cut -f2 line)
fi
done < "$filename"
so the execution in terminal would be
vistid.sh CallTilEdb
but the error I get now is:
/home/user/bin/vistid.sh: line 10: [: unary operator expected
You have a nice approach, but maybe you could use awk to do the same thing... quite faster!
$ awk -v par="CallTilEdb" '$1==par {sum+=$2} END {print sum+0}' hendelse.logg
28
It may look a bit weird if you haven't used awk so far, but here is what it does:
-v par="CallTilEdb" provide an argument to awk, so that we can use par as a variable in the script. You could also do -v par="$1" if you want to use a variable provided to the script as parameter.
$1==par {sum+=$2} this means: if the first field is the same as the content of the variable par, then add the second column's value into the counter sum.
END {print sum+0} this means: once you are done from processing the file, print the content of sum. The +0 makes awk print 0 in case sum was not set... that is, if nothing was found.
In case you really want to make it with bash, you can use read with two parameters, so that you don't have to make use of cut to handle the values, together with some arithmetic operations to sum the values:
#!/bin/bash
declare -t filename=hendelse.logg
declare -t occurance="$1"
declare -i sumTime=0
while read -r name value # read both values with -r for safety
do
if [ "$occurance" == "$name" ]; then # string comparison
((sumTime+=$value)) # sum
fi
done < "$filename"
echo "sum: $sumTime"
So that it works like this:
$ ./vistid.sh CallTilEdb
sum: 28
$ ./vistid.sh CustomerChk
sum: 25
first of all you need to change the way you call cut:
$( echo $line | cut -f1 )
in line 10 you miss the evaluation:
if [ "$occurance" = $( echo $line | cut -f1 ) ]
you can then sum by doing:
sumTime=$[ $sumTime + $( echo $line | cut -f2 ) ]
But you can also use a different approach and put the line values in an array, the final script will look like:
#!/bin/bash
declare -t filename=prova
declare -t occurance="$1"
declare -i sumTime=0
while read -a line
do
if [ "$occurance" = ${line[0]} ]
then
sumTime=$[ $sumtime + ${line[1]} ]
fi
done < "$filename"
echo $sumTime
For the reference,
id="CallTilEdb"
file="Hendelse.logg"
sum=$(echo "0 $(sed -n "s/^$id[^0-9]*\([0-9]*\)/\1 +/p" < "$file") p" | dc)
echo SUM: $sum
prints
SUM: 28
the sed extract numbers from a lines containing the given id, such CallTilEdb
and prints them in the format number +
the echo prepares a string such 0 8 + 16 + 4 + p what is calculation in RPN format
the dc do the calculation
another variant:
sum=$(sed -n "s/^$id[^0-9]*\([0-9]*\)/\1/p" < "$file" | paste -sd+ - | bc)
#or
sum=$(grep -oP "^$id\D*\K\d+" < "$file" | paste -sd+ - | bc)
the sed (or the grep) extracts and prints only the numbers
the paste make a string like number + number + number (-d+ is a delimiter)
the bc do the calculation
or perl
sum=$(perl -slanE '$s+=$F[1] if /^$id/}{say $s' -- -id="$id" "$file")
sum=$(ID="CallTilEdb" perl -lanE '$s+=$F[1] if /^$ENV{ID}/}{say $s' "$file")
Awk translation to script:
#!/bin/bash
declare -t filename=hendelse.logg
declare -t occurance="$1"
declare -i sumTime=0
sumtime=$(awk -v entry=$occurance '
$1==entry{time+=$NF+0}
END{print time+0}' $filename)
Here is a script which reads words from the file replaced.txt and displays the output each word in each line, But I want to display all the outputs in a single line.
#!/bin/sh
echo
echo "Enter the word to be translated"
read a
IFS=" " # Set the field separator
set $a # Breaks the string into $1, $2, ...
for a # a for loop by default loop through $1, $2, ...
do
{
b= grep "$a" replaced.txt | cut -f 2 -d" "
}
done
Content of "replaced.txt" file is given below:
hllo HELLO
m AM
rshbh RISHABH
jn JAIN
hw HOW
ws WAS
ur YOUR
dy DAY
This question can't be appropriate to what I asked, I just need the help to put output of the script in a single line.
Your entire script can be replaced by:
#!/bin/bash
echo
read -r -p "Enter the words to be translated: " a
echo $(printf "%s\n" $a | grep -Ff - replaced.txt | cut -f 2 -d ' ')
No need for a loop.
The echo with an unquoted argument removes embedded newlines and replaces each sequence of multiple spaces and/or tabs with one space.
One hackish-but-simple way to remove trailing newlines from the output of a command is to wrap it in printf %s "$(...) ". That is, you can change this:
b= grep "$a" replaced.txt | cut -f 2 -d" "
to this:
printf %s "$(grep "$a" replaced.txt | cut -f 2 -d" ") "
and add an echo command after the loop completes.
The $(...) notation sets up a "command substitution": the command grep "$a" replaced.txt | cut -f 2 -d" " is run in a subshell, and its output, minus any trailing newlines, is substituted into the argument-list. So, for example, if the command outputs DAY, then the above is equivalent to this:
printf %s "DAY "
(The printf %s ... notation is equivalent to echo -n ... — it outputs a string without adding a trailing newline — except that its behavior is more portably consistent, and it won't misbehave if the string you want to print happens to start with -n or -e or whatnot.)
You can also use
awk 'BEGIN { OFS=": "; ORS=" "; } NF >= 2 { print $2; }'
in a pipe after the cut.