I try to write a script. With this script I need to remove return carriage at the end of the output numbers I parsed from some command output. So I need to transform them to integer. But printf won't format the number the way I want:
echo $var
2.80985e+09
var=$(printf "%s" "$var" | tr -dc '[:digit:]' )
echo $var
28098509
As you may see, printf removes the carriage but also modifies the value of variable. But I would like this value remain same, only return carriage is removed. Which parameter I should use with printf?
Thanks
Maybe you want to do this:
$ printf "%f\n" $var
2809850000.000000
Or this:
$ printf "%f\n" $var | sed -e 's/\..*//'
2809850000
printf did not modify the value of the variable; tr did. You can verify this by:
$ printf "%s\n" "$var"
2.80985e+09
$ printf "%s\n" "$var" | tr -dc '[:digit:]'
28098509
The tr command, as given, removes all non-digit characters.
Your tr command said 'remove every non-digit', so it did that. You should expect programs to do exactly what you tell them to. The whole var=$(...) sequence is bizarre. To remove a carriage return, you could use:
var=$(tr -d '\013' <<< $var)
The <<< redirection sends the string (value of $var) as the standard input of the command.
Related
I have a string:
foo="re-9619-add-selling-office";
I'd like to break up the string on the second - (dash) into variable1 and variable2. I want to end up with variable1=re-9619 and variable2=add-selling-office
I tried it using grep and awk, but now I not sure that's the way to go.
Here is a single sed + read way:
foo="re-9619-add-selling-office"
read var1 var2 < <(sed -E 's/^([^-]*-[^-]*)-/\1 /' <<< "$foo")
# check variables
declare -p var1 var2
declare -- var1="re-9619"
declare -- var2="add-selling-office"
Could you please try following once. Where first variable will have value like re-9619 and second shell variable will have value like add-selling-office
first=$(echo "$foo" | sed 's/\([^-]*-[^-]*\)-.*/\1/')
second=$(echo "$foo" | sed 's/\([^-]*\)-\([^-]*\)-\(.*\)/\3/')
Explanation:
echo "$foo" | sed 's/\([^-]*-[^-]*\)-.*/\1/': Printing value of foo variable and passing its output to sed command. In sed I am using substitute capability to perform substitution, \([^-]*-[^-]*\)-.*(which has everything from starting of value to till 2nd occurrence of - in back reference in it). Then substituting whole value with 1st captured back reference value which will become only re-9619.
echo "$foo" | sed 's/\([^-]*\)-\([^-]*\)-\(.*\)/\3/': Logic is same as above mentioned command. Using sed's capability of substitution with using back reference capability of it. Here we are printing everything after 2nd occurrence of -.
NOTE: second=$(echo "$foo" | sed -E "s/$first-(.*)/\1/") could also help as per #User123's comments.
That can be done using parameter expansions, you don't need an external utility.
$ foo="re-9619-add-selling-office"
$ variable2=${foo#*-*-}
$ variable1=${foo%-"$variable2"}
$
$ echo $variable1
re-9619
$ echo $variable2
add-selling-office
You can use cut:
variable1=$(echo $foo | cut -d '-' -f 1-2)
variable2=$(echo $foo | cut -d '-' -f 3-)
This is the result:
>> echo $variable1
re-9619
>> echo $variable2
add-selling-office
I have a MAC address that looks like this.
01:AA:BB:0C:D0:E1
I want to convert it to lowercase and strip the leading zeros.
1:aa:bb:c:d0:e1
What's the simplest way to do that in a Bash script?
$ echo 01:AA:BB:0C:D0:E1 | sed 's/\(^\|:\)0/\1/g;s/.*/\L\0/'
1:aa:bb:c:d0:e1
\(^\|:\)0 represents either the line start (^) or a :, followed by a 0.
We want to replace this by the capture (either line start or :), which removed the 0.
Then, a second substitution (s/.*/\L\0/) put the whole line in lowercase.
$ sed --version | head -1
sed (GNU sed) 4.2.2
EDIT: Alternatively:
echo 01:AA:BB:0C:D0:E1 | sed 's/0\([0-9A-Fa-f]\)/\1/g;s/.*/\L\0/'
This replaces 0x (with x any hexa digit) by x.
EDIT: if your sed does not support \L, use tr:
echo 01:AA:BB:0C:D0:E1 | sed 's/0\([0-9A-Fa-f]\)/\1/g' | tr '[:upper:]' '[:lower:]'
Here's a pure Bash≥4 possibility:
mac=01:AA:BB:0C:D0:E1
IFS=: read -r -d '' -a macary < <(printf '%s:\0' "$mac")
macary=( "${macary[#]#0}" )
macary=( "${macary[#],,}" )
IFS=: eval 'newmac="${macary[*]}"'
The line IFS=: read -r -d '' -a macary < <(printf '%s:\0' "$mac") is the canonical way to split a string into an array,
the expansion "${macary[#]#0}" is that of the array macary with leading 0 (if any) removed,
the expansion "${macary[#],,}" is that of the array macary in lowercase,
IFS=: eval 'newmac="${macary[*]}"' is a standard way to join the fields of an array (note that the use of eval is perfectly safe).
After that:
declare -p newmac
yields
declare -- newmac="1:aa:bb:c:d0:e1"
as required.
A more robust way is to validate the MAC address first:
mac=01:AA:BB:0C:D0:E1
a='([[:xdigit:]]{2})' ; regex="^$a:$a:$a:$a:$a:$a$"
[[ $mac =~ $regex ]] || { echo "Invalid MAC address" >&2; exit 1; }
And then, using the valid result of the regex match (BASH_REMATCH):
set -- $(printf '%x ' $(printf '0x%s ' "${BASH_REMATCH[#]:1}" ))
IFS=: eval 'printf "%s\n" "$*"'
Which will print:
1:aa:bb:c:d0:e1
Hex values without leading zeros and in lowercase.
If Uppercase is needed, change the printf '%x ' to printf '%X '.
If Leading zeros are needed change the same to printf '%02x '.
I have a file which contains one line of text with tabs
echo -e "foo\tbar\tfoo2\nx\ty\tz" > file.txt
I'd like to get the first column with cut. It works if I do
$ cut -f 1 file.txt
foo
x
But if I read it in a bash script
while read line
do
new_name=`echo -e $line | cut -f 1`
echo -e "$new_name"
done < file.txt
Then I get instead
foo bar foo2
x y z
What am I doing wrong?
/edit: My script looks like that right now
while IFS=$'\t' read word definition
do
clean_word=`echo -e $word | external-command'`
echo -e "$clean_word\t<b>$word</b><br>$definition" >> $2
done < $1
External command removes diacritics from a Greek word. Can the script be optimized any further without changing external-command?
What is happening is that you did not quote $line when reading the file. Then, the original tab-delimited format was lost and instead of tabs, spaces show in between words. And since cut's default delimiter is a TAB, it does not find any and it prints the whole line.
So quoting works:
while read line
do
new_name=`echo -e "$line" | cut -f 1`
#----------------^^^^^^^
echo -e "$new_name"
done < file.txt
Note, however, that you could have used IFS to set the tab as field separator and read more than one parameter at a time:
while IFS=$'\t' read name rest;
do
echo "$name"
done < file.txt
returning:
foo
x
And, again, note that awk is even faster for this purpose:
$ awk -F"\t" '{print $1}' file.txt
foo
x
So, unless you want to call some external command while looping the file, awk (or sed) is better.
I have some variable $a. This variable have non printing characters (carriage return ^M).
>echo $a
some words for compgen
>a+="END"
>echo $a
ENDe words for compgen
How I can remove that char?
I know that echo "$a" display it correct. But it's not a solution in my case.
You could use tr:
tr -dc '[[:print:]]' <<< "$var"
would remove non-printable character from $var.
$ foo=$'abc\rdef'
$ echo "$foo"
def
$ tr -dc '[[:print:]]' <<< "$foo"
abcdef
$ foo=$(tr -dc '[[:print:]]' <<< "$foo")
$ echo "$foo"
abcdef
To remove just the trailing carriage return from a, use
a=${a%$'\r'}
I was trying to send a notification via libnotify, with content that may contain unprintable characters. The existing solutions did not quite work for me (using a whitelist of characters using tr works, but strips any multi-byte characters).
Here is what worked, while passing the đŸ’© test:
message=$(iconv --from-code=UTF-8 -c <<< "$message")
As an equivalent to the tr approach using only shell builtins:
cleanVar=${var//[![:print:]]/}
...substituting :print: with the character class you want to keep, if appropriate.
tr -dc '[[:alpha:]]'
will translate your string to only have alpha characters (if that is needed)
When I execute commands in Bash (or to be specific, wc -l < log.txt), the output contains a linebreak after it. How do I get rid of it?
If your expected output is a single line, you can simply remove all newline characters from the output. It would not be uncommon to pipe to the tr utility, or to Perl if preferred:
wc -l < log.txt | tr -d '\n'
wc -l < log.txt | perl -pe 'chomp'
You can also use command substitution to remove the trailing newline:
echo -n "$(wc -l < log.txt)"
printf "%s" "$(wc -l < log.txt)"
If your expected output may contain multiple lines, you have another decision to make:
If you want to remove MULTIPLE newline characters from the end of the file, again use cmd substitution:
printf "%s" "$(< log.txt)"
If you want to strictly remove THE LAST newline character from a file, use Perl:
perl -pe 'chomp if eof' log.txt
Note that if you are certain you have a trailing newline character you want to remove, you can use head from GNU coreutils to select everything except the last byte. This should be quite quick:
head -c -1 log.txt
Also, for completeness, you can quickly check where your newline (or other special) characters are in your file using cat and the 'show-all' flag -A. The dollar sign character will indicate the end of each line:
cat -A log.txt
One way:
wc -l < log.txt | xargs echo -n
If you want to remove only the last newline, pipe through:
sed -z '$ s/\n$//'
sed won't add a \0 to then end of the stream if the delimiter is set to NUL via -z, whereas to create a POSIX text file (defined to end in a \n), it will always output a final \n without -z.
Eg:
$ { echo foo; echo bar; } | sed -z '$ s/\n$//'; echo tender
foo
bartender
And to prove no NUL added:
$ { echo foo; echo bar; } | sed -z '$ s/\n$//' | xxd
00000000: 666f 6f0a 6261 72 foo.bar
To remove multiple trailing newlines, pipe through:
sed -Ez '$ s/\n+$//'
There is also direct support for white space removal in Bash variable substitution:
testvar=$(wc -l < log.txt)
trailing_space_removed=${testvar%%[[:space:]]}
leading_space_removed=${testvar##[[:space:]]}
If you want to print output of anything in Bash without end of line, you echo it with the -n switch.
If you have it in a variable already, then echo it with the trailing newline cropped:
$ testvar=$(wc -l < log.txt)
$ echo -n $testvar
Or you can do it in one line, instead:
$ echo -n $(wc -l < log.txt)
If you assign its output to a variable, bash automatically strips whitespace:
linecount=`wc -l < log.txt`
printf already crops the trailing newline for you:
$ printf '%s' $(wc -l < log.txt)
Detail:
printf will print your content in place of the %s string place holder.
If you do not tell it to print a newline (%s\n), it won't.
Adding this for my reference more than anything else ^_^
You can also strip a new line from the output using the bash expansion magic
VAR=$'helloworld\n'
CLEANED="${VAR%$'\n'}"
echo "${CLEANED}"
Using Awk:
awk -v ORS="" '1' log.txt
Explanation:
-v assignment for ORS
ORS - output record separator set to blank. This will replace new line (Input record separator) with ""