This lists all English characters:
$ echo {A..Z}
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
But how to list all ASCII characters?
I tried this:
$ echo {\!..\~}
{!..~}
and this:
$ echo {$'!'..$'~'}
{!..~}
But both did not work. Is it possible?
This uses only one printf but a more complicated brace expansion.
printf '%b' \\x{0..7}{{0..9},{a..f}}
It also works, but not as nicely (it outputs a lot of whitespace):
echo -e \\x{0..7}{{0..9},{a..f}}
$ printf '%b\n' "$(printf '\%03o' {0..127})"
123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
To see a representation of the non-printable characters in the output from the above and the characters hidden by the effect of trying to print them as-is, you can pipe it to cat -v:
$ printf '%b\n' "$(printf '\%03o' {0..127})" | cat -v
^#^A^B^C^D^E^F^G^H
^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z^[^\^]^^^_ !"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~^?
To print just from the ASCII code for ! (33) to the ASCII code for ~ (126):
$ printf '%b\n' "$(printf '\%03o' {33..126})"
!"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
and to print from ! to ~ without having to know their numeric values:
$ printf '%b\n' "$(eval printf '\\%03o' $(printf '{%d..%d}' "'!" "'~"))"
!"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
which you can use with shell variables to hold the beginning and ending chars:
$ beg='!'; end='~';
$ printf '%b\n' "$(eval printf '\\%03o' $(printf '{%d..%d}' "'$beg" "'$end"))"
!"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
Related
I have a text file with something like,
!aa
#bb
#cc
$dd
%ee
expected output is,
! aa
# bb
# cc
$ dd
% ee
What I have tried, echo "${foo//#/# }".
This does work fine with one string but it does not work for all the lines in the file. I have tried with this while loop to read all the lines of the file and do the same using echo but it does not work.
while IFS= read -r line; do
foo=$line
sep="!##$%"
echo "${foo//$sep/$sep }"
done < $1
I have tried with awk split but it does not give the expected output. Is there any workaround for this? by using awk or sed.
The following assumes you want to add a space after every character in the !##$% set (even if it is the last character in a line). Test file:
$ cat file.txt
a!a
#bb
c#c
$dd
ee%
foo
%b%r
$ sep='!##$%'
With sed:
$ sed 's/['"$sep"']/& /g' file.txt
a! a
# bb
c# c
$ dd
ee%
foo
% b% r
With awk:
$ awk '{gsub(/['"$sep"']/,"& "); print}' file.txt
a! a
# bb
c# c
$ dd
ee%
foo
% b% r
With plain bash (not recommended, it is too slow):
$ while IFS= read -r line; do
str=""
for (( i=0; i<${#line}; i++ )); do
char="${line:i:1}"
str="$str$char"
[[ "$char" =~ [$sep] ]] && str="$str "
done
printf '%s\n' "$str"
done < file.txt
a! a
# bb
c# c
$ dd
ee%
foo
% b% r
Or (not sure which is the worst):
$ while IFS= read -r line; do
for (( i=0; i<${#sep}; i++ )); do
char="${sep:i:1}"
line="${line//$char/$char }"
done
printf '%s\n' "$line"
done < file.txt
a! a
# bb
c# c
$ dd
ee%
foo
% b% r
Characters you call special in your example seems to be subset of characters known as [[:punct:]] to GNU sed, thus I propose following solution:
sed 's/\([[:punct:]]\)/\1 /g' file.txt
which with file.txt content being
!aa
#bb
#cc
$dd
%ee
output
! aa
# bb
# cc
$ dd
% ee
Explanation: I use capturing group \(...\) which has any character belonging to [:punct:] then I replace what was captured with content of that capture followed by space. I use g to apply it to all occurences in each line, though this has not visible impact for data above. You might elect to drop g if you are sure there will be at most one character to replace in every line.
If you want to know more about [:punct:] or other similar character sets read about Character Classes on Regular-Expressions.info
If the file always contain a symbol at the start of line like that then use this
sed -Ei 's/^(.)/\1 /g' yourfile.txt
The -E option is to tell sed to use regex. -i modifies the file inline, you can remove it if you want to output to console or another file. The ^(.) regex captures the first character on the line and add a space to it (\1 )
Assuming that special characters are non-numeric and non-alphabetic characters, and special characters can appear anywhere in the line, use the following regular expression to replace them.
sed 's/[^a-zA-Z0-9]/& /g' urfile
I am trying to convert a list of quoted strings, separated by commas, into list of strings separated by newlines using bash and sed.
Here is an example of what I am doing:
#!/bin/bash
comma_to_newline() {
sed -En $'s/[ \t]*"([^"]*)",?[ \t]*/\\1\\\n/gp'
}
input='"one","two","three"'
expected="one\ntwo\nthree"
result="$( echo "${input}" | comma_to_newline )"
echo "Expected: <${expected}>"
echo "Result: <${result}>"
if [ "${result}" = "${expected}" ]; then
echo "EQUAL!"
else
echo "NOT EQUAL!"
fi
And the output I am getting is:
Expected: <one
two
three>
Result: <one
two
three>
NOT EQUAL!
I know it has something to do with the newlines characters, but I can't work out what. If I replace the newlines with some other string, such as XXX, it works fine and bash reports the strings as being equal.
Prompted by the comments on my question, I managed to work out what was going on. I was so focussed on coming up with a working sed expression and ensuring that result was correct, that I failed to noticed that the expected string was incorrect.
In order to use \n newlines in a bash string, you have to use the $'one\ntwo\nthree' syntax - see How can I have a newline in a string in sh? for other solutions.
I was developing against bash version 3.2.57 (the version that comes with Mac OS 10.14.6). When assigning a variable using expected="one\ntwo\nthree" then echoing it, they were being displayed as newlines in the console. Newer versions of bash display these strings as escaped - so I assume it is a bug that has been fixed in later versions of bash.
For diagnosing seemingly identical strings, try combining side-by-side diff output with a one char per line hexdump format. Replace:
else
echo "NOT EQUAL!"
fi
...with:
else
echo "NOT EQUAL!"
diff -y \
<(hexdump -v -e '/1 "%_ad# "' -e '/1 " _%_u\_\n"' <<< "${expected}") \
<(hexdump -v -e '/1 "%_ad# "' -e '/1 " _%_u\_\n"' <<< "${result}")
fi
There is extra new line character \n in string returing from your function.
Octal dump
$echo '"one","two","three"' | sed -En $'s/[ \t]*"([^"]*)",?[ \t]*/\\1\\\n/gp' | od -c
0000000 o n e \n t w o \n t h r e e \n \n
0000017
$echo "one\ntwo\nthree" | od -c
0000000 o n e \ n t w o \ n t h r e e \n
0000020
$
Also, use echo -e
$echo "one\ntwo\nthree"
one\ntwo\nthree
$echo -e "one\ntwo\nthree"
one
two
three
$
From man page
-e enable interpretation of backslash escapes
I need to print the following using bash
.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
What I have so far:
echo -n "."; for l in {a..z}; do echo -n $l; done
This will print:
.abcdefghijklmnopqrstuvwxyz but I also need the uppercase letters.
I am trying to do this on one line, so preferably same for loop.
I do not want to type out the alphabet by hand.
You could use two brace expansions:
$ printf -v str '%s' . {a..z} {A..Z}
$ echo "$str"
.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
$ echo . {a..z} {A..Z}| tr -d ' '
.abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ
I have a MAC address that looks like this.
01:AA:BB:0C:D0:E1
I want to convert it to lowercase and strip the leading zeros.
1:aa:bb:c:d0:e1
What's the simplest way to do that in a Bash script?
$ echo 01:AA:BB:0C:D0:E1 | sed 's/\(^\|:\)0/\1/g;s/.*/\L\0/'
1:aa:bb:c:d0:e1
\(^\|:\)0 represents either the line start (^) or a :, followed by a 0.
We want to replace this by the capture (either line start or :), which removed the 0.
Then, a second substitution (s/.*/\L\0/) put the whole line in lowercase.
$ sed --version | head -1
sed (GNU sed) 4.2.2
EDIT: Alternatively:
echo 01:AA:BB:0C:D0:E1 | sed 's/0\([0-9A-Fa-f]\)/\1/g;s/.*/\L\0/'
This replaces 0x (with x any hexa digit) by x.
EDIT: if your sed does not support \L, use tr:
echo 01:AA:BB:0C:D0:E1 | sed 's/0\([0-9A-Fa-f]\)/\1/g' | tr '[:upper:]' '[:lower:]'
Here's a pure Bash≥4 possibility:
mac=01:AA:BB:0C:D0:E1
IFS=: read -r -d '' -a macary < <(printf '%s:\0' "$mac")
macary=( "${macary[#]#0}" )
macary=( "${macary[#],,}" )
IFS=: eval 'newmac="${macary[*]}"'
The line IFS=: read -r -d '' -a macary < <(printf '%s:\0' "$mac") is the canonical way to split a string into an array,
the expansion "${macary[#]#0}" is that of the array macary with leading 0 (if any) removed,
the expansion "${macary[#],,}" is that of the array macary in lowercase,
IFS=: eval 'newmac="${macary[*]}"' is a standard way to join the fields of an array (note that the use of eval is perfectly safe).
After that:
declare -p newmac
yields
declare -- newmac="1:aa:bb:c:d0:e1"
as required.
A more robust way is to validate the MAC address first:
mac=01:AA:BB:0C:D0:E1
a='([[:xdigit:]]{2})' ; regex="^$a:$a:$a:$a:$a:$a$"
[[ $mac =~ $regex ]] || { echo "Invalid MAC address" >&2; exit 1; }
And then, using the valid result of the regex match (BASH_REMATCH):
set -- $(printf '%x ' $(printf '0x%s ' "${BASH_REMATCH[#]:1}" ))
IFS=: eval 'printf "%s\n" "$*"'
Which will print:
1:aa:bb:c:d0:e1
Hex values without leading zeros and in lowercase.
If Uppercase is needed, change the printf '%x ' to printf '%X '.
If Leading zeros are needed change the same to printf '%02x '.
third try to understand what Im doing wrong.
Ive got a list like this:
array[0] = 1111 Here is much text
array[1] = 2222 Here is even more text
array[2] = 1111.1 Here is special text
Now I want to sort the list to have it like this:
1111 Here is much text
1111.1 Here is special text
2222 Here is even more text
Using
for j in ${array[#]}; do echo $j; done | sort -n
it seperates me every single part because of the spaces.
Using
for j in "${array[#]}"; do echo "$j"; done | sort -n
I get a sorted list like 1111 2222 1111.1
array=(
"1111 Here is much text"
"2222 Here is even more text"
"1111.1 Here is special text"
)
printf "%s\n" "${array[#]}" | sort -n
1111 Here is much text
1111.1 Here is special text
2222 Here is even more text
To save it:
sorted=()
while IFS= read -r line; do
sorted+=("$line")
done < <( printf "%s\n" "${array[#]}" | sort -n )
printf "%s\n" "${sorted[#]}"
# same output as above
or
source <( echo 'sorted=('; printf '"%s"\n' "${array[#]}" | sort -n; echo ')' )
printf "%s\n" "${sorted[#]}"
Carriage returns in your file will mess you up. Consider file named "t" with dos-style line endings:
$ cat -e t
line1^M$
line2^M$
line3^M$
$ for n in {1..3} ; do array[n]="$(echo $n $(cat t))"; done
$ printf "%s\n" "${array[#]}"|od -c
0000000 1 l i n e 1 \r l i n e 2 \r
0000020 l i n e 3 \r \n 2 l i n e 1 \r
0000040 l i n e 2 \r l i n e 3 \r \n 3
0000060 l i n e 1 \r l i n e 2 \r l i
0000100 n e 3 \r \n
0000105
$ printf "%s\n" "${array[#]}"
line31
line31
line31
Clearly this is going to mess up anything you feed with this input. Fix the carriage returns.
Your locale is set in such a way that the . is interpreted as a thousands separator, rather than a decimal point, and the numeric values are sorted accordingly (1111.1 is interpreteted as 11111, e.g. with LC_ALL=de_DE). Use
export LC_ALL=C
before you execute sort (and, of course, use proper quoting, as in glenn's and fedorqui's answers).