How to shift each letter of the string by a given number of letters? - bash

How can i shift each letter of a string by a given number of letters down or up in bash, without using a hardcoded dictionary?

Do you mean something like ROT13:
pax$ echo 'hello there' | tr '[a-z]' '[n-za-m]'
uryyb gurer
pax$ echo 'hello there' | tr '[a-z]' '[n-za-m]' | tr '[a-z]' '[n-za-m]'
hello there
For a more general solution where you want to provide an arbitrary rotation (0 through 26), you can use:
#!/usr/bin/bash
dual=abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz
phrase='hello there'
rotat=13
newphrase=$(echo $phrase | tr "${dual:0:26}" "${dual:${rotat}:26}")
echo ${newphrase}

If you want to rotate also the capitals you could use something like this:
cat data.txt | tr '[a-z]' '[n-za-m]' | tr '[A-Z]' '[N-ZA-M]'
where data.txt has whatever you want to rotate.

$ alpha=abcdefghijklmnopqrstuvwxyz
$ rot=3
$ sed "y/${alpha}/${alpha:$rot}${alpha::$rot}/" <<< 'foobar'
irredu

Shift by 12 characters(A becomes M, and vice versa)
Encryption
----------
$> echo ABCDE | tr '[A-Z]' '[M-ZA-L]' // prints MNOPQ
Decryption
----------
$> echo MNOPQ | tr '[M-ZA-L]' '[A-Z]' // prints ABCDE
In the encryption example, we are piping ABCDE to the command tr which is given two arguments. The first one is a matching string. It will match certain strings in your input(in our case ABCDE). The second argument works upon the result of the first argument and modifies it accordingly. So, we're basically matching any uppercase letter present in the input ABCDE and passing it to the second argument. The second argument replaces the characters with their 12th next counterpart. Now, this part is important to understand and might confuse some people, we're basically going from [M-L] in the second argument. Since the tr command doesn't accept this directly, we're breaking it up into two separate chunks. First chunk is [M-Z] and the second one is [A-L]. It's basically like a search-and-replace mechanism. You search with the first argument, modify with the second argument, as simple as that.
For the second example, I've just swapped the first argument with the second one in the tr command. Which acts perfectly as a decryptor. You could write it the same way as the first example, but I find it less time consuming when I have the encryption algorithm and I can just swap the arguments to have a decryption algorithm as well.

Or
cat data.txt | tr 'a-zA-Z' 'n-za-mN-ZA-M'
It will also work

Without using tr, shift 1 to 25 characters
and can be decrypted using 26 - original key
#!/bin/bash
#set -x
i=0
for letters in {A..Z}
do
abc_cap[$i]="$letters"
((i++))
done
i=0
for letters in {a..z}
do
abc_small[$i]="$letters"
((i++))
done
read -r -p "Enter message to be encrypted/decrypted: " -a message
read -r -p "Enter shift amount (26 - orig key for decrypt): " shift_amount
echo -n "Encrypted message: "
if [ "$shift_amount" -gt 25 ] || [ "$shift_amount" -lt 1 ]
then
echo "Shift amount out of range"
exit
fi
for word in "${message[#]}"
do
while read -r -n 1 letter
do
if [[ "$letter" = [a-z] ]]
then
for a in "${!abc_small[#]}"
do
if [ "${abc_small[$a]}" = "$letter" ]
then
a=$(echo "($a + $shift_amount) % 26" | bc)
echo -n "${abc_small[$a]}"
fi
done
elif [[ "$letter" = [A-Z] ]]
then
for a in "${!abc_cap[#]}"
do
if [ "${abc_cap[$a]}" = "$letter" ]
then
a=$(echo "($a + $shift_amount) % 26" | bc)
echo -n "${abc_cap[$a]}"
fi
done
elif [[ "$letter" = "" ]]
then echo -n " "
else echo -n "$letter"
fi
done < <(echo "$word")
done
echo
exit

Problem statement and how this command can help you:
For example The password is stored in the file data.txt, where 13 positions have rotated all lowercase (a-z) and uppercase (A-Z) letters.
The data.txt file contains 1 line encrypted with the ROT13 ( rotation by 13) algorithm. In order to decrypt it, I have to replace every letter with the letter 13 positions ahead.
file contains the data as shown below
cat data.txt
Gur cnffjbeq vf WIAOOSFzMjXXBC0KoSKBbJ8puQm5lIEi
after rotation to 13 character, the password will look like this.
The password is JVNBBFSmZwKKOP0XbFXOoW8chDz5yVRv
The command to Do that is given below.
cat data.txt | tr '[A-Za-z]' '[N-ZA-Mn-za-m]'
Explanation of the Command
cat data.txt read all the character in data.txt file and then pass to tr command, tr commands takes two arguments, the first argument [A-Za-z] read only the characters made of A-Z or a-z. and in the second argument is rotation regular expression.
[13th character from A - ZA-12th character from A and same expression as for small letters]
[N-ZA-Mn-za-m]
N : 13th character from A.
Z : to the end.
A : first character.
N : just a previous character from the 13th character. to complete the circle.
repeat the same expression for small letters.
We rotated by 13, you can replace the 13th and Previous character by any x position to rotate the string by x characters

Related

Take first 16 character and covert it into hex string

I have UUID, 3abbea88-c77d-11eb-b8bc-0242ac130003 and I want to take first 16 character of this string and want Hexadecimal string of first 16 characters using shell script.
I tried,
code=$(echo -n ${${ID##*:}:0:16} | od -A n -t x1)
HEX_ID=$(echo ${code//[[:blank:]]/})
Any better way ?
Expected Output : 33616262656138382d633737642d3131
Using od you can simply limit the number of read characters using the -N option:
HEX_ID=$(od -A n -t x1 -N 16 <<< ${ID##*:} | tr -dc '[:xdigit:]')
Edit: tr is used to suppress non-hexadecimal characters, namely whitespaces and potential newlines.
Perl to the rescue!
perl -le 'print unpack "H32", shift' 3abbea88-c77d-11eb-b8bc-0242ac130003
-l adds newlines to print
unpack takes a string and expands it to a list of values based on a template. H32 means "take characters and interpret them as 32 hex values".
shift reads the first command line argument.
Or, using xxd and head:
echo 3abbea88-c77d-11eb-b8bc-0242ac130003 | xxd -p | head -c32
That's certainly a useless echo.
Probably avoid uppercase for your private variables.
uuid='3abbea88-c77d-11eb-b8bc-0242ac130003'
tmp=${uuid//-/}
hex_id=$(od -A n -t x1 <<<${tmp:0:13})
hex_id=${hex_id//[[:blank:]]/}
hex_id=${hex_id%0a}
The here string unattractively supplies trailing newline to od which we have to trim off.
Bash-only:
while read -r -N 1 c # read input string 1 char at a time
do [[ "$c" == " " ]] || # skip embedded spaces
printf "%02X" "$( # output the hexidecimal value of
printf "%d" \'$c # the ASCII decimal ordinal of $c
)"
done <<< "${text##*:}" # ignoring the leading trash to the :
echo # newline-teminate the output
All in one line:
while read -rn1 c;do [[ "$c" == " " ]]||printf %02X $(printf "%d" \'$c);done<<<"${text##*:}";echo
This is not the fastest approach...
hexdump does it all:
hexdump -n 16 -ve '1/1 "%.2x"'
-n 16 means only process the first 16 bytes
-e '1/1 "%.2x"' means display each byte using given printf format
-v means display normally (without this, it replaces dupe sections with * 🤷)
echo '3abbea88-c77d-11eb-b8bc-0242ac130003' | hexdump -n 16 -ve '1/1 "%.2x"'
output:
33616262656138382d633737642d3131

How to get all characters except the last one in a bash script?

I have a text file which contains filenames and labels to this files
Example String:
0-3081031014094495-0.png 0
I am using this command to iterate over the text file and get last char.
while IFS= read -r line; do
echo $line | tail -c 2
done <$PWD/$i/caffe/test.txt
I also want to get everything before the last char.
Something like echo $line | head -c -2 which I interpret as:
start from the beginning and get everything until the two last chars
Edit:
Thank you for so many really fast answers. Something I did not mention in the original question was that I am using a mac. I thought it would not matter, but trying some of your answers I realized that it matters.
For example using negative substrings is not supported on mac.
James Brown solution was the first which worked for me, so thats why I accepted his answer.
To get everything before the last char in bash:
$ foo="0-3081031014094495-0.png 0"
$ echo ${foo%?}
0-3081031014094495-0.png
From http://tldp.org/LDP/abs/html/string-manipulation.html :
${string%substring}
Deletes shortest match of $substring from back of $string.
from man head and man tail
# head
-c, --bytes=[-]K
print the first K bytes of each file; with the leading `-', print all but the last K bytes of each file
# tail
-c, --bytes=K
output the last K bytes; alternatively, use -c +K to output bytes starting with the Kth of each file
Examples
$ tail -c -2 <<< 'hello world!'
!
$ head -c -2 <<< 'hello world!'; echo
hello world
However last char and all except last char can be obtain more efficiently with shell expansion
$ line='hello world!'
$ echo "${line:${#line}-1}"
$ echo "${line:0:${#line}-1}"
If your input is space separated and the file names cannot contain spaces, read by itself is sufficient.
while read -r filename label; do
printf "filename: %s label: %s\n" "$filename" "$label"
done <"$PWD/$i/caffe/test.txt"
The simplest solution would be to use rev(1), i.e.:
$ echo "$line" | rev | cut -c 1 | rev
0
You can also pass line length to cut, i.e.:
$ echo "$line" | cut -c ${#line}
0
Or to cut few last chars:
$ echo "$line" | cut -c $((${#line} - 2))-
g 0

bash check if a string has a character more than once

The title actually almost explains it all. I would like to check if a string contains a letter (not a specific letter, really any letter) more than once.
for example:
user:
test.sh this list
script:
if [ "$1" has some letter more then once ]
then
do something
fi
Use a Posix character class:
if [[ $1 =~ [[:alpha:]].*[[:alpha:]] ]]; then
echo "more than one letter"
fi
This regex (in bash) will tell you the first lower case letter that is repeated.
And which is it:
#!/bin/bash
regex="([a-z]).*\1"
if [[ $1 =~ $regex ]]; then
echo "more than one letter ${BASH_REMATCH[1]}"
fi
Call as:
$ script.sh "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZz"
more than one letter "z"
Of course, the range of letters could be changed to lower and upper:
[a-zA-Z]
But only if the LC_COLLATE is set to "C", if that is set to UTF-8, then also accented characters could be included in the a-z range. As this may show:
$ ./sc.sh abcdefghijklémnopéqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZz
more than one letter "é"
This will keep letters as what ASCII believe a letter is:
$ LC_COLLATE=C ./sc.sh abcdefghijklémnopéqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZz
more than one letter "z"
The range of characters could be some of the POSIX character ranges:
[[:word:]] [[:alpha:]] [[:lower:]] [[:upper:]]
Please note that what those ranges mean is also changed by the character set in use.
If you want to go by using just basic commands, you can use something like this ...
#!/bin/bash
PATH=/bin/:/usr/bin/:$PATH
if [ `echo $* | tr -d ' ' | sed 's/\(.\)/\1\n/g' | sort | uniq -c | tr -s ' ' | sort -n | grep -v '^ 1 ' | wc -l` -ge 1 ]
then
echo "Input contains duplicate characters"
fi
In case it is unclear, it will be easy to try it out each step on the command line like this ... echo test input | tr -d ' 'see the output, then add the sed part to it and so on and so forth.
The first tr -d ' ' will ensure spaces from your input are not counted as duplicates. For example, if the input is "abcd efgh ijkl", the only character repeating is the space. If you keep tr -d ' ' in there, the script will not count the input to be having duplicate characters, if you remove it, the script will count the input to be having duplicate characters.
Cheers.
-- Parag

Correctly count number of lines a bash variable

I need to count the number of lines of a given variable. For example I need to find how many lines VAR has, where VAR=$(git log -n 10 --format="%s").
I tried with echo "$VAR" | wc -l), which indeed works, but if VAR is empty, is prints 1, which is wrong. Is there a workaround for this? Something better than using an if clause to check whether the variable is empty...(maybe add a line and subtract 1 from the returned value?).
The wc counts the number of newline chars. You can use grep -c '^' for counting lines.
You can see the difference with:
#!/bin/bash
count_it() {
echo "Variablie contains $2: ==>$1<=="
echo -n 'grep:'; echo -n "$1" | grep -c '^'
echo -n 'wc :'; echo -n "$1" | wc -l
echo
}
VAR=''
count_it "$VAR" "empty variable"
VAR='one line'
count_it "$VAR" "one line without \n at the end"
VAR='line1
'
count_it "$VAR" "one line with \n at the end"
VAR='line1
line2'
count_it "$VAR" "two lines without \n at the end"
VAR='line1
line2
'
count_it "$VAR" "two lines with \n at the end"
what produces:
Variablie contains empty variable: ==><==
grep:0
wc : 0
Variablie contains one line without \n at the end: ==>one line<==
grep:1
wc : 0
Variablie contains one line with \n at the end: ==>line1
<==
grep:1
wc : 1
Variablie contains two lines without \n at the end: ==>line1
line2<==
grep:2
wc : 1
Variablie contains two lines with \n at the end: ==>line1
line2
<==
grep:2
wc : 2
You can always write it conditionally:
[ -n "$VAR" ] && echo "$VAR" | wc -l || echo 0
This will check whether $VAR has contents and act accordingly.
For a pure bash solution: instead of putting the output of the git command into a variable (which, arguably, is ugly), put it in an array, one line per field:
mapfile -t ary < <(git log -n 10 --format="%s")
Then you only need to count the number of fields in the array ary:
echo "${#ary[#]}"
This design will also make your life simpler if, e.g., you need to retrieve the 5th commit message:
echo "${ary[4]}"
try:
echo "$VAR" | grep ^ | wc -l

How to perform a for loop on each character in a string in Bash?

I have a variable like this:
words="这是一条狗。"
I want to make a for loop on each of the characters, one at a time, e.g. first character="这", then character="是", character="一", etc.
The only way I know is to output each character to separate line in a file, then use while read line, but this seems very inefficient.
How can I process each character in a string through a for loop?
You can use a C-style for loop:
foo=string
for (( i=0; i<${#foo}; i++ )); do
echo "${foo:$i:1}"
done
${#foo} expands to the length of foo. ${foo:$i:1} expands to the substring starting at position $i of length 1.
With sed on dash shell of LANG=en_US.UTF-8, I got the followings working right:
$ echo "你好嗎 新年好。全型句號" | sed -e 's/\(.\)/\1\n/g'
你
好
嗎
新
年
好
。
全
型
句
號
and
$ echo "Hello world" | sed -e 's/\(.\)/\1\n/g'
H
e
l
l
o
w
o
r
l
d
Thus, output can be looped with while read ... ; do ... ; done
edited for sample text translate into English:
"你好嗎 新年好。全型句號" is zh_TW.UTF-8 encoding for:
"你好嗎" = How are you[ doing]
" " = a normal space character
"新年好" = Happy new year
"。全型空格" = a double-byte-sized full-stop followed by text description
${#var} returns the length of var
${var:pos:N} returns N characters from pos onwards
Examples:
$ words="abc"
$ echo ${words:0:1}
a
$ echo ${words:1:1}
b
$ echo ${words:2:1}
c
so it is easy to iterate.
another way:
$ grep -o . <<< "abc"
a
b
c
or
$ grep -o . <<< "abc" | while read letter; do echo "my letter is $letter" ; done
my letter is a
my letter is b
my letter is c
I'm surprised no one has mentioned the obvious bash solution utilizing only while and read.
while read -n1 character; do
echo "$character"
done < <(echo -n "$words")
Note the use of echo -n to avoid the extraneous newline at the end. printf is another good option and may be more suitable for your particular needs. If you want to ignore whitespace then replace "$words" with "${words// /}".
Another option is fold. Please note however that it should never be fed into a for loop. Rather, use a while loop as follows:
while read char; do
echo "$char"
done < <(fold -w1 <<<"$words")
The primary benefit to using the external fold command (of the coreutils package) would be brevity. You can feed it's output to another command such as xargs (part of the findutils package) as follows:
fold -w1 <<<"$words" | xargs -I% -- echo %
You'll want to replace the echo command used in the example above with the command you'd like to run against each character. Note that xargs will discard whitespace by default. You can use -d '\n' to disable that behavior.
Internationalization
I just tested fold with some of the Asian characters and realized it doesn't have Unicode support. So while it is fine for ASCII needs, it won't work for everyone. In that case there are some alternatives.
I'd probably replace fold -w1 with an awk array:
awk 'BEGIN{FS=""} {for (i=1;i<=NF;i++) print $i}'
Or the grep command mentioned in another answer:
grep -o .
Performance
FYI, I benchmarked the 3 aforementioned options. The first two were fast, nearly tying, with the fold loop slightly faster than the while loop. Unsurprisingly xargs was the slowest... 75x slower.
Here is the (abbreviated) test code:
words=$(python -c 'from string import ascii_letters as l; print(l * 100)')
testrunner(){
for test in test_while_loop test_fold_loop test_fold_xargs test_awk_loop test_grep_loop; do
echo "$test"
(time for (( i=1; i<$((${1:-100} + 1)); i++ )); do "$test"; done >/dev/null) 2>&1 | sed '/^$/d'
echo
done
}
testrunner 100
Here are the results:
test_while_loop
real 0m5.821s
user 0m5.322s
sys 0m0.526s
test_fold_loop
real 0m6.051s
user 0m5.260s
sys 0m0.822s
test_fold_xargs
real 7m13.444s
user 0m24.531s
sys 6m44.704s
test_awk_loop
real 0m6.507s
user 0m5.858s
sys 0m0.788s
test_grep_loop
real 0m6.179s
user 0m5.409s
sys 0m0.921s
I believe there is still no ideal solution that would correctly preserve all whitespace characters and is fast enough, so I'll post my answer. Using ${foo:$i:1} works, but is very slow, which is especially noticeable with large strings, as I will show below.
My idea is an expansion of a method proposed by Six, which involves read -n1, with some changes to keep all characters and work correctly for any string:
while IFS='' read -r -d '' -n 1 char; do
# do something with $char
done < <(printf %s "$string")
How it works:
IFS='' - Redefining internal field separator to empty string prevents stripping of spaces and tabs. Doing it on a same line as read means that it will not affect other shell commands.
-r - Means "raw", which prevents read from treating \ at the end of the line as a special line concatenation character.
-d '' - Passing empty string as a delimiter prevents read from stripping newline characters. Actually means that null byte is used as a delimiter. -d '' is equal to -d $'\0'.
-n 1 - Means that one character at a time will be read.
printf %s "$string" - Using printf instead of echo -n is safer, because echo treats -n and -e as options. If you pass "-e" as a string, echo will not print anything.
< <(...) - Passing string to the loop using process substitution. If you use here-strings instead (done <<< "$string"), an extra newline character is appended at the end. Also, passing string through a pipe (printf %s "$string" | while ...) would make the loop run in a subshell, which means all variable operations are local within the loop.
Now, let's test the performance with a huge string.
I used the following file as a source:
https://www.kernel.org/doc/Documentation/kbuild/makefiles.txt
The following script was called through time command:
#!/bin/bash
# Saving contents of the file into a variable named `string'.
# This is for test purposes only. In real code, you should use
# `done < "filename"' construct if you wish to read from a file.
# Using `string="$(cat makefiles.txt)"' would strip trailing newlines.
IFS='' read -r -d '' string < makefiles.txt
while IFS='' read -r -d '' -n 1 char; do
# remake the string by adding one character at a time
new_string+="$char"
done < <(printf %s "$string")
# confirm that new string is identical to the original
diff -u makefiles.txt <(printf %s "$new_string")
And the result is:
$ time ./test.sh
real 0m1.161s
user 0m1.036s
sys 0m0.116s
As we can see, it is quite fast.
Next, I replaced the loop with one that uses parameter expansion:
for (( i=0 ; i<${#string}; i++ )); do
new_string+="${string:$i:1}"
done
The output shows exactly how bad the performance loss is:
$ time ./test.sh
real 2m38.540s
user 2m34.916s
sys 0m3.576s
The exact numbers may very on different systems, but the overall picture should be similar.
I've only tested this with ascii strings, but you could do something like:
while test -n "$words"; do
c=${words:0:1} # Get the first character
echo character is "'$c'"
words=${words:1} # trim the first character
done
It is also possible to split the string into a character array using fold and then iterate over this array:
for char in `echo "这是一条狗。" | fold -w1`; do
echo $char
done
The C style loop in #chepner's answer is in the shell function update_terminal_cwd, and the grep -o . solution is clever, but I was surprised not to see a solution using seq. Here's mine:
read word
for i in $(seq 1 ${#word}); do
echo "${word:i-1:1}"
done
#!/bin/bash
word=$(echo 'Your Message' |fold -w 1)
for letter in ${word} ; do echo "${letter} is a letter"; done
Here is the output:
Y is a letter
o is a letter
u is a letter
r is a letter
M is a letter
e is a letter
s is a letter
s is a letter
a is a letter
g is a letter
e is a letter
To iterate ASCII characters on a POSIX-compliant shell, you can avoid external tools by using the Parameter Expansions:
#!/bin/sh
str="Hello World!"
while [ ${#str} -gt 0 ]; do
next=${str#?}
echo "${str%$next}"
str=$next
done
or
str="Hello World!"
while [ -n "$str" ]; do
next=${str#?}
echo "${str%$next}"
str=$next
done
sed works with unicode
IFS=$'\n'
for z in $(sed 's/./&\n/g' <(printf '你好嗎')); do
echo hello: "$z"
done
outputs
hello: 你
hello: 好
hello: 嗎
Another approach, if you don't care about whitespace being ignored:
for char in $(sed -E s/'(.)'/'\1 '/g <<<"$your_string"); do
# Handle $char here
done
Another way is:
Characters="TESTING"
index=1
while [ $index -le ${#Characters} ]
do
echo ${Characters} | cut -c${index}-${index}
index=$(expr $index + 1)
done
fold and while read are great for the job as shown in some answers here. Contrary to those answers, I think it's much more intuitive to pipe in the order of execution:
echo "asdfg" | fold -w 1 | while read c; do
echo -n "$c "
done
Outputs: a s d f g
I share my solution:
read word
for char in $(grep -o . <<<"$word") ; do
echo $char
done
TEXT="hello world"
for i in {1..${#TEXT}}; do
echo ${TEXT[i]}
done
where {1..N} is an inclusive range
${#TEXT} is a number of letters in a string
${TEXT[i]} - you can get char from string like an item from an array

Resources