bash subsequence matching speed-up - bash

I am wondering if there is an easy way to check if a string is a subsequence of another string in bash, actually a subsequence with an extra rule. I will explain.
Some subsequences of "apple" are "aple", "al", "pp" and "ale". The subsequences with an extra rule, I want to get are those that start and end with the same letter as the string so only "aple" and "ale" fit my desire.
I have made the following program:
#!/bin/bash
while read line
do
search=$(echo "$line" | tr -s 'A-Za-z' | sed 's/./\.\*&/g;s/^\.\*//' )
expr match "$1" "$search" >/dev/null && echo "$line"
done
It is executed as followed:
./program.sh greogdgedlqfe < words.txt
This program works, but is very slow.
It takes every line of the file, modify it to regex expression and then check if they match and then print the original line. So example:
one of the lines has the word google
$search becomes g.*o.*g.*l.*e (repeated letters become squeezed, extra rule )
then we check that expression with the given parameter and if it matches, we print the line: google
This works fine, however when the file words.txt gets too big, this program becomes too slow. How can I speed up my program, possibly by faster matching subsequences.
Edit after possible solution of Kamilcuk
That solution returns quick,quiff,quin,qwerty for the string "qwertyuihgfcvbnhjk" and only quick should be returned, so it is almost correct, but not quite yet.

Try it like so:
grep -x "$(<<<"$1" tr -s 'A-Za-z' | sed 's/./&*/g;s/\*$//;s/\*//1')" words.txt
Tested against:
set -- apple
cat >words.txt <<EOF
aple
al
pp
ale
fdafda
apppppppple
apple
google
EOF
outputs:
aple
ale
apppppppple
apple
And for set -- greogdgedlqfe it outputs just google.
If I understand you correctly, a "subsequent" of apple is everything that mathes ap*l*e.
Tested on repl

You can use a pattern instead of a regex. Just insert asterisk after each letter of each word (excpet the last letter) and use a normal pattern match.
#!/bin/bash
while read line
do
pattern=""
for ((i=${#line}-1 ; i>=0 ; --i)) ; do
pattern="${line:i:1}*"$pattern
done
pattern=${pattern%'*'}
if [[ "$1" == $pattern ]] ; then
echo "$line"
fi
done

bash does not need to use expr (an external program) for regular-expression matching; it provides built-in access to your system's library.
#!/bin/bash
while read line
do
search=$(echo "$line" | tr -s 'A-Za-z' | sed 's/./\.\*&/g;s/^\.\*//' )
[[ $1 =~ $search ]] && echo "$line"
done

Hard to beat perl with regexp.
Performance
The key to performance is to avoid forking extra processes. Most bash solutions presented here (with the exception of the KamilCuk grep based solution, which is not always correct) will require multiple calls to sed, tr, etc. Perl will outperform those solution. Even if a pure bash solution can be implemented (using bash RE, patterns), Perl is likely to outperform it, when the size of the word list is large.
Consider program.pl appl < words.txt
#! /usr/bin/perl
use strict ;
my $word = shift #ARGV ;
while ( <> ) {
chomp ;
my $p = $_ ;
tr/A-Za-z//s ;
s/(.)/.*$1/g ;
s/^\.\*// ;
print $p, "\n" if $word =~ "^$_\$" ;
} ;
Update 1: Perl implementation of KamilCuk solution + fix.
After minor fix, I believe possible to use the idea in the grep-based solution to create a Perl program that will be even faster. It create a single REGEXP, and test each word in the word list file. I think this is as optimal as possible with Perl.
#! /usr/bin/perl
use strict ;
$_ = shift #ARGV ;
tr/A-Za-z//s ;
s/(.)/$1*/g ;
s/\*// ;
s/\*$// ;
my $re = "^$_\$" ;
print "RE=$re\n" ;
while ( <> ) {
chomp ;
print $_, "\n" if /$re/ ;
} ;

Related

Reading a particular Digit from a given number in shell [duplicate]

I have a string in a Bash shell script that I want to split into an array of characters, not based on a delimiter but just one character per array index. How can I do this? Ideally it would not use any external programs. Let me rephrase that. My goal is portability, so things like sed that are likely to be on any POSIX compatible system are fine.
Try
echo "abcdefg" | fold -w1
Edit: Added a more elegant solution suggested in comments.
echo "abcdefg" | grep -o .
You can access each letter individually already without an array conversion:
$ foo="bar"
$ echo ${foo:0:1}
b
$ echo ${foo:1:1}
a
$ echo ${foo:2:1}
r
If that's not enough, you could use something like this:
$ bar=($(echo $foo|sed 's/\(.\)/\1 /g'))
$ echo ${bar[1]}
a
If you can't even use sed or something like that, you can use the first technique above combined with a while loop using the original string's length (${#foo}) to build the array.
Warning: the code below does not work if the string contains whitespace. I think Vaughn Cato's answer has a better chance at surviving with special chars.
thing=($(i=0; while [ $i -lt ${#foo} ] ; do echo ${foo:$i:1} ; i=$((i+1)) ; done))
As an alternative to iterating over 0 .. ${#string}-1 with a for/while loop, there are two other ways I can think of to do this with only bash: using =~ and using printf. (There's a third possibility using eval and a {..} sequence expression, but this lacks clarity.)
With the correct environment and NLS enabled in bash these will work with non-ASCII as hoped, removing potential sources of failure with older system tools such as sed, if that's a concern. These will work from bash-3.0 (released 2005).
Using =~ and regular expressions, converting a string to an array in a single expression:
string="wonkabars"
[[ "$string" =~ ${string//?/(.)} ]] # splits into array
printf "%s\n" "${BASH_REMATCH[#]:1}" # loop free: reuse fmtstr
declare -a arr=( "${BASH_REMATCH[#]:1}" ) # copy array for later
The way this works is to perform an expansion of string which substitutes each single character for (.), then match this generated regular expression with grouping to capture each individual character into BASH_REMATCH[]. Index 0 is set to the entire string, since that special array is read-only you cannot remove it, note the :1 when the array is expanded to skip over index 0, if needed.
Some quick testing for non-trivial strings (>64 chars) shows this method is substantially faster than one using bash string and array operations.
The above will work with strings containing newlines, =~ supports POSIX ERE where . matches anything except NUL by default, i.e. the regex is compiled without REG_NEWLINE. (The behaviour of POSIX text processing utilities is allowed to be different by default in this respect, and usually is.)
Second option, using printf:
string="wonkabars"
ii=0
while printf "%s%n" "${string:ii++:1}" xx; do
((xx)) && printf "\n" || break
done
This loop increments index ii to print one character at a time, and breaks out when there are no characters left. This would be even simpler if the bash printf returned the number of character printed (as in C) rather than an error status, instead the number of characters printed is captured in xx using %n. (This works at least back as far as bash-2.05b.)
With bash-3.1 and printf -v var you have slightly more flexibility, and can avoid falling off the end of the string should you be doing something other than printing the characters, e.g. to create an array:
declare -a arr
ii=0
while printf -v cc "%s%n" "${string:(ii++):1}" xx; do
((xx)) && arr+=("$cc") || break
done
If your string is stored in variable x, this produces an array y with the individual characters:
i=0
while [ $i -lt ${#x} ]; do y[$i]=${x:$i:1}; i=$((i+1));done
The most simple, complete and elegant solution:
$ read -a ARRAY <<< $(echo "abcdefg" | sed 's/./& /g')
and test
$ echo ${ARRAY[0]}
a
$ echo ${ARRAY[1]}
b
Explanation: read -a reads the stdin as an array and assigns it to the variable ARRAY treating spaces as delimiter for each array item.
The evaluation of echoing the string to sed just add needed spaces between each character.
We are using Here String (<<<) to feed the stdin of the read command.
I have found that the following works the best:
array=( `echo string | grep -o . ` )
(note the backticks)
then if you do: echo ${array[#]} ,
you get: s t r i n g
or: echo ${array[2]} ,
you get: r
Pure Bash solution with no loop:
#!/usr/bin/env bash
str='The quick brown fox jumps over a lazy dog.'
# Need extglob for the replacement pattern
shopt -s extglob
# Split string characters into array (skip first record)
# Character 037 is the octal representation of ASCII Record Separator
# so it can capture all other characters in the string, including spaces.
IFS= mapfile -s1 -t -d $'\37' array <<<"${str//?()/$'\37'}"
# Strip out captured trailing newline of here-string in last record
array[-1]="${array[-1]%?}"
# Debug print array
declare -p array
string=hello123
for i in $(seq 0 ${#string})
do array[$i]=${string:$i:1}
done
echo "zero element of array is [${array[0]}]"
echo "entire array is [${array[#]}]"
The zero element of array is [h]. The entire array is [h e l l o 1 2 3 ].
Yet another on :), the stated question simply says 'Split string into character array' and don't say much about the state of the receiving array, and don't say much about special chars like and control chars.
My assumption is that if I want to split a string into an array of chars I want the receiving array containing just that string and no left over from previous runs, yet preserve any special chars.
For instance the proposed solution family like
for (( i=0 ; i < ${#x} ; i++ )); do y[i]=${x:i:1}; done
Have left overs in the target array.
$ y=(1 2 3 4 5 6 7 8)
$ x=abc
$ for (( i=0 ; i < ${#x} ; i++ )); do y[i]=${x:i:1}; done
$ printf '%s ' "${y[#]}"
a b c 4 5 6 7 8
Beside writing the long line each time we want to split a problem, so why not hide all this into a function we can keep is a package source file, with a API like
s2a "Long string" ArrayName
I got this one that seems to do the job.
$ s2a()
> { [ "$2" ] && typeset -n __=$2 && unset $2;
> [ "$1" ] && __+=("${1:0:1}") && s2a "${1:1}"
> }
$ a=(1 2 3 4 5 6 7 8 9 0) ; printf '%s ' "${a[#]}"
1 2 3 4 5 6 7 8 9 0
$ s2a "Split It" a ; printf '%s ' "${a[#]}"
S p l i t I t
If the text can contain spaces:
eval a=( $(echo "this is a test" | sed "s/\(.\)/'\1' /g") )
$ echo hello | awk NF=NF FS=
h e l l o
Or
$ echo hello | awk '$0=RT' RS=[[:alnum:]]
h
e
l
l
o
I know this is a "bash" question, but please let me show you the perfect solution in zsh, a shell very popular these days:
string='this is a string'
string_array=(${(s::)string}) #Parameter expansion. And that's it!
print ${(t)string_array} -> type array
print $#string_array -> 16 items
This is an old post/thread but with a new feature of bash v5.2+ using the shell option patsub_replacement and the =~ operator for regex. More or less same with #mr.spuratic post/answer.
str='There can be only one, the Highlander.'
regexp="${str//?/(&)}"
[[ "$str" =~ $regexp ]] &&
printf '%s\n' "${BASH_REMATCH[#]:1}"
Or by just: (which includes the whole string at index 0)
declare -p BASH_REMATCH
If that is not desired, one can remove the value of the first index (index 0), with
unset -v 'BASH_REMATCH[0]'
instead of using printf or echo to print the value of the array BASH_REMATCH
One can check/see the value of the variable "$regexp" with either
declare -p regexp
Output
declare -- regexp="(T)(h)(e)(r)(e)( )(c)(a)(n)( )(b)(e)( )(o)(n)(l)(y)( )(o)(n)(e)(,)( )(t)(h)(e)( )(H)(i)(g)(h)(l)(a)(n)(d)(e)(r)(.)"
or
echo "$regexp"
Using it in a script, one might want to test if the shopt is enabled or not, although the manual says it is on/enabled by default.
Something like.
if ! shopt -q patsub_replacement; then
shopt -s patsub_replacement
fi
But yeah, check the bash version too! If you're not sure which version of bash is in use.
if ! ((BASH_VERSINFO[0] >= 5 && BASH_VERSINFO[1] >= 2)); then
printf 'No dice! bash version 5.2+ is required!\n' >&2
exit 1
fi
Space can be excluded from regexp variable, change it from
regexp="${str//?/(&)}"
To
regexp="${str//[! ]/(&)}"
and the output is:
declare -- regexp="(T)(h)(e)(r)(e) (c)(a)(n) (b)(e) (o)(n)(l)(y) (o)(n)(e) (t)(h)(e) (H)(i)(g)(h)(l)(a)(n)(d)(e)(r)(.)"
Maybe not as efficient as the other post/answer but it is still a solution/option.
If you want to store this in an array, you can do this:
string=foo
unset chars
declare -a chars
while read -N 1
do
chars[${#chars[#]}]="$REPLY"
done <<<"$string"x
unset chars[$((${#chars[#]} - 1))]
unset chars[$((${#chars[#]} - 1))]
echo "Array: ${chars[#]}"
Array: f o o
echo "Array length: ${#chars[#]}"
Array length: 3
The final x is necessary to handle the fact that a newline is appended after $string if it doesn't contain one.
If you want to use NUL-separated characters, you can try this:
echo -n "$string" | while read -N 1
do
printf %s "$REPLY"
printf '\0'
done
AWK is quite convenient:
a='123'; echo $a | awk 'BEGIN{FS="";OFS=" "} {print $1,$2,$3}'
where FS and OFS is delimiter for read-in and print-out
For those who landed here searching how to do this in fish:
We can use the builtin string command (since v2.3.0) for string manipulation.
↪ string split '' abc
a
b
c
The output is a list, so array operations will work.
↪ for c in (string split '' abc)
echo char is $c
end
char is a
char is b
char is c
Here's a more complex example iterating over the string with an index.
↪ set --local chars (string split '' abc)
for i in (seq (count $chars))
echo $i: $chars[$i]
end
1: a
2: b
3: c
zsh solution: To put the scalar string variable into arr, which will be an array:
arr=(${(ps::)string})
If you also need support for strings with newlines, you can do:
str2arr(){ local string="$1"; mapfile -d $'\0' Chars < <(for i in $(seq 0 $((${#string}-1))); do printf '%s\u0000' "${string:$i:1}"; done); printf '%s' "(${Chars[*]#Q})" ;}
string=$(printf '%b' "apa\nbepa")
declare -a MyString=$(str2arr "$string")
declare -p MyString
# prints declare -a MyString=([0]="a" [1]="p" [2]="a" [3]=$'\n' [4]="b" [5]="e" [6]="p" [7]="a")
As a response to Alexandro de Oliveira, I think the following is more elegant or at least more intuitive:
while read -r -n1 c ; do arr+=("$c") ; done <<<"hejsan"
declare -r some_string='abcdefghijklmnopqrstuvwxyz'
declare -a some_array
declare -i idx
for ((idx = 0; idx < ${#some_string}; ++idx)); do
some_array+=("${some_string:idx:1}")
done
for idx in "${!some_array[#]}"; do
echo "$((idx)): ${some_array[idx]}"
done
Pure bash, no loop.
Another solution, similar to/adapted from Léa Gris' solution, but using read -a instead of readarray/mapfile :
#!/usr/bin/env bash
str='azerty'
# Need extglob for the replacement pattern
shopt -s extglob
# Split string characters into array
# ${str//?()/$'\x1F'} replace each character "c" with "^_c".
# ^_ (Control-_, 0x1f) is Unit Separator (US), you can choose another
# character.
IFS=$'\x1F' read -ra array <<< "${str//?()/$'\x1F'}"
# now, array[0] contains an empty string and the rest of array (starting
# from index 1) contains the original string characters :
declare -p array
# Or, if you prefer to keep the array "clean", you can delete
# the first element and pack the array :
unset array[0]
array=("${array[#]}")
declare -p array
However, I prefer the shorter (and easier to understand for me), where we remove the initial 0x1f before assigning the array :
#!/usr/bin/env bash
str='azerty'
shopt -s extglob
tmp="${str//?()/$'\x1F'}" # same as code above
tmp=${tmp#$'\x1F'} # remove initial 0x1f
IFS=$'\x1F' read -ra array <<< "$tmp" # assign array
declare -p array # verification

BASH trying to create scrabble word generator, generated error: line 15: ?: syntax error: operand expected (error token is "?")

This is the script that will run if there are any "?" characters in the command line argument of the main script. I'm using i to run through the alphabet and essentially setting each "?" to $i for each letter in the alphabet. I think that the error is occurring because a special character is showing up at the end of the character array that can't be compared to "?", but I don't know how to deal with that. I did use ShellCheck already.
EDIT: The text file that is generated by the main script includes all permutations (no repeats) of the letters provided in the command line argument with a hard return between each permutation, and I am pretty sure that the hard return character is what is generating the error. Here is an example of permute.txt
at?
a?t
ta?
t?a
?at
?ta
And here is the slightly edited script:
#reads the text file produced by another script into the array
readarray words < ./permute.txt
#runs through loop 26 times with i as a letter in the alphabet
for i in {a..z}; do
#runs through every word in the array provided by the text file
for j in "${words[#]}"; do
s=$j #sets $j to a variable to mess with
declare -a a
word=""
#splits string s into character array
while read -n 1 c; do a+=($c); done <<< "$s"
#checks array for "?", used as a blank tile and changes that character to $i
for k in "${a[#]}"; do if [ "$k" = "?" ]; then a[$k]=$i; fi; done
#puts the array back together into a string
for ((k=0; k<${#a}; k++)); do echo $word; word=$word${a[$k]}; done
valid=$(grep -w $word /usr/share/dict/words) #checks for word in dictionary
#statement that fixes bug where words with apostrophes end up in the output
if [ ${#valid} -eq ${#i} ]; then
echo "$valid"
fi
done
done
Do I understand rigth, that you want wo replace the ? in the permutations with all characters a..z and then check if they are in the dictionary?
I would try without manually array work and let sed work for you (replacement of ?).
for i in {a..z}; do
for word in $(sed "s/\?/$i/" permute.txt); do
grep -w $word /usr/share/dict/words
done
done
You could even use bash's built in replacement in variables:
for i in {a..z}; do
for word in $(< permute.txt); do
grep -w ${word/\?/$i} /usr/share/dict/words
done
done
btw: I don't understand the last if statement. Since $i is a single character and $valid is a whole line, that at least contains a (in the example) 3-character word, if [ ${#valid} -eq ${#i} ] will never be true.

Using bash, how to assign integer to variable using echo

I'd like to understand bash a bit better as I'm apparently horrible at it...
I'm trying to generate a sequence of constant width integers, but then test them to do something exceptional for particular values. Like so:
for n in $(seq -w 1 150)
do
# The next line does not work: doit.sh: line 9: XX: command not found
#decval= $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' )
#if [[ ${decal} -eq 98 ]] ; then
if [[ $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' ) -eq 98 ]] ; then
echo "Do something different for 98"
elif [[ $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' ) -eq 105 ]] ; then
echo "Do something different for 98"
fi
done
This script works for my purposes, but if I try and make the assignment 'decval= $(…' I get an error 'command not found'. I don't understand this, can someone explain?
Also, is there an improvement I can make to this script if I have a large number of exceptions to prevent a long list of if ; then elif … ?
The problem is in the space between = and $:
decval= $(…
You should write without spaces:
decval=$(...
Because, if you write the space, your shell reads decval= as declval="" and treats the result of $(echo...) as the name of a command to execute, and obviously it doesn't find the command.
Also (just a small optimization), you can write:
sed 's/^0\+//'
instead of
sed 's/^0//g' | sed 's/^0//g'
Here:
0\+ means 0 one or more times;
g is removed, because g means replace all occurences in the string, and you have only one occurence (^ can be only one time in a string).
Also, you can check your variable even with leading zeros, without sed:
[[ "$n" =~ "0*98" ]]

case insensitive string comparison in bash

The following line removes the leading text before the variable $PRECEDING
temp2=${content#$PRECEDING}
But now i want the $PRECEDING to be case-insensitive. This works with sed's I flag. But i can't figure out the whole cmd.
No need to call out to sed or use shopt. The easiest and quickest way to do this (as long as you have Bash 4):
if [ "${var1,,}" = "${var2,,}" ]; then
echo "matched"
fi
All you're doing there is converting both strings to lowercase and comparing the results.
Here's a way to do it with sed:
temp2=$(sed -e "s/^.*$PRECEDING//I" <<< "$content")
Explanation:
^.*$PRECEDING: ^ means start of string, . means any character, .* means any character zero or more times. So together this means "match any pattern from start of string that is followed by (and including) string stored in $PRECEDING.
The I part means case-insensitive, the g part (if you use it) means "match all occurrences" instead of just the 1st.
The <<< notation is for herestrings, so you save an echo.
The only bash way I can think of is to check if there's a match (case-insensitively) and if yes, exclude the appropriate number of characters from the beginning of $content:
content=foo_bar_baz
PRECEDING=FOO
shopt -s nocasematch
[[ $content == ${PRECEDING}* ]] && temp2=${content:${#PRECEDING}}
echo $temp2
Outputs: _bar_baz
your examples have context-switching techniques.
better is (bash v4):
VAR1="HELLoWORLD"
VAR2="hellOwOrld"
if [[ "${VAR1^^}" = "${VAR2^^}" ]]; then
echo MATCH
fi
link: Converting string from uppercase to lowercase in Bash
If you don't have Bash 4, I find the easiest way is to first convert your string to lowercase using tr
VAR1=HelloWorld
VAR2=helloworld
VAR1_LOWER=$(echo "$VAR1" | tr '[:upper:]' '[:lower:]')
VAR2_LOWER=$(echo "$VAR2" | tr '[:upper:]' '[:lower:]')
if [ "$VAR1_LOWER" = "$VAR2_LOWER" ]; then
echo "Match"
else
echo "Invalid"
fi
This also makes it really easy to assign your output to variables by changing your echo to OUTPUT="Match" & OUTPUT="Invalid"

How to split one string into multiple strings separated by at least one space in bash shell?

I have a string containing many words with at least one space between each two. How can I split the string into individual words so I can loop through them?
The string is passed as an argument. E.g. ${2} == "cat cat file". How can I loop through it?
Also, how can I check if a string contains spaces?
I like the conversion to an array, to be able to access individual elements:
sentence="this is a story"
stringarray=($sentence)
now you can access individual elements directly (it starts with 0):
echo ${stringarray[0]}
or convert back to string in order to loop:
for i in "${stringarray[#]}"
do
:
# do whatever on $i
done
Of course looping through the string directly was answered before, but that answer had the the disadvantage to not keep track of the individual elements for later use:
for i in $sentence
do
:
# do whatever on $i
done
See also Bash Array Reference.
Did you try just passing the string variable to a for loop? Bash, for one, will split on whitespace automatically.
sentence="This is a sentence."
for word in $sentence
do
echo $word
done
This
is
a
sentence.
Probably the easiest and most secure way in BASH 3 and above is:
var="string to split"
read -ra arr <<<"$var"
(where arr is the array which takes the split parts of the string) or, if there might be newlines in the input and you want more than just the first line:
var="string to split"
read -ra arr -d '' <<<"$var"
(please note the space in -d ''; it cannot be omitted), but this might give you an unexpected newline from <<<"$var" (as this implicitly adds an LF at the end).
Example:
touch NOPE
var="* a *"
read -ra arr <<<"$var"
for a in "${arr[#]}"; do echo "[$a]"; done
Outputs the expected
[*]
[a]
[*]
as this solution (in contrast to all previous solutions here) is not prone to unexpected and often uncontrollable shell globbing.
Also this gives you the full power of IFS as you probably want:
Example:
IFS=: read -ra arr < <(grep "^$USER:" /etc/passwd)
for a in "${arr[#]}"; do echo "[$a]"; done
Outputs something like:
[tino]
[x]
[1000]
[1000]
[Valentin Hilbig]
[/home/tino]
[/bin/bash]
As you can see, spaces can be preserved this way, too:
IFS=: read -ra arr <<<' split : this '
for a in "${arr[#]}"; do echo "[$a]"; done
outputs
[ split ]
[ this ]
Please note that the handling of IFS in BASH is a subject on its own, so do your tests; some interesting topics on this:
unset IFS: Ignores runs of SPC, TAB, NL and on line starts and ends
IFS='': No field separation, just reads everything
IFS=' ': Runs of SPC (and SPC only)
Some last examples:
var=$'\n\nthis is\n\n\na test\n\n'
IFS=$'\n' read -ra arr -d '' <<<"$var"
i=0; for a in "${arr[#]}"; do let i++; echo "$i [$a]"; done
outputs
1 [this is]
2 [a test]
while
unset IFS
var=$'\n\nthis is\n\n\na test\n\n'
read -ra arr -d '' <<<"$var"
i=0; for a in "${arr[#]}"; do let i++; echo "$i [$a]"; done
outputs
1 [this]
2 [is]
3 [a]
4 [test]
BTW:
If you are not used to $'ANSI-ESCAPED-STRING' get used to it; it's a timesaver.
If you do not include -r (like in read -a arr <<<"$var") then read does backslash escapes. This is left as exercise for the reader.
For the second question:
To test for something in a string I usually stick to case, as this can check for multiple cases at once (note: case only executes the first match, if you need fallthrough use multiple case statements), and this need is quite often the case (pun intended):
case "$var" in
'') empty_var;; # variable is empty
*' '*) have_space "$var";; # have SPC
*[[:space:]]*) have_whitespace "$var";; # have whitespaces like TAB
*[^-+.,A-Za-z0-9]*) have_nonalnum "$var";; # non-alphanum-chars found
*[-+.,]*) have_punctuation "$var";; # some punctuation chars found
*) default_case "$var";; # if all above does not match
esac
So you can set the return value to check for SPC like this:
case "$var" in (*' '*) true;; (*) false;; esac
Why case? Because it usually is a bit more readable than regex sequences, and thanks to Shell metacharacters it handles 99% of all needs very well.
Just use the shells "set" built-in. For example,
set $text
After that, individual words in $text will be in $1, $2, $3, etc. For robustness, one usually does
set -- junk $text
shift
to handle the case where $text is empty or start with a dash. For example:
text="This is a test"
set -- junk $text
shift
for word; do
echo "[$word]"
done
This prints
[This]
[is]
[a]
[test]
$ echo "This is a sentence." | tr -s " " "\012"
This
is
a
sentence.
For checking for spaces, use grep:
$ echo "This is a sentence." | grep " " > /dev/null
$ echo $?
0
$ echo "Thisisasentence." | grep " " > /dev/null
$ echo $?
1
echo $WORDS | xargs -n1 echo
This outputs every word, you can process that list as you see fit afterwards.
(A) To split a sentence into its words (space separated) you can simply use the default IFS by using
array=( $string )
Example running the following snippet
#!/bin/bash
sentence="this is the \"sentence\" 'you' want to split"
words=( $sentence )
len="${#words[#]}"
echo "words counted: $len"
printf "%s\n" "${words[#]}" ## print array
will output
words counted: 8
this
is
the
"sentence"
'you'
want
to
split
As you can see you can use single or double quotes too without any problem
Notes:
-- this is basically the same of mob's answer, but in this way you store the array for any further needing. If you only need a single loop, you can use his answer, which is one line shorter :)
-- please refer to this question for alternate methods to split a string based on delimiter.
(B) To check for a character in a string you can also use a regular expression match.
Example to check for the presence of a space character you can use:
regex='\s{1,}'
if [[ "$sentence" =~ $regex ]]
then
echo "Space here!";
fi
For checking spaces just with bash:
[[ "$str" = "${str% *}" ]] && echo "no spaces" || echo "has spaces"
$ echo foo bar baz | sed 's/ /\n/g'
foo
bar
baz
For my use case, the best option was:
grep -oP '\w+' file
Basically this is a regular expression that matches contiguous non-whitespace characters. This means that any type and any amount of whitespace won't match. The -o parameter outputs each word matches on a different line.
Another take on this (using Perl):
$ echo foo bar baz | perl -nE 'say for split /\s/'
foo
bar
baz

Resources