The following bash snippet gives "conditional binary operator expected"
countries=$1
while read -r line; do
if [[ echo "$line" | grep BUSINESS | grep -E "$countries" ]]; then
echo $line >> "$Business_Accounts"
fi
done
What's going wrong?
Just change your if statement like below,
if [[ $(echo "$line" | grep BUSINESS | grep -E "$countries") ]]; then
OR
You could do like the above in a single grep command like in the below example because grep or awk or sed processes the input line by line.
$ contries="foo"
$ echo 'foo BUSINESS bar
bar' | grep -P "^(?=.*BUSINESS).*$con"
foo BUSINESS bar
$ Business_account=$(echo 'foo BUSINESS bar
bar' | grep -P "^(?=.*BUSINESS).*$con")
$ echo "$Business_account"
foo BUSINESS bar
In a single line, it would be like,
Business_account=$(grep -P "^(?=.*BUSINESS).*$contries" file)
Related
Problem
I want to find a simple, single line way to pipe a string depending on a certain condition
Attempt
The above code was my attempt at making a pipe conditional depending on a variable called textfolding.
textfolding="ON"
echo "some text blah balh test foo" if [[ "$textfolding" == "ON" ]]; then | fold -s -w "$fold_width" | sed -e "s|^|\t|g"; fi
This obviously did not work.
Final
How could I achieve this on the same one line?
You can't make the pipe itself conditional, but you can include an if block as an element of the pipeline:
echo "some text blah balh test foo" | if [[ "$textfolding" == "ON" ]]; then fold -s -w "$fold_width" | sed -e "s|^|\t|g"; else cat; fi
Here's a more readable version:
echo "some text blah balh test foo" |
if [[ "$textfolding" == "ON" ]]; then
fold -s -w "$fold_width" | sed -e "s|^|\t|g"
else
cat
fi
Note that since the if block is part of the pipeline, you need to include something like an else cat clause (as I did above) so that whether the if condition is true or not, something will pass the piped data through. Without the cat, it'd just get dropped on the metaphorical floor.
How about conditional execution?
textfolding="ON"
string="some text blah balh test foo"
[[ $textfolding == "ON" ]] && echo $string | fold -s -w $fold_width | sed -e "s|^|\t|g" || echo $string
I have 3 different files that I want to compare
words_freq
words_freq_deduped
words_freq_alpha
For each file, I run a command like so, which I iterate on constantly to compare the results.
For example, I would do this:
$ cat words_freq | grep -v '[soe]'
$ cat words_freq_deduped | grep -v '[soe]'
$ cat words_freq_alpha | grep -v '[soe]'
and then review the results, and then do it again, with an additional filter
$ cat words_freq | grep -v '[soe]' | grep a | grep r | head -n20
a
$ cat words_freq_deduped | grep -v '[soe]' | grep a | grep r | head -n20
b
$ cat words_freq_alpha | grep -v '[soe]' | grep a | grep r | head -n20
c
This continues on until I've analyzed my data.
I would like to write a script that could take the piped portions, and pass it to each of these files, as I iterate on the grep/head portions of the command.
e.g. The following would dump the results of running the 3 commands above AND also compare the 3 results, and dump additional calculations on them
$ myScript | grep -v '[soe]' | grep a | grep r | head -n20
the letters were in all 3 runs, and it took 5 seconds
a
b
c
How can I do this using bash/python or zsh for the myScript part?
EDIT: After asking the question, it occurred to me that I could use eval to do it, like so, which I've added as an answer as well
The following approach allows me to process multiple files by using eval, which I know is frowned upon - any other suggestions are greatly appreciated!
$ myScript "grep -v '[soe]' | grep a | grep r | head -n20"
myScript
#!/usr/bin/env bash
function doIt(){
FILE=$1
CMD="cat $1 | $2"
echo processing file "$FILE"
eval "$CMD"
echo
}
doIt words_freq "$#"
doIt words_freq_deduped "$#"
doIt words_freq_alpha "$#"
You can't avoid your shell from running pipes itself, so using it like that isn't very practical - you'd need to either quote everything and then eval it, which would make it hard to pass arguments with spaces, or quote every pipe, which you can then eval, making it so you have to quote every pipe. But yeah, these solutions are kinda hacky.
I'd suggest doing one of these two:
Keep your editor open, and put whatever you want to run inside the doIt function itself before you run it. Then run it in your shell without any arguments:
#!/usr/bin/env bash
doIt() {
# grep -v '[soe]' < "$1"
grep -v '[soe]' < "$1" | grep a | grep r | head -n20
}
doIt words_freq
doIt words_freq_deduped
doIt words_freq_alpha
Or, you could always use a "for" in your shell, which you can use Ctrl+r to find in your history when you want to use:
$ for f in words_freq*; do grep -v '[soe]' < "$f" | grep a | grep r | head -n20; done
But if you really want your approach, I tried to make it accept spaces, but it ended up being even hackier:
#!/usr/bin/env bash
doIt() {
local FILE=$1
shift
echo processing file "$FILE"
local args=()
for n in $(seq 1 $#); do
arg=$1
shift
if [[ $arg == '|' ]]; then
args+=('|')
else
args+=("\"$arg\"")
fi
done
eval "cat '$FILE' | ${args[#]}"
}
doIt words_freq "$#"
doIt words_freq_deduped "$#"
doIt words_freq_alpha "$#"
With this version you can use it like this:
$ ./myScript grep "a a" "|" head -n1
Notice that it need you to quote the |, and that it now handles arguments with spaces.
Not fully understood problem correctly.
I understood you want to write a script without pipes, by including the filtering logic into the script.
And feeding the filtering patterns as arguments.
Here is a gawk script (standard Linux awk).
With one sweep on 3 input files, without piping.
script.awk
BEGIN {
RS="!#!#!#!#!#!#!#";
# set record separator to something unlikely matched, causing each file to be read entirely as a single record
}
$0 !~ excludeRegEx # if file does not match excludeRegEx
&& $0 ~ includeRegEx1 # and match includeRegEx1
&& $0 ~ includeRegEx2 { # and match includeRegEx2
system "head -n20 "FILENAME; # call shell command "head -n20 " on current filename
}
Running script.awk
awk -v excludeRegEx='[soe]' \
-v includeRegEx1='a' \
-v includeRegEx2='r' \
-f script.awk words_freq words_freq_deduped words_freq_alpha
The following approach allows me to process multiple files by using eval, which I know is frowned upon - any other suggestions are greatly appreciated!
$ myScript "grep -v '[soe]' | grep a | grep r | head -n20"
myScript
#!/usr/bin/env bash
function doIt(){
FILE=$1
CMD="cat $1 | $2"
echo processing file "$FILE"
eval "$CMD"
echo
}
doIt words_freq "$#"
doIt words_freq_deduped "$#"
doIt words_freq_alpha "$#"
Trying to append an entry to the last line of sshd.conf however it appends it to the end of the previous line.
echo -e "DenyGroups $(echo ${admin_membership_ad_group} | cut -f2 -d= |cut -f1 -d,|awk '{print tolower($0)}')" >> /etc/ssh/sshd.conf
and
echo "\nDenyGroups $(echo ${admin_membership_ad_group} | cut -f2 -d= |cut -f1 -d,|awk '{print tolower($0)}')" >> /etc/ssh/sshd.conf
Expectation:
Lastline
DenyGroups somegroup
Result:
LastlineDenyGroups somegroup
With GNU sed:
sed -i '$a DenyGroups '"${admin_membership_ad_group}" /etc/ssh/sshd.conf
$: refers to the last line
a: append
In your last command you have forgotten the -e, see the test below:
$ # Create test file
$ echo -n LastLine > tst
$ # See what happens if you forget -e
$ echo -n '\n + echo without -e' >> tst
$ # Now add -e
$ echo -e "\nDenyGroups $(echo admin_membership_ad_group)" >> tst
$ # See the results
$ cat tst
LastLine\n + echo without -e
DenyGroups admin_membership_ad_group
Note that in the other shells builtin echo behaves differently. For instance in zsh the expansion of \n will happen even without specifying the -e option.
So I have an expression that I want to extract some lines from a text and count them. I can grep them as follows:
$ cat medsCounts_totals.csv | grep -E 'NumMeds": 0' | wc -l
Which is fine. Now I want to loop over with the string ...
$ for i in {0..10}; do expr="NumMeds\": $i"; echo $expr; done
However, when I try to use $expr
for i in {0..10}; do expr="NumMeds:\" $i"; cat medsCounts_totals.csv | grep -E "$expr" | wc -l ; done
I get nothing. How do I solve this problem in an elegant manner?
there is a typo in
for i in {0..10}; do expr="NumMeds:\" $i"; cat medsCounts_totals.csv | grep -E "$expr" | wc -l ; done
it should be
"NumMeds\": $i"
I want to uppercase just the first character in my string with bash.
foo="bar";
//uppercase first character
echo $foo;
should print "Bar";
One way with bash (version 4+):
foo=bar
echo "${foo^}"
prints:
Bar
foo="$(tr '[:lower:]' '[:upper:]' <<< ${foo:0:1})${foo:1}"
One way with sed:
echo "$(echo "$foo" | sed 's/.*/\u&/')"
Prints:
Bar
$ foo="bar";
$ foo=`echo ${foo:0:1} | tr '[a-z]' '[A-Z]'`${foo:1}
$ echo $foo
Bar
To capitalize first word only:
foo='one two three'
foo="${foo^}"
echo $foo
One two three
To capitalize every word in the variable:
foo="one two three"
foo=( $foo ) # without quotes
foo="${foo[#]^}"
echo $foo
One Two Three
(works in bash 4+)
Using awk only
foo="uNcapItalizedstrIng"
echo $foo | awk '{print toupper(substr($0,0,1))tolower(substr($0,2))}'
Here is the "native" text tools way:
#!/bin/bash
string="abcd"
first=`echo $string|cut -c1|tr [a-z] [A-Z]`
second=`echo $string|cut -c2-`
echo $first$second
just for fun here you are :
foo="bar";
echo $foo | awk '{$1=toupper(substr($1,0,1))substr($1,2)}1'
# or
echo ${foo^}
# or
echo $foo | head -c 1 | tr [a-z] [A-Z]; echo $foo | tail -c +2
# or
echo ${foo:1} | sed -e 's/^./\B&/'
It can be done in pure bash with bash-3.2 as well:
# First, get the first character.
fl=${foo:0:1}
# Safety check: it must be a letter :).
if [[ ${fl} == [a-z] ]]; then
# Now, obtain its octal value using printf (builtin).
ord=$(printf '%o' "'${fl}")
# Fun fact: [a-z] maps onto 0141..0172. [A-Z] is 0101..0132.
# We can use decimal '- 40' to get the expected result!
ord=$(( ord - 40 ))
# Finally, map the new value back to a character.
fl=$(printf '%b' '\'${ord})
fi
echo "${fl}${foo:1}"
This works too...
FooBar=baz
echo ${FooBar^^${FooBar:0:1}}
=> Baz
FooBar=baz
echo ${FooBar^^${FooBar:1:1}}
=> bAz
FooBar=baz
echo ${FooBar^^${FooBar:2:2}}
=> baZ
And so on.
Sources:
Bash Manual: Shell Parameter Expansion
Full Bash Guide: Parameters
Bash Hacker's Wiki Parameter Expansion
Inroductions/Tutorials:
Cyberciti.biz: 8. Convert to upper to lower case or vice versa
Opensource.com: An introduction to parameter expansion in Bash
This one worked for me:
Searching for all *php file in the current directory , and replace the first character of each filename to capital letter:
e.g: test.php => Test.php
for f in *php ; do mv "$f" "$(\sed 's/.*/\u&/' <<< "$f")" ; done
Alternative and clean solution for both Linux and OSX, it can also be used with bash variables
python -c "print(\"abc\".capitalize())"
returns Abc
This is POSIX sh-compatible as far as I know.
upper_first.sh:
#!/bin/sh
printf "$1" | cut -c1 -z | tr -d '\0' | tr [:lower:] [:upper:]
printf "$1" | cut -c2-
cut -c1 -z ends the first string with \0 instead of \n. It gets removed with tr -d '\0'. It also works to omit the -z and use tr -d '\n' instead, but this breaks if the first character of the string is a newline.
Usage:
$ upper_first.sh foo
Foo
$
In a function:
#!/bin/sh
function upper_first ()
{
printf "$1" | cut -c1 -z | tr -d '\0' | tr [:lower:] [:upper:]
printf "$1" | cut -c2-
}
old="foo"
new="$(upper_first "$old")"
echo "$new"
Posix compliant and with less sub-processes:
v="foo[Bar]"
printf "%s" "${v%"${v#?}"}" | tr '[:lower:]' '[:upper:]' && printf "%s" "${v#?}"
==> Foo[Bar]
first-letter-to-lower () {
str=""
space=" "
for i in $#
do
if [ -z $(echo $i | grep "the\|of\|with" ) ]
then
str=$str"$(echo ${i:0:1} | tr '[A-Z]' '[a-z]')${i:1}$space"
else
str=$str${i}$space
fi
done
echo $str
}
first-letter-to-upper-xc () {
v-first-letter-to-upper | xclip -selection clipboard
}
first-letter-to-upper () {
str=""
space=" "
for i in $#
do
if [ -z $(echo $i | grep "the\|of\|with" ) ]
then
str=$str"$(echo ${i:0:1} | tr '[a-z]' '[A-Z]')${i:1}$space"
else
str=$str${i}$space
fi
done
echo $str
}
first-letter-to-lower-xc(){
v-first-letter-to-lower | xclip -selection clipboard
}
Not exactly what asked but quite helpful
declare -u foo #When the variable is assigned a value, all lower-case characters are converted to upper-case.
foo=bar
echo $foo
BAR
And the opposite
declare -l foo #When the variable is assigned a value, all upper-case characters are converted to lower-case.
foo=BAR
echo $foo
bar
What if the first character is not a letter (but a tab, a space, and a escaped double quote)? We'd better test it until we find a letter! So:
S=' \"รณ foo bar\"'
N=0
until [[ ${S:$N:1} =~ [[:alpha:]] ]]; do N=$[$N+1]; done
#F=`echo ${S:$N:1} | tr [:lower:] [:upper:]`
#F=`echo ${S:$N:1} | sed -E -e 's/./\u&/'` #other option
F=`echo ${S:$N:1}
F=`echo ${F} #pure Bash solution to "upper"
echo "$F"${S:(($N+1))} #without garbage
echo '='${S:0:(($N))}"$F"${S:(($N+1))}'=' #garbage preserved
Foo bar
= \"Foo bar=