CSV Column Counts via Command Line Alias - bash

I have a Bash scripting question that's no longer important to me but has still been driving me crazy because I can't figure it out.
I was trying to compare column counts of CSV files via the command line, and found some instructions here that looked promising. The original command worked perfectly, but every attempt to create an alias failed.
Original command line - WORKS!
head -n 1 FILENAME.CSV | awk '{split($0, pieces, ","); for (i=1; i<=length(pieces); i++) print pieces[i];}' | wc -l
Ex. molsen#molsen-M14xR2:~/projects/ee-export$ head -n 1 ee-content.csv | awk '{split($0, pieces, ","); for (i=1; i<=length(pieces); i++) print pieces[i];}' | wc -l
7
Aliased command line - FAILS!
alias csvcc="head -n 1 $1 | awk '{split($0, pieces, ","); for (i=1; i<=length(pieces); i++) print pieces[i];}' | wc -l"
Ex. molsen#molsen-M14xR2:~/projects/ee-export$ csvcc ee-content.csv
awk: cmd. line:1: {split(bash, pieces, ,); for (i=1; i<=length(pieces); i++) print pieces[i];}
awk: cmd. line:1: ^ syntax error
awk: cmd. line:1: {split(bash, pieces, ,); for (i=1; i<=length(pieces); i++) print pieces[i];}
awk: cmd. line:1: ^ syntax error
awk: cmd. line:1: {split(bash, pieces, ,); for (i=1; i<=length(pieces); i++) print pieces[i];}
awk: cmd. line:1: ^ 0 is invalid as number of arguments for split
1023 ee-content.csv
I've tried all sorts of variations on this command with different quoting and escaping, but they all produce errors. I even looked into strong vs. weak quoting, but didn't find anything useful. Also, I came up with this simpler command, but it fails in the same way:
COMMAND LINE USING SED - WORKS!
head -n 1 file.csv | sed 's/\,/\n/g' | wc -l
molsen#molsen-M14xR2:~/projects/ee-export$ head -n 1 ee-content.csv | sed 's/\,/\n/g' | wc -l
7
ALIAS USING SED - FAILS (COUNTS LINES, NOT COLUMNS)
alias csvcc="head -n 1 $1 | sed 's/\,/\n/g' | wc -l"
molsen#molsen-M14xR2:~/projects/ee-export$ csvcc ee-content.csv
1023 ee-content.csv
Can anyone can tell me what I'm missing here? Thanks for your help!

rzymek's Solution 1 doesn't work because csvcc <ee-content.csv expands to
head -n 1 | sed 's/\,/\n/g' | wc -l <ee-content.csv
i. e., the input file is not fed to head, but to wc -l; this explains why it gives the line count.
Another solution with alias and awk:
alias csvcc="awk -F, 'NR==1 { print NF }'"

Aliases don't use $1, scripts do.
Solution 1.
alias csvcc="head -n 1 | sed 's/\,/\n/g' | wc -l"
^--- no $1, input will be read from stdin
Usage:
csvcs < ee-content.csv
Solution 2.
Put the line in a bash script file in $HOME/bin/csvcc:
#!/bin/bash
head -n 1 | sed 's/\,/\n/g' | wc -l
Then make the script executable:
chmod +x ~/bin/csvcc
On ubuntu $HOME/bin is added to $PATH by default. If it's not on your system, than add this line to your ~/.profile:
export PATH="$HOME/bin:$PATH"
Usage:
csvcs ee-content.csv

Related

unexpected EOF while looking for matching error

I am running below script and getting
error script.sh: line 9: unexpected EOF while looking for matching `''
script.sh: line 15: syntax error: unexpected end of file.
Though I tried to run line 9 manually n it runs without error.
alias gxt="awk -F "_" '{print \$1}' test | uniq"
count = $(cat test | awk -F "_" '{print $1} | uniq | wc -l)
for i in {1..count};
do
User=$(gxt | head -n $i)
recharge=$(grep -E "$User.recharge" test| awk -F "_" '{print $3}' | xargs )
total1=( $((${recharge// /+})))
sales=$(grep -E "$User.sale" test| awk -F "_" '{print $3}' | xargs )
total2=( $((${sales// /+})))
balance=`expr $total1 - $total2`
echo $User.balance.$balance >> result
done
Other than the issues already reported and those exposed by shellcheck, there is another issue:
for i in {1..count};
'count' cannot be a variable. It can only be a constant.
Change it to
for ((i = 1; i <= count; i++)); do whatever ; done
count=$(cat test | awk -F "_" '{print $1}` | uniq | wc -l)
missing ' at the end of {print $1}
Inadvertently added spaces around =

awk: cmd. line:1: { print \$9} : syntax error

I am trying to run the following command to get the "gis" & "cpu" usage from shell script. However when i try to run this command even manually, then i am getting the below error.
Any senior/expert please help on this.
$ top -b | head -n 8 | grep -w gis | awk '{ print \$9}'
awk: cmd. line:1: { print \$9}
awk: cmd. line:1: ^ backslash not last character on line
awk: cmd. line:1: { print \$9}
awk: cmd. line:1: ^ syntax error
$ top -b | head -n 8 | grep -w gis | awk '{ print \$10}'
# Same error i am getting for second command as well.
You don't need to escape the $. You are using single quotes, so the shell won't interpolate the $9. If you were using double quotes, you would need to escape it.
So, you have to...
top -b | head -n 8 | grep -w gis | awk '{ print $9}'
or
top -b | head -n 8 | grep -w gis | awk "{ print \$9}"

redirect output of command to a variable in bash script

I am trying to redirect the output of the following command to a variable $bei
awk '/Total number of/ && /multidriven/' ../reports/synthesis /hdl.check_design.rpt | grep -o '[0-9]' | awk '{s+=$1} END {print s}'
this command is working correctly if execute directly, and I am trying to redirect it into a variable to be used later.
I've tried 2 ways to do it:
bei= `awk '/Total number of/ && /multidriven/' ../reports/synthesis/hdl.check_design.rpt | grep -o '[0-9]' | awk '{s+=$1} END {print s}'`
bei= $(awk '/Total number of/ && /multidriven/' ../reports/synthesis/hdl.check_design.rpt | grep -o '[0-9]' | awk '{s+=$1} END {print s}')
But they both don't work and I receive "command not found" error. Does anyone have any idea about this and could you please help me? Thanks a lot
Reason for failure seem to be extra space between bei and $(awk.....) .
Example :
sh-4.1$ x =$(date) #With space
sh: x: command not found
sh-4.1$ x=$(date) #Without space
sh-4.1$ echo $x
Thu Sep 8 06:41:07 EDT 2016
sh-4.1$

shell commands won't run in bash script

I'm trying to pass a parameter to a bash script:
"words.sh surf"
#!/bin/bash
#words.sh
#purpose:
#
# to return a list of words
# sorted by the length of each word
#
i=$1
cat ~/wordlist | grep $i | awk '{ print length(),$0 | "sort -n" }'
and I'm getting this message:
usage: grep [-abcDEFGHhIiJLlmnOoPqRSsUVvwxZ] [-A num] [-B num] [-C[num]].
awk: syntax error at source line 1
context is.
{ print length(), >>> -/bin/bash <<< | "sort -n" }
awk: illegal statement at source line 1
if I run the statement as a command :
cat ~/wordlist | grep surf | awk '{ print length(),$0 | "sort -n" }'
I get the correct output:
...
...
12 hypersurface
12 surfboarding
12 undersurface
12 unsurfeiting
13 subtersurface
What am I doing wrong here?
You said it's working for you, but trust me it's got issues. Re-write it as:
awk -v i="$1" '$0 ~ i{print length(), $0 }' | sort -n

error in awk of shell script

I am getting the below error ith my code.What is missing in it? My goal is to print 13.0.5.8 in $version
#!/bin/ksh
file="abc_def_APP_13.0.5.8"
if echo "$file" | grep -E "abc_def_APP"; then
echo "Version found: $file"
version1=(echo $file | awk -F_ '{print $NF}' | cut -d. -f1-3)
version2=(echo $file | awk -F_ '{print $NF}' | cut -d. -f4-)
echo $version1
echo $version2
version=$version$version2
echo $version
else
echo "Version not found"
fi
Please find below the error:
./version.sh: line 7: syntax error near unexpected token `|'
./version.sh: line 7: ` version1=(echo $file | awk -F_ '{print $NF}' | cut -d. -f1-3)'
./version.sh: line 9: syntax error near unexpected token `|'
./version.sh: line 9: ` version2=(echo $file | awk -F_ '{print $NF}' | cut -d. -f4-)'
./version.sh: line 18: syntax error near unexpected token `else'
There's no need for awk at all. Just trim every character before the last underscore, like so:
file="abc_def_APP_13.0.5.8"
version="${file##*_}"
echo "$version"
See http://mywiki.wooledge.org/BashFAQ/073 for documentation on this technique, or see "parameter expansion" in bash's own docs.
To treat the last segment separately is also straightforward:
file="abc_def_APP_13.0.5.8"
version="${file##*_}" # result: 13.0.5.8
version_end="${version##*.}" # result: 8
version_start="${version%.*}" # result: 13.0.5
echo "${version_start}/${version_end}" # result: 13.0.5/8
Because this happens internally to bash, without executing any external commands (such as awk), it should be considerably faster to execute than other approaches given.
The problem is your backticks are missing $ you need to fix the following two lines like so:
version1=$(echo $file | awk -F_ '{print $NF}' | cut -d. -f1-3)
version2=$(echo $file | awk -F_ '{print $NF}' | cut -d. -f4-)
This will fix the syntactical errors. The following line doesn't make much sense as $version hasn't been initialize yet:
version=$version$version2
Did you mean:
version="${version1}.${version2}"
A side note you are using the -E option with grep but you aren't using any extended regexp features, in fact you are doing a fixed string string search so -F is more appropriate. You probably also want to use the -q option to suppress the output from grep.
Personally I would do:
file="abc_def_APP_13.0.5.8"
echo "$file" | awk '/abc_def_APP/{print "Version found: "$0;
print $4,$5,$6;
print $7;
print $4,$5,$6,$7;
next}
{print "Version not found"}' FS='[_.]' OFS=.
If you just want the version number in the variable version then why not simply:
version=$(echo "$file" | grep -o '[0-9].*')
It can all be done in a single awk command and without additional cut command. Consider following command:
read version1 version2 < <(echo $file|awk -F "[_.]" '{
printf("%s.%s.%s ", $4, $5, $6); printf("%s", $7);
for (i=8; i<=NF; i++) printf(".%s", $i); print ""}')
echo "$version1 :: $version2"
OUTPUT
13.0.5 :: 8

Resources