How to align this script output.
for instance in `find /bxp/*/*/*/prod/*/apache_*/httpd/htdocs/ -type f -name status.txt` ; do
echo "`hostname`: `ls -ltr | ${instance}` : `cat ${instance}`"
done
Output looks like:
r008abc, /bxp/xip/xip.pentaho-server_pentaho-server-assembly/pentaho.prod.jobengine/prod/xip.pentaho-server_web.partition_0.0.1/apache_5.3.3-2.2.
26/httpd/htdocs/status.txt, Web server is disabled
However i want the output be like:
r008abc| xip - xip.pentaho-server_web.partition_0.0.1 | Web server is disabled
xip - is nothing but the second column of the $instance - xip.pentaho-server_web.partition_0.0.1 is 6th column of the $instance. How can I achieve this. I tried awk command but it was not helpful. Your suggestion is appreciated.
Command I tried
for instance in `find /bxp/*/*/*/prod/*/apache_*/httpd/htdocs/ -type f -name status.txt` ; do
echo "`hostname`: `"ls -ltr | awk -F '/' '{print $3}"' ${instance}` : `cat ${instance}`"
done
something like this oneliner:
find /bxp/*/*/*/prod/*/apache_*/httpd/htdocs/ -type f -name status.txt | awk -F/ -v host=$(hostname) '{cmd="cat \047" $0 "\047"; if ((cmd | getline out) > 0){printf "%s| %s - %s | %s\n", host,$3, $7, out} close(cmd);}'
Explanation of awk-script:
awk -F/ # use / as field separator
-v host=$(hostname) # set var host
'{
cmd="cat \047" $0 "\047" # define cmd
if ((cmd | getline out) > 0) # read output of cmd
printf "%s| %s - %s | %s\n",host,$3,$7,out; # print formatted result
close(cmd);
}'
Related
Please, help me with one simple script on bash. I wrote a script to backup one server with many vhosts. This script works fine, except one config file. When script parses this file, in results it adds some extra symbols
Cycle of parsing
for n in `find /etc/nginx/conf.d/ -type f -name *.conf`
do
dir_for_bckp=`cat $n | awk '{ print $1,$2 }' | egrep "^root" | awk '{ print $2 }' | sed 's/\;//g'`
name_bckp=`echo $dir_for_bckp | sed 's/\//\_/g'`
tar -zcvf $tmp_dir/backup/$name_bckp.tar.gz $dir_for_bckp/ 1>/dev/null
done
Output for problematic file in debug mode
+ for n in '`find /etc/nginx/conf.d/ -type f -name *.conf`'
++ cat /etc/nginx/conf.d/gitlab.conf
++ awk '{ print $1,$2 }'
++ awk '{ print $2 }'
++ sed 's/\;//g'
++ egrep '^root'
+ dir_for_bckp=$'/home/git/gitlab/public\r'
++ sed 's/\//\_/g'
+ name_bckp=$'_home_git_gitlab_public\r'
My question - why dir_for_bckp=$'/home/git/gitlab/public\r' ? Must be dir_for_bckp=/home/git/gitlab/public
Parsing other config files is ok!
Example
+ for n in '`find /etc/nginx/conf.d/ -type f -name *.conf`'
++ cat /etc/nginx/conf.d/site-com.conf
++ awk '{ print $1,$2 }'
++ egrep '^root'
++ sed 's/\;//g'
++ awk '{ print $2 }'
+ dir_for_bckp=/home/site.com/html
++ sed 's/\//\_/g'
+ name_bckp=_home_site.com_html
If I manually parse problematic file, output also ok.
cat /etc/nginx/conf.d/gitlab.conf | awk '{ print $1,$2 }' | egrep "^root" | awk '{ print $2 }' | sed 's/\;//g'
/home/git/gitlab/public
It seems the config file was created with MSWindows line ends, i.e. each line ends with \r\n, not only \n. Use fromdos or dos2unix to fix the file.
I am trying to get the output for the echo -e command as shown below
Command used:
echo -e "cd \${2}\nfilesModifiedBetweenDates=\$(find . -type f -exec ls -l --time-style=full-iso {} \; | awk '{print \$6,\$NF}' | awk '{gsub(/-/,\"\",\$1);print}' | awk '\$1>= '$fromDate' && \$1<= '$toDate' {print \$2}' | tr \""\n"\" \""\;"\")\nIFS="\;" read -ra fileModifiedArray <<< "\$filesModifiedBetweenDates"\nfor fileModified in \${fileModifiedArray[#]}\ndo\n egrep -w "\$1" "\$fileModified" \ndone"
cd ${2}
Expected output:
cd ${2}
filesModifiedBetweenDates=$(find . -type f -exec ls -l --time-style=full-iso {} \; | awk '{print $6,$NF}' | awk '{gsub(/-/,"",$1);print}' | awk '$1>= '20140806' && $1<= '20140915' {print $2}' | tr "\n" ";")
IFS=; read -ra fileModifiedArray <<< $filesModifiedBetweenDates
for fileModified in ${fileModifiedArray[#]}
do
egrep -w $1 $fileModified
done
Original Ouput:
cd ${2}
filesModifiedBetweenDates=$(find . -type f -exec ls -l --time-style=full-iso {} \; | awk '{print $6,$NF}' | awk '{gsub(/-/,"",$1);print}' | awk '$1>= '20140806' && $1<= '20140915' {print $2}' | tr "n" ";")
IFS=; read -ra fileModifiedArray <<< $filesModifiedBetweenDates
for fileModified in ${fileModifiedArray[#]}
do
egrep -w $1 $fileModified
done
How can i handle "\" in this ?
For long blocks of text, it's much simpler to use a quoted here document than trying to embedded a multi-line string into a single argument to echo or printf.
cat <<"EOF"
cd ${2}
filesModifiedBetweenDates=$(find . -type f -exec ls -l --time-style=full-iso {} \; | awk '{print $6,$NF}' | awk '{gsub(/-/,"",$1);print}' | awk '$1>= '20140806' && $1<= '20140915' {print $2}' | tr "\n" ";")
IFS=; read -ra fileModifiedArray <<< $filesModifiedBetweenDates
for fileModified in ${fileModifiedArray[#]}
do
egrep -w $1 $fileModified
done
EOF
You'd better use printf to have a better control:
$ printf "tr %s %s\n" '"\n"' '";"'
tr "\n" ";"
As you see, we indicate the parameters within double quotes: printf "text %s %s" and then we define what content should be stored in this parameters.
In case you really have to use echo, then escape the \:
$ echo -e 'tr "\\n" ";"'
tr "\n" ";"
Interesting read: Why is printf better than echo?
I am looking for a command which helps me use if in awk and equates it to the current date.
A/B folder has files with different dates. I need to filter out files of the present day whenever script runs
A) Gives an output with all the dates,
date=`date +"%Y-%m-%d"`
s3cmd ls --recursive s3://A/B/ | grep A-B | grep .tar | awk '{ if ($1 -eq "$date" ) print $1" "$2 " " $3 " " $4 }' | sort -r
B)Replaces $1 which contains dates with "$date" to all of them
date=`date +"%Y-%m-%d"`
s3cmd ls --recursive s3://A/B/ | grep A-B | grep .tar | awk '{ if ($1 = "$date" ) print $1" "$2 " " $3 " " $4 }' | sort -r
C)Does not give any output. leaves blank
date=`date +"%Y-%m-%d"`
s3cmd ls --recursive s3://A/B/ | grep A-B | grep .tar | awk '{ if ($1 == "$date" ) print $1" "$2 " " $3 " " $4 }' | sort -r
if I remove "" it does not give me any output in all the cases.
The shell doesn't substitute variables inside double quotes. You should assign an awk variable from the shell variable. Also, the equality comparison is ==, not -eq or =.
awk -v date="$date" '$1 == date { print $1" "$2 " " $3 " " $4 }'
You don't really need awk for that. Just use find and say
find /path/to/search/ -type f ! -newermt $(date +"%Y-%m-%d")
$(..) is command substitution and what it will do is expand to current date in the format YYYY-MM-DD.
! -newermt is find option to look for files older than specified date
-type f will only look for files
awk is not shell. It is a completely separate tool with it's own syntax and capabilities. Therefore you should not expect to be able to use shell variables or shell syntax in an awk script. Try this:
s3cmd ls --recursive s3://A/B/ |
awk -v date="$date" -v OFS=" " '/A-B/ && /.tar/ && ($1 == date) { print $1, $2, $3, $4 }' |
sort -r
You probably actually meant \.tar instead of .tar though and as #jaypal said, this is a job for find, not ls piped to awk.
I need to find row Count and size of each TXT files.
It needs to search all the directories and just show result as :
FileName|Cnt|Size
ABC.TXT|230|23MB
Here is some code:
v_DIR=$1
echo "the directory to cd is "$1
x=`ls -l $0 | awk '{print $9 "|" $5}'`
y=`awk 'END {print NR}' $0`
echo $x '|' $y
Try something like
find -type f -name '*.txt' -exec bash -c 'lines=$(wc -l "$0" | cut -d " " -f1); size=$(du -h "$0" | cut -f1); echo "$0|$lines|$size"' {} \;
How to pass a parameter to awk to compare the string with pipe input?
For example, followings are used to filter files created before Aug 2011 under specific folder
#!/bin/bash
$FILTER_DIR=$1
# file date before it should be listed.
FILTER_BEFORE="2011-08"
# $6 in awk statement is date of file name($8)
find $1 -type f | \
sed 's/^/ls -l /g' | \
sh | \
awk ' if ( $6 le $FILTER_BEFORE ) { print $8 }'
The result list all files under $FILER_DIR without filtering.
It seems AWK didnot receive $FILTER_BEFORE from bash properly.
Any comment is appreciated!!
if using gawk, pass it as a parameter
find $1 -type f |
sed 's/^/ls -l /g' |
sh |
awk -v filter_before=${FILTER_BEFORE} '{ if ( $6 <= filter_before ) { print $8 } }'
You will need to use double quotes and escape the other AWK variables so they don't get interpreted by bash.
find $1 -type f | \
sed 's/^/ls -l /g' | \
sh | \
awk " if ( \$6 le \"$FILTER_BEFORE\" ) { print \$8 }"
Alternatively you can break out just the variable into double quotes so you can avoid escaping.
find $1 -type f | \
sed 's/^/ls -l /g' | \
sh | \
awk ' if ( $6 le "'"$FILTER_BEFORE"'" ) { print $8 }'
I'd go with:
touch -t 201107302359 30_july_2011
find . -type f ! -newer 30_july_2011
Or this (GNU find only):
find . -type f ! -newermt '2011-07-30 23:59'
Following statements seem work properly.
Thanks for everybody's help.
find $1 -type f | \
sed 's/^/ls -l /g' | \
sh | \
awk -v filter_before=${FILTER_BEFORE} '{ if ( $6 < filter_before ) { print $8 } }'