I have bash script which checks presence of certain files and that the content has a valid format. It uses variable prefixes so i can easily add/remove new files w/o the need of further adjustments.
Problem is that i need to run this on AIX servers where bash is not present. I've adjusted the script except the part with variable prefixes. After some attempts i am lost and have no idea how to properly migrate the following piece of code so it runs under sh ( $(echo ${!ifile_#}) ). Alternatively i have ksh or csh if plain sh is not an option.
Thank you in advance for any help/hints
#!/bin/sh
# Source files
ifile_one="/path/to/file/one.csv"
ifile_two="/path/to/file/two.csv"
ifile_three="/path/to/file/three.csv"
ifile_five="/path/to/file/four.csv"
min_columns='10'
existing_files=""
nonexisting_files=""
valid_files=""
invalid_files=""
# Check that defined input-files exists and can be read.
for input_file in $(echo ${!ifile_#})
do
if [ -r ${!input_file} ]; then
existing_files+="${!input_file} "
else
nonexisting_files+="${!input_file} "
fi
done
echo "$existing_files"
echo "$nonexisting_files"
# Check that defined input files have proper number of columns.
for input_file_a in $(echo "$existing_files")
do
check=$(grep -v "^$" $input_file_a | sed 's/[^;]//g' | awk -v min_columns="$min_columns" '{ if (length == min_columns) {print "OK"} else {print "KO"} }' | grep -i KO)
if [ ! -z "$check" ]; then
invalid_files+="${input_file_a} "
else
valid_files+="${input_file_a} "
fi
done
echo "$invalid_files"
echo "$valid_files"
Bash returns expected output (of the four ECHOes):
/path/to/file/one.csv /path/to/file/two.csv /path/to/file/three.csv
/path/to/file/four.csv
/path/to/file/three.csv
/path/to/file/one.csv /path/to/file/two.csv
ksh/sh throws:
./report.sh[14]: "${!ifile_#}": 0403-011 The specified substitution is not valid for this command.
Thanks #Benjamin W. and #user1934428 , ksh93 arrays are the answer.
So bellow code works for me as desired.
#!/bin/ksh93
typeset -A ifile
ifile[one]="/path/to/file/one.csv"
ifile[two]="/path/to/file/two.csv"
ifile[three]="/path/to/file/three.csv"
ifile[whatever]="/path/to/file/something.csv"
existing_files=""
nonexisting_files=""
for input_file in "${!ifile[#]}"
do
if [ -r ${ifile[$input_file]} ]; then
existing_files+="${ifile[$input_file]} "
else
nonexisting_files+="${ifile[$input_file]} "
fi
done
Related
I would like to use this code snippet to update a files date and time stamp using there file name:
Example file names:
2009.07.04-03.42.01.mov
2019.06.08-01.12.08.mov
I get the following error "The action “Run Shell Script” encountered an error: “touch: out of range or illegal time specification: [[CC]YY]MMDDhhmm[.SS]”
How would I modify this code snippet?
for if in "$#"
do
date_Time=$(echo "$if" | awk '{ print substr( $0, 1, length($0)-7 ) }' | sed 's/\.//g' | sed 's/-//')
touch -t "$date_Time" "$if"
done
UPDATE (01/05/2022):......
I would also like the code to work for the following filename formats...
And file names with no time info (time would default to 12pm):
2009.07.04.mov
2019.06.08.mov
And file names with description info:
2009.07.04-file-description.mov
2019.06.08-video-file info.mp4
2019.06.08-video-old-codec.avi
The error message suggests that you passed in file names which do not match your examples. Perhaps modify your code to display an error message if it is called with no files at all, and remove the path if it is passed files with directory names.
As an aside, if is a keyword, so you probably don't want to use it as a variable name, even though it is possible.
#!/bin/sh
if [ $# == 0 ]; then
echo "Syntax: $0 files ..." >&2
exit 1
fi
for f in "$#"
do
date_Time=$(echo "$f" | awk '{ sub(/.*\//, ""); gsub(/[^0-9]+/, ""); print substr( $0, 1, length($0)-7 ) }')
touch -t "$date_Time" "$if"
done
Notice also how I factored out the sed scripts; Awk can do everything sed can do, so I included the final transformation in the main script. (As an aside, sed 's/[-.]//g' would do both in one go; or you could do sed -e 's/\.//' -e 's/-//' with a single sed invocation.)
If you use Bash, you could simplify this further:
#!/bin/bash
if [ $# == 0 ]; then
echo "Syntax: $0 files ..." >&2
exit 1
fi
for f in "$#"
do
base=${f##*/}
dt=${base//[!0-9]/}
dt=${dt:0:12}
case $dt in
[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9])
touch -t "$dt" "$f";;
*) echo "$0: $f did not seem to contain a valid date ($dt)" >&2;;
esac
done
Notice also how the code now warns if it cannot extract exactly 14 digits from the file name. The parameter expansions are somewhat clumsy but a lot more efficient than calling Awk on each file name separately (and the Awk code wasn't particularly elegant or robust either anyway).
Quick and small bash function
setDateFileFromName() {
local _file _dtime
for _file ;do
_dtime="${_file%.*.mov}"
_dtime="${_dtime##*/}"
touch -t ${_dtime//[!0-9]/} "$_file"
done
}
Then
setDateFileFromName /path/to store dir/????.??.??-??.??.??.mov
Remark. This work with filenames formated as your sample. Any change in filename format will break this!
I am learning bash. I would like to get the return value and matched line by grep at once.
if cat 'file' | grep 'match_word'; then
match_by_grep="$(cat 'file' | grep 'match_word')"
read a b <<< "${match_by_grep}"
fi
In the code above, I used grep twice. I cannot think of how to do it by grep once. I am not sure match_by_grep is always empty even when there is no matched words because cat may output error message.
match_by_grep="$(cat 'file' | grep 'match_word')"
if [[ -n ${match_by_grep} ]]; then
# match_by_grep may be an error message by cat.
# So following a and b may have wrong value.
read a b <<< "${match_by_grep}"
fi
Please tell me how to do it. Thank you very much.
You can avoid the double use of grep by storing the search output in a variable and seeing if it is not empty.
Your version of the script without double grep.
#!/bin/bash
grepOutput="$(grep 'match_word' file)"
if [ ! -z "$grepOutput" ]; then
read a b <<< "${grepOutput}"
fi
An optimization over the above script ( you can remove the temporary variable too)
#!/bin/bash
grepOutput="$(grep 'match_word' file)"
[[ ! -z "$grepOutput" ]] && (read a b <<< "${grepOutput}")
Using double-grep once for checking if-condition and once to parse the search result would be something like:-
#!/bin/bash
if grep -q 'match_word' file; then
grepOutput="$(grep 'match_word' file)"
read a b <<< "${grepOutput}"
fi
When assigning a variable with a string containing a command expansion, the return code is that of the (rightmost) command being expanded.
In other words, you can just use the assignment as the condition:
if grepOutput="$(cat 'file' | grep 'match_word')"
then
echo "There was a match"
read -r a b <<< "${grepOutput}"
(etc)
else
echo "No match"
fi
Is this what you want to achieve?
grep 'match_word' file ; echo $?
$? has a return value of the command run immediately before.
If you would like to keep track of the return value, it will be also useful to have PS1 set up with $?.
Ref: Bash Prompt with Last Exit Code
The output comes from a command I run from our netscaler. It outputs the following ... One thing to note is that the middle two numbers change but the even/odd criteria is always on the last digit. We never have more than 2 digits, so we'll never hit 10.
WC-01-WEB1
WC-01-WEB4
WC-01-WEB3
WC-01-WEB5
WC-01-WEB8
I need to populate a file called "even" and "odds." If we're dealing with numbers I can figure it out, but having the number within a string is throwing me off.
Example code but I'm missing the part where I need to match the string.
if [ $even_servers -eq 0 ]
then
echo $line >> evenfile
else
echo $line >> oddfile
fi
This is a simple awk command:
awk '/[02468]$/{print > "evenfile"}; /[13579]$/{print > "oddfile"}' input.txt
There must be better way.
How about this version:
for v in `cat <my_file>`; do export type=`echo $v | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $v >> evenfile ; else echo $v >> oddfile; fi; done
I assume your list of servers is stored in the filename <my_file>. The basic idea is to tokenize on WEB using awk and process the chars after WEB to determine even-ness. Once this is known, we export the value to a variable type and use this to selectively dump to the appropriate file.
For the case when the name is the output of another command:
export var=`<another command>`; export type=`echo $var | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $var >> evenfile ; else echo $var >> oddfile; fi;
Replace <another command> with your perl script.
As always grep is your friend:
grep "[2468]$" input_file > evenfile
grep "[^2468]$" input_file > oddfile
I hope this helps.
having issue getting command to execute threw a function in a BASH script.
The command: [named -V|grep BIND|awk '{printf ($2);}'] works in a shell but will not set the output to a varable.
Desired output for $VER should be: 9.8.1-P1
I believe the issue is the |
However, I am receiving:
BIND 9.8.1-P1 built with '--prefix=/usr' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--sysconfdir=/etc/bind' '--localstatedir=/var' '--enable-threads' '--enable-largefile' '--with-libtool' '--enable-shared' '--enable-static' '--with-openssl=/usr' '--with-gssapi=/usr' '--with-gnu-ld' '--with-geoip=/usr' '--enable-ipv6' 'CFLAGS=-fno-strict-aliasing -DDIG_SIGCHASE -O2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro' 'CPPFLAGS=-D_FORTIFY_SOURCE=2'
if you have any info please let me know
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
function version {
if [ `builtin type -p $3` ]; then
VER=`$2`
if [[ -n $VER ]]; then
# echo "$VER" > $DIR/ver/$1
echo "VER=$VER"
PID=$(pidof $3)
if [[ -n "$PID" ]]; then
# echo "$PID" > $DIR/pid/$1
echo "PID=$PID"
fi
fi
else
echo "$1 not installed"
fi
}
version "bind" "named -V|grep BIND|awk '{printf ($2);}'" "named"
You want: VER=$(eval "$2") -- The quotes are very important to contain the eval'ed script as a single word.
You will also need to alter the 2nd argument:
"named -V|awk '/BIND/ {print \$2}'"
# ^^^
Without the backslash, the shell would see $2 inside double quotes and substitute it.
The grep is removed simply because it's not necessary: awk can search for patterns.
See BashFAQ #50 for a detailed discussion of why commands should not be stored in strings (and how and why this fails in practice), and BashFAQ #48 describing why eval in particular is error-prone.
A far safer approach is to store code in functions, and pass those functions by name:
get_named_version() { named -V | awk '/BIND/ {print $2}'; }
version bind get_named_version named
...will work correctly with your original function.
My bash script doesn't work the way I want it to:
#!/bin/bash
total="0"
count="0"
#FILE="$1" This is the easier way
for FILE in $*
do
# Start processing all processable files
while read line
do
if [[ "$line" =~ ^Total ]];
then
tmp=$(echo $line | cut -d':' -f2)
count=$(expr $count + 1)
total=$(expr $total + $tmp)
fi
done < $FILE
done
echo "The Total Is: $total"
echo "$FILE"
Is there another way to modify this script so that it reads arguments into $1 instead of $FILE? I've tried using a while loop:
while [ $1 != "" ]
do ....
done
Also when I implement that the code repeats itself. Is there a way to fix that as well?
Another problem that I'm having is that when I have multiple files hi*.txt it gives me duplicates. Why? I have files like hi1.txt hi1.txt~ but the tilde file is of 0 bytes, so my script shouldn't be finding anything.
What i have is fine, but could be improved. I appreciate your awk suggestions but its currently beyond my level as a unix programmer.
Strager: The files that my text editor generates automatically contain nothing..it is of 0 bytes..But yeah i went ahead and deleted them just to be sure. But no my script is in fact reading everything twice. I suppose its looping again when it really shouldnt. I've tried to silence that action with the exit commands..But wasnt successful.
while [ "$1" != "" ]; do
# Code here
# Next argument
shift
done
This code is pretty sweet, but I'm specifying all the possible commands at one time. Example: hi[145].txt
If supplied would read all three files at once.
Suppose the user enters hi*.txt;
I then get all my hi files read twice and then added again.
How can I code it so that it reads my files (just once) upon specification of hi*.txt?
I really think that this is because of not having $1.
It looks like you are trying to add up the totals from the lines labelled 'Total:' in the files provided. It is always a good idea to state what you're trying to do - as well as how you're trying to do it (see How to Ask Questions the Smart Way).
If so, then you're doing in about as complicated a way as I can see. What was wrong with:
grep '^Total:' "$#" |
cut -d: -f2 |
awk '{sum += $1}
END { print sum }'
This doesn't print out "The total is" etc; and it is not clear why you echo $FILE at the end of your version.
You can use Perl or any other suitable program in place of awk; you could do the whole job in Perl or Python - indeed, the cut work could be done by awk:
grep "^Total:" "$#" |
awk -F: '{sum += $2}
END { print sum }'
Taken still further, the whole job could be done by awk:
awk -F: '$1 ~ /^Total/ { sum += $2 }
END { print sum }' "$#"
The code in Perl wouldn't be much harder and the result might be quicker:
perl -na -F: -e '$sum += $F[1] if m/^Total:/; END { print $sum; }' "$#"
When iterating over the file name arguments provided in a shell script, you should use '"$#"' in place of '$*' as the latter notation does not preserve spaces in file names.
Your comment about '$1' is confusing to me. You could be asking to read from the file whose name is in $1 on each iteration; that is done using:
while [ $# -gt 0 ]
do
...process $1...
shift
done
HTH!
If you define a function, it'll receive the argument as $1. Why is $1 more valuable to you than $FILE, though?
#!/bin/sh
process() {
echo "doing something with $1"
}
for i in "$#" # Note use of "$#" to not break on filenames with whitespace
do
process "$i"
done
while [ "$1" != "" ]; do
# Code here
# Next argument
shift
done
On your problem with tilde files ... those are temporary files created by your text editor. Delete them if you don't want them to be matched by your glob expression (wildcard). Otherwise, filter them in your script (not recommended).