I need to run the same script multiple times in parallel with log management of each script execution
I need an optimal method please?
script name : script.sh
logs : log_execution1.log log_execution2.log ...
or
logs : log_execution-PID.log log_execution-PID.log ...
thanks
This can be done around a loop:
$ ntimes=42
$ for i in $(seq 1 $ntimes) ; do (script.sh > log_execution$i.log &) ; done
I did it like that :
it's work fine now.
while [ -e file.dat ]
do
./a.ksh
sleep 5
done
file : a.ksh
#!/bin/ksh
echo "Hello StackOverFlow" > log_execution-$$.log
exit 0
Result
-rw-r--r-- 1 aai aai 22 Apr 19 11:11 log_execution-27310.log
-rw-r--r-- 1 aai aai 0 Apr 19 11:11 log_execution-26005.log
-rw-r--r-- 1 aai aai 22 Apr 19 11:11 log_execution-27327.log
thank you for your help
Related
I have 2 bash scripts, one calling another but depending on how I call it, it does or does not display my ls command.
script2.sh
#!/bin/bash
function test() {
i=0
while IFS= read -r line; do
IFS=',' read -ra ITEM <<<"$line"
printf "\n[${i}] ${ITEM}"
((i = i + 1))
done <<<$(ls $1)
printf "\nPress any other keys to abort.\n\n"
read -p "Please enter your selection: " ANSWER
echo $ANSWER
}
script1a.sh WORKS
#!/bin/bash
. ./scripts/bash/script2.sh
PARAM='-lag'
(test $PARAM)
Returns:
[0] total 5
[1] drwxr-xr-x 1 1049089 0 Oct 29 09:10 .
[2] drwxr-xr-x 1 1049089 0 Oct 9 23:11 ..
[3] -rw-r--r-- 1 1049089 87 Jul 6 14:19 .eslintignore
[4] -rw-r--r-- 1 1049089 449 Jul 10 13:56 .forceignore
[5] drwxr-xr-x 1 1049089 0 Oct 29 09:11 .git
Press any other keys to abort.
Please enter your selection:
script1b.sh FAILS
#!/bin/bash
. ./scripts/bash/script2.sh
PARAM='-lag'
myanswer=$(test $PARAM)
Returns:
Please enter your selection:
Anyone knows why this odd behavior and how to get around it? Thanks in advance.
unfortunately I'm quite new at bash, and I want to write a script that will start in a main directory, and check all subdirectories one by one for the presence of certain files, and if those files are present, perform an operation on them. For now, I have written a simplified version to test whether I can do the first part (checking for the files in each directory). This code runs without any errors that I can tell, but it does not echo anything to say that it has successfully found the files which I know are there.
#!/bin/bash
runlist=(1 2 3 4 5 6 7 8 9)
for f in *; do
if [[ -d {$f} ]]; then
#if f is a directory then cd into it
cd "{$f}"
for b in $runlist; do
if [[ -e "{$b}.png" ]]; then
echo "Found {$b}"
#if the file exists then say so
fi
done
cd -
fi
done
'''
Welcome to stackoverflow.
The following will do the trick (a combination of find, array, and if then else):
# list of files we are looking for
runlist=(1 2 4 8 16 32 64 128)
#find each of above anywhere below current directory
# using -maxdepth 1 because, based on on your exam you want to look one level only
# if that's not what you want then take out -maxdepth 1 from the find command
for b in ${runlist[#]}; do
echo
PATH_TO_FOUND_FILE=`find . -name $b.png`
if [ -z "$PATH_TO_FOUND_FILE" ]
then
echo "nothing found" >> /dev/null
else
# You wanted a postive confirmation, so
echo found $b.png
# Now do something with the found file. Let's say ls -l: change that to whatever
ls -l $PATH_TO_FOUND_FILE
fi
done
Here is an example run:
mamuns-mac:stack foo$ ls -lR
total 8
drwxr-xr-x 4 foo 1951595366 128 Apr 11 18:03 dir1
drwxr-xr-x 3 foo 1951595366 96 Apr 11 18:03 dir2
-rwxr--r-- 1 foo 1951595366 652 Apr 11 18:15 find_file_and_do_something.sh
./dir1:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 8.png
./dir2:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 64.png
mamuns-mac:stack foo$ ./find_file_and_do_something.sh
found 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/1.png
found 8.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/8.png
found 64.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 ./dir2/64.png
It only prints the "else" statement for everything but I know for a fact the files exist that it's looking for. I've tried adapting some of the other answers but I thought this should definitely work.
Does anyone know what's wrong with my syntax?
# Contents of script
for ID_SAMPLE in $(cut -f1 metadata.tsv | tail -n +2);
do if [ -f ./output/${ID_SAMPLE} ]; then
echo Skipping ${ID_SAMPLE};
else
echo Processing ${ID_SAMPLE};
fi
done
Additional information
# Output directory
(base) -bash-4.1$ ls -lhS output/
total 170K
drwxr-xr-x 8 jespinoz tigr 185 Jan 3 16:16 ERR1701760
drwxr-xr-x 8 jespinoz tigr 185 Jan 17 18:03 ERR315863
drwxr-xr-x 8 jespinoz tigr 185 Jan 16 23:23 ERR599042
drwxr-xr-x 8 jespinoz tigr 185 Jan 17 00:10 ERR599072
drwxr-xr-x 8 jespinoz tigr 185 Jan 16 13:00 ERR599078
# Example of inputs
(base) -bash-4.1$ cut -f1 metadata.tsv | tail -n +2 | head -n 10
ERR1701760
ERR599078
ERR599079
ERR599070
ERR599071
ERR599072
ERR599073
ERR599074
ERR599075
ERR599076
# Output of script
(base) -bash-4.1$ bash test.sh | head -n 10
Processing ERR1701760
Processing ERR599078
Processing ERR599079
Processing ERR599070
Processing ERR599071
Processing ERR599072
Processing ERR599073
Processing ERR599074
Processing ERR599075
Processing ERR599076
# Checking a directory
(base) -bash-4.1$ ls -l ./output/ERR1701760
total 294
drwxr-xr-x 2 jespinoz tigr 386 Jan 15 21:00 checkpoints
drwxr-xr-x 2 jespinoz tigr 0 Jan 10 01:36 tmp
-f is for checking whether the name is a file, but all your names are directories. Use -d to check that.
if [ -d "./output/$ID_SAMPLE" ]
then
If you want to check whether the name exists with any type, use -e.
So I tried
ncftpls -l
which gives me a list
-rw-r--r-- 1 100 ftpgroup 3817084 Jan 29 15:50 1548773401.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817089 Jan 29 15:51 1548773461.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817083 Jan 29 15:52 1548773521.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817085 Jan 29 15:53 1548773582.tar.gz
-rw-r--r-- 1 100 ftpgroup 3817090 Jan 29 15:54 1548773642.tar.gz
But all I want is to check the timestamp (which is the name of the tar.gz)
How to only get the timestamp list ?
As requested, all I wanted to do is delete old backups, so awk was a good idea (at least it was effective) even it wasn't the right params. My method to delete old backup is probably not the best but it works
ncftpls *authParams* | (awk '{match($9,/^[0-9]+/, a)}{ print a[0] }') | while read fileCreationDate; do
VALIDITY_LIMIT="$((`date +%s`-600))"
a=$VALIDITY_LIMIT
b=$fileCreationDate
if [ $b -lt $a ];then
deleteFtpFile $b
fi
done;
You can use awk to only display the timestamps from the output like so:
ncftpls -l | awk '{ print $5 }'
I would like a Bash script that can take input from a file or stdin, much like grep, for example
$ cat hw.txt
Hello world
$ grep wor hw.txt
Hello world
$ echo 'Hello world' | grep wor
Hello world
$ grep wor <<< 'Hello world'
Hello world
all works beautifully. However with the following script
read b < "${1-/dev/stdin}"
echo $b
It fails if using a herestring
$ hw.sh hw.txt
Hello world
$ echo 'Hello world' | hw.sh
Hello world
$ hw.sh <<< 'Hello world'
/opt/a/hw.sh: line 1: /dev/stdin: No such file or directory
Using /dev/stdin in this manner can be problematic because you are attempting to get a handle to stdin using a name in the filesystem (/dev/stdin) rather than using the file descriptor which bash has already handed you as stdin (file descriptor 0).
Here's a small script for you to test:
#!/bin/bash
echo "INFO: Listing of /dev"
ls -al /dev/stdin
echo "INFO: Listing of /proc/self/fd"
ls -al /proc/self/fd
echo "INFO: Contents of /tmp/sh-thd*"
cat /tmp/sh-thd*
read b < "${1-/dev/stdin}"
echo "b: $b"
On my cygwin installation this produces the following:
./s <<< 'Hello world'
$ ./s <<< 'Hello world'
INFO: Listing of /dev
lrwxrwxrwx 1 austin None 15 Jan 23 2012 /dev/stdin -> /proc/self/fd/0
INFO: Listing of /proc/self/fd
total 0
dr-xr-xr-x 2 austin None 0 Mar 11 14:27 .
dr-xr-xr-x 3 austin None 0 Mar 11 14:27 ..
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 0 -> /tmp/sh-thd-1362969584
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 1 -> /dev/tty0
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 2 -> /dev/tty0
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 3 -> /proc/5736/fd
INFO: Contents of /tmp/sh-thd*
cat: /tmp/sh-thd*: No such file or directory
./s: line 12: /dev/stdin: No such file or directory
b:
What this output shows is that bash is creating a temporary file to hold your HERE document (/tmp/sh-thd-1362969584) and making it available on file descriptor 0, stdin. However, the temporary file has already been unlinked from the file system and so is not accessible by reference through a file system name such as /dev/stdin. You can get the contents by reading file descriptor 0, but not by trying to open /dev/stdin.
On Linux, the ./s script above gives the following, showing that the file has been unlinked:
INFO: Listing of /dev
lrwxrwxrwx 1 root root 15 Mar 11 09:26 /dev/stdin -> /proc/self/fd/0
INFO: Listing of /proc/self/fd
total 0
dr-x------ 2 austin austin 0 Mar 11 14:30 .
dr-xr-xr-x 7 austin austin 0 Mar 11 14:30 ..
lr-x------ 1 austin austin 64 Mar 11 14:30 0 -> /tmp/sh-thd-1362965400 (deleted) <---- /dev/stdin not found
lrwx------ 1 austin austin 64 Mar 11 14:30 1 -> /dev/pts/12
lrwx------ 1 austin austin 64 Mar 11 14:30 2 -> /dev/pts/12
lr-x------ 1 austin austin 64 Mar 11 14:30 3 -> /proc/10659/fd
INFO: Contents of /tmp/sh-thd*
cat: /tmp/sh-thd*: No such file or directory
b: Hello world
Change your script to use the stdin supplied, rather than trying to reference through /dev/stdin.
if [ -n "$1" ]; then
read b < "$1"
else
read b
fi
bash parses some file names (like /dev/stdin) specially, so that they are recognized even if they are not actually present in the file system. If your script doesn't have #!/bin/bash at the top, and /dev/stdin isn't in your file system, your script may be run using /bin/sh, which would expect /dev/stdin to actually be a file.
(This should, perhaps, not be an answer, but rather a comment to Austin's answer.)
$ cat ts.sh
read b < "${1-/dev/stdin}"
echo $b
$ ./ts.sh <<< 'hello world'
hello world
No problem for me. I'm using bash 4.2.42 on Mac OS X.
You got a typo here
read b < "${1-/dev/stdin}"
Try
read b < "${1:-/dev/stdin}"