I have 2 bash scripts, one calling another but depending on how I call it, it does or does not display my ls command.
script2.sh
#!/bin/bash
function test() {
i=0
while IFS= read -r line; do
IFS=',' read -ra ITEM <<<"$line"
printf "\n[${i}] ${ITEM}"
((i = i + 1))
done <<<$(ls $1)
printf "\nPress any other keys to abort.\n\n"
read -p "Please enter your selection: " ANSWER
echo $ANSWER
}
script1a.sh WORKS
#!/bin/bash
. ./scripts/bash/script2.sh
PARAM='-lag'
(test $PARAM)
Returns:
[0] total 5
[1] drwxr-xr-x 1 1049089 0 Oct 29 09:10 .
[2] drwxr-xr-x 1 1049089 0 Oct 9 23:11 ..
[3] -rw-r--r-- 1 1049089 87 Jul 6 14:19 .eslintignore
[4] -rw-r--r-- 1 1049089 449 Jul 10 13:56 .forceignore
[5] drwxr-xr-x 1 1049089 0 Oct 29 09:11 .git
Press any other keys to abort.
Please enter your selection:
script1b.sh FAILS
#!/bin/bash
. ./scripts/bash/script2.sh
PARAM='-lag'
myanswer=$(test $PARAM)
Returns:
Please enter your selection:
Anyone knows why this odd behavior and how to get around it? Thanks in advance.
Related
thank you so much for any advice and feedback on this matter.
This is is my situation:
I have a directory with several hundred files, all that start with foo* and end with *.txt. However, they differ in between the beginning and end with a unique identifier that is "Group#.#" and the files look like so:
foo.Group1.1.txt
foo.Group1.2.txt
foo.Group1.4.txt
foo.Group2.45.txt
.
.
.
foo.Group16.9.txt
The files begin with Group1 and end at Group 16. They are simple one column txt files, each file has several thousand lines. Each row is a number.
I want to so a series of concatenations with these files in which I concatenate all but those files with the "Group1" and then all the files except "Group1" and "Group2" and then all the files except "Group1", "Group2", and "Group3" and so on, until I am left with just the last Group: "Group16"
In order to do this I use a bash extended globbing expression with a negation syntax to concatenate all files except those that have "Group1" as their ID.
I make a directory "jacks" and output the concatenated file into a txt file within this subdirectory:
cat !(*Group1.*) > jacks/jackknife1.freqs.txt
I can then continue using this command, but adding "Group2" and "Group3" for subsequent concatenations.
cat !(*Group1.*|*Group2.*) > jacks/jackknife2.freqs.txt
cat !(*Group1.*|*Group2.*|*Group3.*) > jacks/jackknife3.freqs.txt
Technically, this works. And 16 Groups isn't too terrible to do this manually.
But I am wondering if there is a way, perhaps using loops or bash scripting to automate this process and speed it up?
I would appreciate any advice or leads on this question!
thank you very much,
daniela
Some tries around bash globbing
Try using echo before cat !
touch foo.Group{1..3}.{1..5}.txt
ls -l
total 0
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group1.1.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group1.2.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group1.3.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group1.4.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group1.5.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group2.1.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group2.2.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group2.3.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group2.4.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group2.5.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group3.1.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group3.2.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group3.3.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group3.4.txt
-rw-r--r-- 1 user user 0 Oct 21 18:37 foo.Group3.5.txt
Then
echo !(*Group1.*)
foo.Group2.1.txt foo.Group2.2.txt foo.Group2.3.txt foo.Group2.4.txt foo.Group2.5.txt foo.Group3.1.txt foo.Group3.2.txt foo.Group3.3.txt foo.Group3.4.txt foo.Group3.5.txt
Ok, and
echo !(*Group[23].*)
foo.Group1.1.txt foo.Group1.2.txt foo.Group1.3.txt foo.Group1.4.txt foo.Group1.5.txt
Or
echo !(*Group*(1|3).*)
foo.Group2.1.txt foo.Group2.2.txt foo.Group2.3.txt foo.Group2.4.txt foo.Group2.5.txt
Or even
echo !(*Group*(1|*.3).*)
foo.Group2.1.txt foo.Group2.2.txt foo.Group2.4.txt foo.Group2.5.txt foo.Group3.1.txt foo.Group3.2.txt foo.Group3.4.txt foo.Group3.5.txt
and
echo !(*Group*(1|*.[2-4]).*)
foo.Group2.1.txt foo.Group2.5.txt foo.Group3.1.txt foo.Group3.5.txt
I will let you think about last two sample! ;-)
Any resources or advice would help, since I am pretty rubbish at scripting
So, I need to go to this path: /home/client/data/storage/customer/data/2020/09/15
And check to see if there are 5 or more consecutive files that contain the word "REJECTED":
ls -ltr
-rw-rw-r-- 1 root root 5059 Sep 15 00:05 customer_rlt_20200915000514737_20200915000547948_8206b49d-b585-4360-8da0-e90b8081a399.zip
-rw-rw-r-- 1 root root 5023 Sep 15 00:06 customer_rlt_20200915000547619_20200915000635576_900b44dc-1cf4-4b1b-a04f-0fd963591e5f.zip
-rw-rw-r-- 1 root root 39856 Sep 15 00:09 customer_rlt_20200915000824108_20200915000908982_b87b01b3-a5dc-4a80-b19d-14f31ff667bc.zip
-rw-rw-r-- 1 root root 39719 Sep 15 00:09 customer_rlt_20200915000901688_20200915000938206_38261b59-8ebc-4f9f-9e2d-3e32eca3fd4d.zip
-rw-rw-r-- 1 root root 12829 Sep 15 00:13 customer_rlt_20200915001229811_20200915001334327_1667be2f-f1a7-41ae-b9ca-e7103d9abbf8.zip
-rw-rw-r-- 1 root root 12706 Sep 15 00:13 customer_rlt_20200915001333922_20200915001357405_609195c9-f23a-4984-936f-1a0903a35c07.zip
Example of rejected file:
customer_rlt_20200513202515792_20200513202705506_5b8deae0-0405-413c-9a81-d1cc2171fa51REJECTED.zip
What I have so far:
!/bin/bash
YYYY=$(date +%Y);
MM=$(date +%m)
DD=$(date +%d)
#Set constants
CODE_OK=0
CODE_WARN=1
CODE_CRITICAL=2
CODE_UNKNOWN=3
#Set Default Values
FILE="/home/client/data/storage/customer/data/${YYYY}/${MM}/{DD}"
if [ ! -f $FILE ]
then
echo "NO TRANSACTIONS FOUND"
exit $CODE_CRITICAL
fi
You can do something quick in AWK:
$ cat consec.awk
/REJECTED/ {
if (match_line == NR - 1) {
consecutives++
} else {
consecutives = 1
}
if (consecutives == 5) {
print "5 REJECTED"
exit
}
match_line = NR
}
$ touch 1 2REJECTED 3REJECTED 5REJECTED 6REJECTED 7REJECTED 8
$ ls -1 | awk -f consec.awk
5 REJECTED
$ rm 3REJECTED; touch 3
$ ls -1 | awk -f consec.awk
$
This works by matching line containing REJECTED, counting consecutive lines (checked with match_line == NR - 1, which means "the last matching line was the previous line") and printing "5 REJECTED" if the number of consecutive lines is 5.
I've used ls -1 (note digit 1, not letter l) to sort by filename in this example. You could use ls -1rt (digit 1 again) to sort by file modification time, as in your original post.
unfortunately I'm quite new at bash, and I want to write a script that will start in a main directory, and check all subdirectories one by one for the presence of certain files, and if those files are present, perform an operation on them. For now, I have written a simplified version to test whether I can do the first part (checking for the files in each directory). This code runs without any errors that I can tell, but it does not echo anything to say that it has successfully found the files which I know are there.
#!/bin/bash
runlist=(1 2 3 4 5 6 7 8 9)
for f in *; do
if [[ -d {$f} ]]; then
#if f is a directory then cd into it
cd "{$f}"
for b in $runlist; do
if [[ -e "{$b}.png" ]]; then
echo "Found {$b}"
#if the file exists then say so
fi
done
cd -
fi
done
'''
Welcome to stackoverflow.
The following will do the trick (a combination of find, array, and if then else):
# list of files we are looking for
runlist=(1 2 4 8 16 32 64 128)
#find each of above anywhere below current directory
# using -maxdepth 1 because, based on on your exam you want to look one level only
# if that's not what you want then take out -maxdepth 1 from the find command
for b in ${runlist[#]}; do
echo
PATH_TO_FOUND_FILE=`find . -name $b.png`
if [ -z "$PATH_TO_FOUND_FILE" ]
then
echo "nothing found" >> /dev/null
else
# You wanted a postive confirmation, so
echo found $b.png
# Now do something with the found file. Let's say ls -l: change that to whatever
ls -l $PATH_TO_FOUND_FILE
fi
done
Here is an example run:
mamuns-mac:stack foo$ ls -lR
total 8
drwxr-xr-x 4 foo 1951595366 128 Apr 11 18:03 dir1
drwxr-xr-x 3 foo 1951595366 96 Apr 11 18:03 dir2
-rwxr--r-- 1 foo 1951595366 652 Apr 11 18:15 find_file_and_do_something.sh
./dir1:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 8.png
./dir2:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 64.png
mamuns-mac:stack foo$ ./find_file_and_do_something.sh
found 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/1.png
found 8.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/8.png
found 64.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 ./dir2/64.png
I have a folder with such contents
nass#starmaze:~/audio_setup/scripts$ ls -l ../jmess/
total 32
-rw-rw-r-- 1 nass users 1573 Νοέ 16 2014 jmess_fxio56-78feedsHDA-play12.jmess
-rw-rw-r-- 1 nass nass 1573 Δεκ 13 2014 jmess_pb-2.jmess
-rw-rw-r-- 1 nass nass 1573 Δεκ 20 2014 jmess_pb-3.jmess
-rw-rw-r-- 1 nass nass 1939 Ιούν 12 13:05 jmess_starmazeOnMaster.jmess
-rw-rw-r-- 1 nass nass 2163 Δεκ 15 2014 jmess_starmazeOnMaster.jmess.bak1-art
-rw-rw-r-- 1 nass nass 2161 Δεκ 15 2014 jmess_starmazeOnMaster.jmess.bak2-bcr
-rw-rw-r-- 1 nass nass 2389 Δεκ 22 2014 jmess_starmazeOnMaster.jmess.bak3-hoo
-rw-rw-r-- 1 nass nass 2163 Δεκ 15 2014 jmess_starmazeOnMaster.jmess.bak4-dsp
I want to be able to pick up the newest file, but only from the subset of files that do not contain the word "Master" in them. And I want to put that in a bash script.
So this
ls -t1 "${JCMESS_FOLDER}" | head -n1
provides the newest file in the folder , while this
ls -t1 "${JCMESS_FOLDER}"/!(*Master*) | head -n1
provides the newest file among the subset that I am interested in.
However, when I place the latter in a bash script as
$NEWEST_JCMESS_FILE=$( ls -t1 "${JCMESS_FOLDER}"/!(*Master*) | head -n1 )
it does not work:
./06.load_jcmess: command substitution: line 8: syntax error near unexpected token `('
./06.load_jcmess: command substitution: line 8: ` ls -t1 "${JCMESS_FOLDER}"/!(*Master*) | head -n1 )'
I am not sure what is wrong in this case and I ahve not been able to successfully find an answer for this.
thank you in advance for your help
This is BashFAQ #3:
newest() {
local candidate result=$1; shift # start with first argument as candidate
[[ -e $result ]] || return # handle case where nothing matched
for candidate; do # for loop default behavior is to loop over "$#"
[[ $candidate -nt $result ]] && result=$candidate
done
printf '%s\n' "$result"
}
shopt -s extglob # enable extglobs, ie. !(...)
newest_file=$(newest "$JCMESS_FOLDER"/!(*Master*))
I would like a Bash script that can take input from a file or stdin, much like grep, for example
$ cat hw.txt
Hello world
$ grep wor hw.txt
Hello world
$ echo 'Hello world' | grep wor
Hello world
$ grep wor <<< 'Hello world'
Hello world
all works beautifully. However with the following script
read b < "${1-/dev/stdin}"
echo $b
It fails if using a herestring
$ hw.sh hw.txt
Hello world
$ echo 'Hello world' | hw.sh
Hello world
$ hw.sh <<< 'Hello world'
/opt/a/hw.sh: line 1: /dev/stdin: No such file or directory
Using /dev/stdin in this manner can be problematic because you are attempting to get a handle to stdin using a name in the filesystem (/dev/stdin) rather than using the file descriptor which bash has already handed you as stdin (file descriptor 0).
Here's a small script for you to test:
#!/bin/bash
echo "INFO: Listing of /dev"
ls -al /dev/stdin
echo "INFO: Listing of /proc/self/fd"
ls -al /proc/self/fd
echo "INFO: Contents of /tmp/sh-thd*"
cat /tmp/sh-thd*
read b < "${1-/dev/stdin}"
echo "b: $b"
On my cygwin installation this produces the following:
./s <<< 'Hello world'
$ ./s <<< 'Hello world'
INFO: Listing of /dev
lrwxrwxrwx 1 austin None 15 Jan 23 2012 /dev/stdin -> /proc/self/fd/0
INFO: Listing of /proc/self/fd
total 0
dr-xr-xr-x 2 austin None 0 Mar 11 14:27 .
dr-xr-xr-x 3 austin None 0 Mar 11 14:27 ..
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 0 -> /tmp/sh-thd-1362969584
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 1 -> /dev/tty0
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 2 -> /dev/tty0
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 3 -> /proc/5736/fd
INFO: Contents of /tmp/sh-thd*
cat: /tmp/sh-thd*: No such file or directory
./s: line 12: /dev/stdin: No such file or directory
b:
What this output shows is that bash is creating a temporary file to hold your HERE document (/tmp/sh-thd-1362969584) and making it available on file descriptor 0, stdin. However, the temporary file has already been unlinked from the file system and so is not accessible by reference through a file system name such as /dev/stdin. You can get the contents by reading file descriptor 0, but not by trying to open /dev/stdin.
On Linux, the ./s script above gives the following, showing that the file has been unlinked:
INFO: Listing of /dev
lrwxrwxrwx 1 root root 15 Mar 11 09:26 /dev/stdin -> /proc/self/fd/0
INFO: Listing of /proc/self/fd
total 0
dr-x------ 2 austin austin 0 Mar 11 14:30 .
dr-xr-xr-x 7 austin austin 0 Mar 11 14:30 ..
lr-x------ 1 austin austin 64 Mar 11 14:30 0 -> /tmp/sh-thd-1362965400 (deleted) <---- /dev/stdin not found
lrwx------ 1 austin austin 64 Mar 11 14:30 1 -> /dev/pts/12
lrwx------ 1 austin austin 64 Mar 11 14:30 2 -> /dev/pts/12
lr-x------ 1 austin austin 64 Mar 11 14:30 3 -> /proc/10659/fd
INFO: Contents of /tmp/sh-thd*
cat: /tmp/sh-thd*: No such file or directory
b: Hello world
Change your script to use the stdin supplied, rather than trying to reference through /dev/stdin.
if [ -n "$1" ]; then
read b < "$1"
else
read b
fi
bash parses some file names (like /dev/stdin) specially, so that they are recognized even if they are not actually present in the file system. If your script doesn't have #!/bin/bash at the top, and /dev/stdin isn't in your file system, your script may be run using /bin/sh, which would expect /dev/stdin to actually be a file.
(This should, perhaps, not be an answer, but rather a comment to Austin's answer.)
$ cat ts.sh
read b < "${1-/dev/stdin}"
echo $b
$ ./ts.sh <<< 'hello world'
hello world
No problem for me. I'm using bash 4.2.42 on Mac OS X.
You got a typo here
read b < "${1-/dev/stdin}"
Try
read b < "${1:-/dev/stdin}"