i have a question about using shell to read files. That is to say, i have a folder like this:
folder
new_file.log (this is a file)
new_file2.log (this is a file)
new_file3.log (this is a file)
new (this is a subfolder)
new_file_from_subfolder.log
new_file2_from_subfolder.log
new_file3_from_subfolder.log
what i want is to read all the content from (direct) files, not files from the subfolder. In the above case, i need new_file.log to new_file3.log.
I know there is a simple way:
$ cat new*.log
but i also like to write a bash script:
for file in $(ls -a)
do
if [[ "$file" != "." && "$file" != ".." ]]; then
if [[ -f "$file" && "$file" == "^new" ]]; then **here is the problem**
[do something]
fi
fi
done
my problem is labeled as above. the bash code seems doesnot like
"$file" == ^new
if i run the bash, it basically does nothing, which means that files fail to meet the condition.
anything wrong?
[[ $foo = $bar ]] is a glob expression, not a regex; ^ has no special meaning there.
You probably want either the glob expression
[[ $file = new* ]]
or the regex
[[ $file =~ ^new ]]
Of course, in a real-world scenario, you'd just iterate only over the names that match your prefix:
for file in new*; do
: something with "$file"
done
...or, recursively (using FD 3 so you can still interact with the user in this code):
while IFS= read -u 3 -r -d '' file; do
: something with "$file"
done 3< <(find . -type f -name 'new*' -print0)
You're headed down the wrong track. Here's how to iterate over all regular files starting with new:
for file in new*
do
if [[ -f $file ]]
then
dosomething "$file"
fi
done
Related
I have some files:
tridiag_6_a.txt
tridiag_6_b.txt
tridiag_6_c.txt
gauss_6_a.txt
gauss_6_b.txt
and I want to get:
tridiag1.txt
tridiag2.txt
tridiag3.txt
gauss1.txt
gauss2.txt
How can I do this? (Mac OS terminal) I'm stuck on:
$ rename 's/^(.+?)_.*/$1$N.txt/g' *.txt
But this increments through all renames.
Use a Bash loop:
prev_prefix=""
count=1
for file in *.txt; do
[[ -f "$file" ]] || continue
prefix="${file/_*/}" # get all characters up to the first underscore
if [[ "$prev_previx" != "$prefix" ]]; then
count=1
prev_prefix="$prefix"
fi
mv "$file" "$prefix$count".txt
((count++))
done
for file in "$1"/*
do
if [ ! -d "${file}" ] ; then
if [[ $file == *.c ]]
then
blah
blah
Above code traverses all the .c files in a directory and does some action.I want to include .cpp,.h,.cc files as well.How can I check multiple file extensions in the same if condition ?
Thanks
You can combine conditions using boolean operators :
if [[ "$file" == *.c ]] || [[ "$file" == *.cpp ]] || [[ "$file" == *.h ]] || [[ "$file" == *.cc ]]; then
#...
fi
Another alternative would be to use a regex :
if [[ "$file" =~ \.(c|cpp|h|cc)$ ]]; then
#...
fi
Using extended patterns,
# Only necessary prior to bash 4.1; since then,
# extglob is temporarily turn on for the pattern argument to !=
# and =/== inside [[ ... ]]
shopt -s extglob nullglob
for file in "$1"/*; do
if [[ -f $file && $file = *.#(c|cc|cpp|h) ]]; then
...
fi
done
The extended pattern can also be to generate the file list; in that case, you definitely need the shopt command:
shopt -s extglob nullglob
for file in "$1"/*.#(c|cc|cpp|h); do
...
done
Why not just iterate over selected file extensions?
#!/bin/bash
for file in ${1}/*.[ch] ${1}/*.cc ${1}/*.cpp; do
if [ -f $file ]; then
# action for the file
echo $file
fi
done
I am new to bash scripting and I have to create this script that takes 3 directories as arguments and copies in the third one all the files in the first one that are NOT in the second one.
I did it like this:
#!/bin/bash
if [ -d $1 && -d $2 && -d $3 ]; then
for FILE in [ ls $1 ]; do
if ! [ find $2 -name $FILE ]; then
cp $FILE $3
done
else echo "Error: one or more directories are not present"
fi
The error I get when I try to execute it is: "line 7: syntax error near unexpected token `done' "
I don't really know how to make it work!
Also even if I'm using #!/bin/bash I still have to explicitly call bash when trying to execute, otherwise it says that executing is not permitted, anybody knows why?
Thanks in advance :)
Couple of suggestions :
No harm double quoting variables
cp "$FILE" "$3" # prevents wordsplitting, helps you filenames with spaces
for statement fails for the fundamental reason -bad syntax- it should've been:
for FILE in ls "$1";
But then, never parse ls output. Check [ this ].
for FILE in ls "$1"; #drastic
Instead of the for-loop in step2 use a find-while-read combination:
find "$1" -type f -print0 | while read -rd'' filename #-type f for files
do
#something with $filename
done
Use lowercase variable names for your script as uppercase variables are reserved for the system. Check [this].
Use tools like [ shellcheck ] to improve script quality.
Edit
Since you have mentioned the input directories contain only files, my alternative approach would be
[[ -d "$1" && -d "$2" && -d "$3" ]] && for filename in "$1"/*
do
[ ! -e "$2/${filename##*/}" ] && cp "$filename" "$3"
done
If you are baffled by ${filename##*/} check [ shell parameter expansion ].
Sidenote: In linux, although discouraged it not uncommon to have non-standard filenames like file name.
Courtesy: #chepner & #mklement0 for their comments that greatly improved this answer :)
Your script:
if ...; then
for ...; do
if ...; then
...
done
else
...
fi
Fixed structure:
if ...; then
for ...; do
if ...; then
...
fi # <-- missing
done
else
...
fi
If you want the script executable, then make it so:
$ chmod +x script.sh
Notice that you also have other problems in you script. It is better written as
dir1="$1"
dir2="$2"
dir3="$3"
for f in "$dir1"/*; do
if [ ! -f "$dir2/$(basename "$f")" ]; then
cp "$f" "$dir3"
fi
done
this is not totally correct:
for FILE in $(ls $1); do
< whatever you do here >
done
There is a big problem with that loop if in that folder there is a filename like this: 'I am a filename with spaces.txt'.
Instead of that loop try this:
for FILE in "$1"/*; do
echo "$FILE"
done
Also you have to close every if statement with fi.
Another thing, if you are using BASH ( #!/usr/bin/env bash ), it is highly recommended to use double brackets in your test conditions:
if [[ test ]]; then
...
fi
For example:
$ a='foo bar'
$ if [[ $a == 'foo bar' ]]; then
> echo "it's ok"
> fi
it's ok
However, this:
$ if [ $a == 'foo bar' ]; then
> echo "it's ok";
> fi
bash: [: too many arguments
You've forgot fi after the innermost if.
Additionally, neither square brackets nor find do work this way. This one does what your script (as it is now) is intended to on my PC:
#!/bin/bash
if [[ -d "$1" && -d "$2" && -d "$3" ]] ; then
ls -1 "$1" | while read FILE ; do
ls "$2/$FILE" >/dev/null 2>&1 || cp "$1/$FILE" "$3"
done
else echo "Error: one or more directories are not present"
fi
Note that after a single run, when $2 and $3 refer to different directories, those files are still not present in $2, so next time you run the script they will be copied once more despite they already are present in $3.
I have a bash script that requires a glob expression as a parameter. However I am having trouble using inputs as globs i.e say my input is
Shell_script '*.c'
and my code is iterating through an array of files and filtering them through pattern matching. In this case files which do not have the .c extension. (In this example, the first input could be any pattern otherwise)
count=${#array[#]}
for (( q = 0; q < count; q++ ));
do
if [[ ${array[q]} == $1 ]]; then
:
else unset array[q]
fi
done
.....
Any ideas?
Matching array contents against a glob is entirely possible:
#!/bin/bash
# this array has noncontiguous indexes to demonstrate a potential bug in the original code
array=( [0]="hello.c" [3]="cruel.txt" [5]="world.c" )
glob=$1
for idx in "${!array[#]}"; do
val=${array[$idx]}
if [[ $val = $glob ]]; then
echo "File $val matches glob expression $glob" >&2
else
echo "File $val does not match glob expression $glob; removing" >&2
unset array[$idx]
fi
done
Similarly, you can expand a glob against filesystem contents, though you'll want to clear IFS first to avoid string-splitting:
# here, the expectation is that your script would be invoked as: ./yourscript '*.c'
IFS=
for f in $1; do
[[ -e $f || -L $f ]] || { echo "No file matching $f found" >&2; }
echo "Iterating over file $f"
done
That said, in general, this is extremely unidiomatic, as opposed to letting the calling shell expand the glob before your script is started, and reading the list of matched files off your argument vector. Thus:
# written this way, your script can just be called ./yourscript *.c
for f; do
[[ -e $f || -L $f ]] || { echo "No file matching $f found" >&2; }
echo "Iterating over file $f"
done
You can loop over your list of files like this. If you run your script as
./test.sh "*.c". Then inside your script you can do:
for file in $1
do
#use your file
done
I have a bunch of images that I need to rename, so I can use them and I was wondering how to do this.
The way they need to be is that first 5 will be kept and then for the 6th I would write a number from 1-3. I only know that the first 5 are static; on pics belonging to same "family" and can be used for comparison and the 6th char is not known.
Example:
12345random.jpg
12345randomer.jpg
0987654more_random.jpg
09876awesome.jpg
09876awesomer.jpg
09876awesomest.jpg
09876soawesomegalaxiesexplode.jpg
would become.
12345.jpg
123452.jpg
09876.jpg
098761.jpg
098762.jpg
It would be cool if it would only handle the loop so that 3 pics could be only renamed and rest skipped.
I found some stuff on removing letters to certain point, but nothing that use, since I am quite poor at bash scripting.
Here is my approach, but it kind of sucks, since I tried modifying scripts I found, but the idea is there
//I could not figure how to remove the chars after 5th not the other way around
for file in .....*; do echo mv $file `echo $file | cut -c6-`; done
done
//problem also is that once the names conflict it produces only 1 file named 12345.jpg 2nd one will not be created
//do not know how to read file names to array
name=somefile
if [[ -e $name.jpg]] ; then
i=0
while [[ -e $name-$i.jpg]] ; do
let i++
done
name=$name-$i
fi
touch $name.jpg
You can have:
new_file=${file%%[^0-9]*.jpg}.jpg
As a concept you can have this to rename files:
for file in *.jpg; do
[[ $file == [0-9]*[^0-9]*.jpg ]] || continue ## Just a simple check.
new_file=${file%%[^0-9]*.jpg}.jpg
[[ -e $new_file ]] || continue ## Do not overwrite. Delete line if not wanted.
echo "Renaming $file to $new_file." ## Optional message.
mv -- "$file" "$new_file" || echo "Failed to rename $file to $new_file."
done
If you're going to process files that also contain directory names, you'll need some more changes:
for file in /path/to/other/dirs/*.jpg *.jpg; do
base=${file##*/}
[[ $base == [0-9]*[^0-9]*.jpg ]] || continue
if [[ $file == */* ]]; then
new_file=${file%/*}/${base%%[^0-9]*.jpg}.jpg
else
new_file=${file%%[^0-9]*.jpg}.jpg
fi
[[ -e $new_file ]] || continue
echo "Renaming $file to $new_file."
mv -- "$file" "$new_file"
done
you can also try the following code
but be careful all the files should be in .jpg format and pass the name of folder as an argument
#!/bin/bash
a=`ls $1`
for b in $a
do
echo $b
if (( i<4 ))
then
c=`echo $b | cut -c1-5`
let i=i+1
c="$c$i.jpg"
echo $c
else
c=`echo $b | cut -c1-5`
c="$c.jpg"
break
fi
mv $1$b $1$c
done