I have a directory with the following content:
/B7_001
/B7_002
B7_001_name.mat
B7_002_name.mat
Each subdirectory in this directory has the following structure:
/B7_001/results_activity/sham/task/
/B7_002/results_activity/sham/task/
I want to copy or move each file down the subdirectory tree of the subdirectory matching the first part of it's name.
For example copy file
"B7_001_name.mat" in to /B7_001/results_activity/sham/task/.
I haven’t had much success with the following code, so any tips would be appreciated. Thanks!
for i in B7_*
do
cp ${i}_name.mat /${i}/results_activity/sham/task/
done
Try to understand the following:
shopt -s nullglob # play it safe when using globs
for i in B7_*_name.mat; do
echo "i=$i"
d=${i%_name.mat}
echo "d=$d"
echo mv "$i" "/$d/results_activity/sham/task"
done
Run as is, it's harmless and will not perform any actions; only print some stuff to your terminal.
for i in B7_*_name.mat: that's a for loop where the variable i will successively take the filenames matching the glob B7_*_name.mat.
with this variable, we remove the _name.mat part using the shell parameter expansion:
d=${i%_name.mat}
which means that d is the expansion $i where the trailing (because of %) _name.mat is removed.
How about using find - while combination like below :
find . -type f -print0 | while read -r -d '' line
do
if [[ "$line" =~ ^\./B7_001 ]]
then
cp "$line" B7_001/results_activity/sham/task/
elif [[ "$line" =~ ^\./B7_002 ]]
then
cp "$line" B7_002/results_activity/sham/task/
fi
done
Related
I would want to iterate over contents of a directory and list only ordinary files.
The path of the directory is given as an user input. The script works if the input is current directory but not with others.
I am aware that this can be done using ls.. but i need to use a for .. in control structure.
#!/bin/bash
echo "Enter the path:"
read path
contents=$(ls $path)
for content in $contents
do
if [ -f $content ];
then
echo $content
fi
done
ls is only returning the file names, not including the path. You need to either:
Change your working directory to the path in question, or
Combine the path with the names for your -f test
Option #2 would just change:
if [ -f $content ];
to:
if [ -f "$path/$content" ];
Note that there are other issues here; ls may make changes to the output that break this, depending on wrapping. If you insist on using ls, you can at least make it (somewhat) safer with:
contents="$(command ls -1F "$path")"
You have two ways of doing this properly:
Either loop through the * pattern and test file type:
#!/usr/bin/env bash
echo "Enter the path:"
read -r path
for file in "$path/"*; do
if [ -f "$file" ]; then
echo "$file"
fi
done
Or using find to iterate a null delimited list of file-names:
#!/usr/bin/env bash
echo "Enter the path:"
read -r path
while IFS= read -r -d '' file; do
echo "$file"
done < <(
find "$path" -maxdepth 1 -type f -print0
)
The second way is preferred since it will properly handle files with special characters and offload the file-type check to the find command.
Use file, set to search for files (-type f) from $path directory:
find "$path" -type f
Here is what you could write:
#!/usr/bin/env bash
path=
while [[ ! $path ]]; do
read -p "Enter path: " path
done
for file in "$path"/*; do
[[ -f $file ]] && printf '%s\n' "$file"
done
If you want to traverse all the subdirectories recursively looking for files, you can use globstar:
shopt -s globstar
for file in "$path"/**; do
printf '%s\n' "$file"
done
In case you are looking for specific files based on one or more patterns or some other condition, you could use the find command to pick those files. See this post:
How to loop through file names returned by find?
Related
When to wrap quotes around a shell variable?
Why you shouldn't parse the output of ls
Is double square brackets [[ ]] preferable over single square brackets [ ] in Bash?
Given a folder (that my script get the of this folder as argument) , how can I creates array that will contain the names of all the files in this folder (and the files that exists at any folder in this folder and the other folder - recursively)?
I tried to do it like that :
#!/bin/bash
function get_all_the_files {
for i in "${1}"/*; do
if [ -d "$i" ]; then
get_all_the_files ${1}
else
if [ -f "${i}" ]; then
arrayNamesOfAllTheFiles=(${arrayNamesOfAllTheFiles[#]} "${i}")
fi
fi
done
}
arrayNamesOfAllTheFiles=()
get_all_the_files folder
declare -p arrayNamesOfAllTheFiles
But it's not working. What is the problem and how can I fix it?
To stick with your design (looping on the files and inserting only the regular files), populating the array at each step, but have Bash perform the recursion via the glob, you can use the following:
# the globstar shell option enables the ** glob pattern for recursion
shopt -s globstar
# the nullglob shell option makes non-matching globs expand to nothing (recommended)
shopt -s nullglob
array=()
for file in /path/to/folder/**; do
if [[ ! -h $file && -f $file ]]; then
array+=( "$file" )
fi
done
With the test [[ ! -h $file && -f $file ]] we test that the file is not a symbolic link and a regular file (without testing that the file is not a symbolic link, you would also have the symbolic links that resolve to a regular file).
You also learned about the array+=( "stuff" ) pattern to append to an array, instead of array=( "${array[#]}" "stuff" ).
Another possibility (with Bash ≥ 4.4 where the -d option of mapfile is implemented) and with GNU find (that supports the -print0 predicate):
mapfile -d '' array < <(find /path/to/folder -type f -print0)
You almost had it right. There is a small typo in the recursive call:
if [ -d "$i" ]; then
get_all_the_files ${1}
else
should be
if [ -d "$i" ]; then
get_all_the_files ${i}
else
I will add that use of arrays like this in bash is very unidiomatic. If you are trying to work with recursive trees of files, its more usual to use tools like find and xargs.
find . -type f -print0 | xargs -0 command-or-script-to-run-on-each-file
Currently learning some bash scripting and having an issue with a question involving listing all files in a given directory and stating if they are a file or directory. The issue I am having is that I only get either my current directory or if a specify a directory it will just say that it is a directory eg. /home/user/shell_scripts will return shell_scipts is a directory rather than the files contained within it.
This is what I have so far:
dir=$dir
for file in $dir; do
if [[ -d $file ]]; then
echo "$file is a directory"
if [[ -f $file ]]; then
echo "$file is a regular file"
fi
done
Your line:
for file in $dir; do
will expand $dir just to a single directory string. What you need to do is expand that to a list of files in the directory. You could do this using the following:
for file in "${dir}/"* ; do
This will expand the "${dir}/"* section into a name-only list of the current directory. As Biffen points out, this should guarantee that the file list wont end up with split partial file names in file if any of them contain whitespace.
If you want to recurse into the directories in dir then using find might be a better approach. Simply use:
for file in $( find ${dir} ); do
Note that while simple, this will not handle files or directories with spaces in them. Because of this, I would be tempted to drop the loop and generate the output in one go. This might be slightly different than what you want, but is likely to be easier to read and a lot more efficient, especially with large numbers of files. For example, To list all the directories:
find ${dir} -maxdepth 1 -type d
and to list the files:
find ${dir} -maxdepth 1 -type f
if you want to iterate into directories below, then remove the -maxdepth 1
This is a good use for globbing:
for file in "$dir/"*
do
[[ -d "$file" ]] && echo "$file is a directory"
[[ -f "$file" ]] && echo "$file is a regular file"
done
This will work even if files in $dir have special characters in their names, such as spaces, asterisks and even newlines.
Also note that variables should be quoted ("$file"). But * must not be quoted. And I removed dir=$dir since it doesn't do anything (except break when $dir contains special characters).
ls -F ~ | \
sed 's#.*/$#/& is a Directory#;t quit;s#.*#/& is a File#;:quit;s/[*/=>#|] / /'
The -F "classify" switch appends a "/" if a file is a directory. The sed code prints the desired message, then removes the suffix.
for file in $(ls $dir)
do
[ -f $file ] && echo "$file is File"
[ -d $file ] && echo "$file is Directory"
done
or replace the
$(ls $dir)
with
`ls $`
If you want to list files that also start with . use:
for file in "${dir}/"* "${dir}/"/.[!.]* "${dir}/"/..?* ; do
Could someone please help with this script. I need to use grep to loop to through the filenames that need to be changed.
#!/bin/bash
file=
for file in $(ls $1)
do
grep "^.old" | mv "$1/$file" "$1/$file.old"
done
bash can handle regular expressions without using grep.
for f in "$1"/*; do
[[ $f =~ \.old ]] && continue
# Or a pattern instead
# [[ $f == *.old* ]] && continue
mv "$f" "$f.old"
done
You can also move the name checking into the pattern itself:
shopt -s extglob
for f in "$1/"!(*.old*); do
mv "$f" "$f.old"
done
If I understand your question correctly, you want to make rename a file (i.e. dir/file.txt ==> dir/file.old) only if the file has not been renamed before. The solution is as follow.
#!/bin/bash
for file in "$1/"*
do
backup_file="${file%.*}.old"
if [ ! -e "$backup_file" ]
then
echo mv "$file" "$backup_file"
fi
done
Discussion
The script currently does not actual make back up, it only displays the action. Run the script once and examine the output. If this is what you want, then remove the echo from the script and run it again.
Update
Here is the no if solution:
ls "$1/"* | grep -v ".old" | while read file
do
echo mv "$file" "${file}.old"
done
Discussion
The ls command displays all files.
The grep command filter out those files that has the .old extension so they won't be displayed.
The while loop reads the file names that do not have the .old extension, one by one and rename them.
I have a problem with this script. The script is supposed to go trough all the files and all sub-directories and sub-files (recursively). If the file ends with the extension .txt i need to replace a char/word in the text with a new char/word and then copy it into a existing directory. The first argument is the directory i need to start the search, the second is the old char/word, third the new char/word and fourth the directory to copy the files to. The script goes trough the files but only does the replacement and copies the files from the original directory. Here is the script
#!/bin/bash
funk(){
for file in `ls $1`
do
if [ -f $file ]
then
ext=${file##*.}
if [ "$ext" = "txt" ]
then
sed -i "s/$2/$3/g" $file
cp $file $4
fi
elif [ -d $file ]
then
funk $file $2 $3 $4
fi
done
}
if [ $# -lt 4 ]
then
echo "Need more arg"
exit 2;
fi
cw=$1
a=$2
b=$3
od=$4
funk $cw $a $b $od
You're using a lot of bad practices here: lack of quotings, you're parsing the output of ls... all this will break as soon as a filename contains a space of other funny symbol.
You don't need recursion if you either use bash's globstar optional behavior, or find.
Here's a possibility with the former, that will hopefully show you better practices:
#!/bin/bash
shopt -s globstar
shopt -s nullglob
funk() {
local search=${2//\//\\/}
local replace=${3//\//\\/}
for f in "$1"/**.txt; do
sed -i "s/$search/$replace/g" -- "$f"
cp -nvt "$4" -- "$f"
done
}
if (($#!=4)); then
echo >&2 "Need 4 arguments"
exit 1
fi
funk "$#"
The same function funk using find:
#!/bin/bash
funk() {
local search=${2//\//\\/}
local replace=${3//\//\\/}
find "$1" -name '*.txt' -type f -exec sed -i "s/$search/$replace/g" -- {} \; -exec cp -nvt "$4" -- {} \;
}
if (($#!=4)); then
echo >&2 "Need 4 arguments"
exit 1
fi
funk "$#"
In cp I'm using
the -n switch: no clobber, so as to not overwrite an existing file. Use it if your version of mv supports it, unless you actually want to overwrite files.
the -v switch: verbose, will show you the moved files (optional).
the -t switch: -t followed by a directory tells to copy into this directory. It's a very good thing to use cp this way: imagine instead of giving an existing directory, you give an existing file: without this feature, this file will get overwritten several times (well, this will be the case if you omit the -n option)! with this feature the existing file will remain safe.
Also notice the use of --. If your cp and sed supports it (it's the case for GNU sed and cp), use it always! it means end of options now. If you don't use it and if a filename start with a hyphen, it would confuse the command trying to interpret an option. With this --, we're safe to put a filename that may start with a hyphen.
Notice that in the search and replace patterns I replaced all slashes / by their escaped form \/ so as not to clash with the separator in sed if a slash happens to appear in search or replace.
Enjoy!
As pointed out, looping over find output is not a good idea. It also doesn't support slashes in search&replace.
Check gniourf_gniourf's answer.
How about using find for that?
#!/bin/bash
funk () {
local dir=$1; shift
local search=$1; shift
local replace=$1; shift
local dest=$1; shift
mkdir -p "$dest"
for file in `find $dir -name *.txt`; do
sed -i "s/$search/$replace/g" "$file"
cp "$file" "$dest"
done
}
if [[ $# -lt 4 ]] ; then
echo "Need 4 arguments"
exit 2;
fi
funk "$#"
Though you might have files with the same names in the subdirectories, then those will be overwritten. Is that an issue in your case?