I created two scripts to make a checksum of a lot of files, in one of them I use the 'find' command to search the files in a specific directory, but when the file has a name with space character, file one.txt for example, the script takes this as two files: file and one. I know that error is in the 'find' command line, but I don't know what I do wrong.
My script:
#!/bin/bash
find $1 -type f -exec bash ./task2.sh "{}" \;
In order to search for a file with spaces in the name you either have to enclose the argument in quotes or escape the spaces. Like:
find "test file.txt" or alternatively find test\ file.txt.
The easiest fix in your particular use case is to enclose $1 in quotes:
find "$1" -type f -exec bash ./task2.sh "{}" \;
Related
I want to replace some strings in my raw csv file for further use and I search for the internet and create the script so far. But it seems they doesn't work. Hope anyone can help me
The csv file is like this and I want to delete "^M" and "# Columns: " so that I can read my file.
# Task: bending1^M
# Frequency (Hz): 20^M
# Clock (millisecond): 250^M
# Duration (seconds): 120^M
# Columns: time,avg_rss12,var_rss12,avg_rss13,var_rss13,avg_rss23,var_rss23^M
#!/usr/bin/env bash
function scandir(){
cd `dirname $0`
echo `pwd`
local cur_dir parent_dir workir
workdir=$1
cd ${workdir}
if [ ${workdir}="/" ]
then
cur_dir=""
else
cur_dir=$(pwd)
fi
for dirlist in $(ls ${cur_dir})
do
if test -d ${dirlist}
then
cd ${dirlist}
scandir ${cur_dir}/${dirlist}
cd ..
else
vi ${cur_dir}/${dirlist} << EOF
:%s/\r//g
:%s/\#\ Columns:\ //g
:wq
EOF
fi
done
}
Your whole script looks like just:
find "$workdir" -type f | xargs -n1 sed -i -e 's/\r//g; s/^# Columns://'
Notes to your script:
Check your scripts for validity on https://www.shellcheck.net/
The of << EOF here document is invalid. The closing word EOF has to start from the beginning of the line inside the script:
vi ${cur_dir}/${dirlist} << EOF
:%s/\r//g
:%s/\#\ Columns:\ //g
:wq
EOF
#^^ no spaces in front of EOF, also no spaces/tabs after EOF
# the whole line needs to be exactly 'EOF'
There cannot be any spaces, tabs in front of it. Also, I don't think vi is not the best tool to run substitutions on a file, also I don't know how it acts with tabs or spaces infront of :. You may want to try to run it without whitespace characters in front of ::
vi ${cur_dir}/${dirlist} << EOF
:%s/\r//g
:%s/\#\ Columns:\ //g
:wq
EOF
Backticks ` are deprecated, less readable and don't allow for easy nesting. Use $( ... ) command substitution instead.
echo `pwd` is just invalid use of echo, just use pwd.
for dirlist in $(ls parsing ls output is bad. Use find command instead, or if you have to, shell globulation, ie. for dirlist in *.
if [ ${workdir}="/" ] is invalid. This tests if the string "${workdir}=/ is not null. Bash is space aware, it needs a space between = and operands. It should be if [ "${workdir}" = "/" ].
Always quote your variables. Don't cd ${dirlist} do cd "${dirlist}" and so on.
Well posted answer are corrects, but I would recommand this syntax:
find "$1" -type f -name '*.csv' -exec sed -e 's/\r$//;s/^# Columns: //' -i~ {} +
Using + instead of \; at end of find command will permit sed to work on many files at once, reducing forks and make whole job quicker.
The ~ after -i option will rename existing files by appending tilde at end of names instead of deleting them.
Using -type f will ensure working on files only (no symlinks, dirs, socket, fifos, devices...)
You can reduce the entire script to one command, and you do not have to use Vim to process the files:
find ${workdir} -name '*.csv' -exec sed -i -e 's/\r$//; /^#/d' '{}' \;
Explanation:
find <dir> -name <pattern> -exec <command> \; will search <dir> for files matchingand execute` on each file. You want to search for CSV files and do something with them (run a command on them).
The command run on every (CSV) file that was found will be sed -i -e 's/\r$//; /^#/d'. This means to edit the files in-place (-i) and run two transformations on them. s/\r$// will remove the ^M from each line, and /^#/d will remove all lines that start with a #.
'{}' is replaced with the files found by find and \; marks the end of the command run by find (see the find manual page for this).
Most of your script emulates part of the find command. That is not a good idea.
Also, for simple text processing, it is easier and faster to use sed instead of invoking an editor such as Vim.
I want to find all the occurrences of "getId" inside a directory which has subdirectories as follows:
*/*/*/*/*/*/myfile.gz
i tried thisfind -name *myfile.gz -print0 | xargs -0 zgrep -i "getId" but it didn't work. Can anyone tell me the best and simplest approach to get this?
find ./ -name '*gz' -exec zgrep -aiH 'getSorById' {} \;
find allows you to execute a command on the file using "-exe" and it replaces "{}" with the file name, you terminate the command with "\;"
I added "-H" to zgrep so it also prints out the file path when it has a match, as its helpful. "-a" treats binary files as text (since you might get tar-ed gzipped files)
Lastly, its best to quote your strings in case bash starts globbing them.
https://linux.die.net/man/1/grep
https://linux.die.net/man/1/find
Use the following find approach:
find . -name *myfile.gz -exec zgrep -ai 'getSORByID' {} \;
This will print all possible lines containing getSORByID substring
#!/bin/bash
if [[ "$#" -lt 3 ]]; then
echo "USAGE: enter the following arguments:
1st argument = starting directory path (e.g. . for working directory)
2nd argument = old file extension (e.g. txt)
3rd argument = new file extension (e.g. html
Note: Do not enter any . or \. for the 2nd and 3rd arguments
"
else
find "$1" -name *."$2"* -exec rename 's/\."${2}"$/."${3}"/' '{}' \;
fi
Example input:
bash rename_file_extensions.sh . txt html
Example output:
Use of uninitialized value $2 in regexp compilation at (eval 6) line 1.
line 1 contains the if statement above. Argument $2 is txt. So, I am confused as to what it is referring to.
I can run the following line and it works perfectly, but I want to have it accept bash arguments:
find . -name "*.txt.html*" -exec rename 's/\.txt.html.html.html$/.html/' '{}' \;
In this case, as you can see, there were a lot of incorrect file extensions. This line corrected all of them.
I tried editing the find line to:
find "${1}" -name '*."$2"*' -exec rename 's/\."${2}"$/."${3}"/' '{}' \;
This allowed me to move forward without an error; however, there was no output, and it did not change any txt extensions to html when I ran the following command, bash rename_file_extensions.sh . txt html.
Please help!
Thank you!
to get arguments interpreted by the shell, drop the single quotes:
try
find "$1" -name "*.$2*" -exec rename "s/\.${2}\$/.${3}/" '{}' \;
or (if you want to keep quotes)
find "$1" -name "*.$2*" -exec rename 's/\.'"${2}"'$/.'"${3}"'/' '{}' \;
(and I would enclose the full argument of -name in double quotes or bash could expand the argument directly if some files match in the current directory)
I've created a script that removes all zero-length files from a directory.
#!/bin/bash
find . -size 0 -type f -exec rm -i '{}' \;
It works well, except that it only works in the directory the script is actually located in and its sub-directories. I want to be able to use a directory as a command line argument (bash scriptname dirname) while executing the script and have it only search that directory and it's sub-directories, not the actual directory the script is located in. Is there a way to do this?
With $x you can access the x-th command line argument of your bash script. So in your case this should be something like
find $1 -size 0 -type f -exec rm -i '{}' \;
In Bash you can pass argument to your script. These argument can be used using $ sign. For example:
./hello 123 abc xyz
here $0 is your program name
$1 is 123 and so on
You can pass values and use them in your program from $1 to $9.
If your version of find accepts multiple paths, you can pass all the positional parameters like this:
find "$#" -size 0...
Note: you should not use $* for file names! it expands parameters into a string, so any file names with spaces or new lines in will break the command. "$#" preserves these, so is safe to use for this. If find doesn't accept multiple paths, you can loop over the parameters like this:
for dir in "$#"; do
find "$dir" -size 0...
done
I have found several similar questions that have solutions, except they don't involve variables.
I have a particular pattern in a tree of files and directories - the pattern is the word TEMPLATE. I want a script file to rename all of the files and directories by replacing the word TEMPLATE with some other name that is contained in the variable ${newName}
If I knew that the value of ${newName} was say "Fred lives here", then the command
find . -name '*TEMPLATE*' -exec bash -c 'mv "$0" "${0/TEMPLATE/Fred lives here}"' {} \;
will do the job
However, if my script is:
newName="Fred lives here"
find . -name '*TEMPLATE*' -exec bash -c 'mv "$0" "${0/TEMPLATE/${newName}}"' {} \;
then the word TEMPLATE is replaced by null rather than "Fred lives here"
I need the "" around $0 because there are spaces in the path name, so I can't do something like:
find . -name '*TEMPLATE*' -exec bash -c 'mv "$0" "${0/TEMPLATE/"${newName}"}"' {} \;
Can anyone help me get this script to work so that all files and directories that contain the word TEMPLATE have TEMPLATE replaced by whatever the value of ${newName} is
eg, if newName="A different name" and a I had directory of
/foo/bar/some TEMPLATE directory/with files then the directory would be renamed to
/foo/bar/some A different name directory/with files
and a file called some TEMPLATE file would be renamed to
some A different name file
You have two options.
1) The easiest solution is export newName. If you don't export the variable, then it's not available in subshells, and bash -c is a subshell. That's why you're getting TEMPLATE replaced by nothing.
2) Alternatively, you can try to construct a correctly quoted command line containing the replacement of $newName. If you knew that $newName were reasonably well-behaved (no double quotes or dollar signs, for example), then it's easy:
find . -name '*TEMPLATE*' \
-exec bash -c 'mv "$0" "${0/TEMPLATE/'"${newName}"'}"' {} \;
(Note: bash quoting is full of subtleties. The following has been edited several times, but I think it is now correct.)
But since you can't count on that, probably, you need to construct the command line by substituting both the filename and the substitution as command line parameters. But before we do that, let's fix the $0. You shouldn't be using $0 as a parameter. The correct syntax is:
bash -c '...$1...$1...' bash "argument"
Note the extra bash (many people prefer to use _); it's there to provide a sensible name for the subprocess.
So with that in mind:
find . -name '*TEMPLATE*' \
-exec bash -c 'mv "$1" "${1/TEMPLATE/$2}"' bash {} "$newName" \;
You an get around having to use quotes with IFS=$'\n' and since bash -c is a subshell an export of any variable is required. This works:
#!/bin/bash
IFS=$'\n'
export newName="Fred lives here"
find . -name '*TEMPLATE*' -exec bash -c 'mv "$0" "${0/TEMPLATE/${newName}}"' {} \;
If you do not mind two more lines and would like a script that is easier to read (no export required):
#!/bin/bash
IFS=$'\n'
newName="Fred lives here"
for file in $(find . -name '*TEMPLATE*'); do
mv ${file} ${file/TEMPLATE/${newName}}
done