I am running a shell script on my mac, and i am getting a "No Such file or directory.
The input is: the replacement_name, and the working dir.
The output is: changing all files in the directory from $file to $newfilename
#!/bin/sh
echo "-------------------------------------------------"
echo "Arguments:"
echo "Old File String: $1"
echo "New File Name Head: $2"
echo "Directory to Change: $3"
echo "-------------------------------------------------"
oldname="$1"
newname="$2"
abspath="$3"
echo "Updating all files in '$abspath' to $newname.{extension}"
for file in $(ls $abspath);
do
echo $file
echo $file | sed -e "s/$oldname/$newname/g"
newfilename=$("echo $file| sed -e \"s/$oldname/$newname/g\"")
echo "NEW FILE: $newfilename"
mv $abspath/$file $abspath/$newfilename
done
It seems that it doesnt like assigning the result of my 1-liner to a variable.
old_filename_string_template.dart
test_template.dart
./bulk_rename.sh: line 16: echo old_filename_string.dart| sed -e "s/old_filename_string/test/g": No such file or directory
NEW FILE:
Test Information:
mkdir /_temp_folder
touch old_filename_string_template.a old_filename_string_template.b old_filename_string_template.c old_filename_string1_template.a old_filename_string1_template.b old_filename_string1_template.c old_filename_string3_template.a old_filename_string3_template.b old_filename_string3_template.c
./convert.sh old_filename_string helloworld /_temp_folder
The double quotes here make the shell look for a command whose name (filename, alias, or function name) is the entire string between the quotes. Obviously, no such command exists.
> newfilename=$("echo $file| sed -e \"s/old_filename_string/$1/g\"")
Removing the double quotes inside the parentheses and the backslashes before the remaining ones will fix this particular error.
The construct $(command sequence) is called a command substitution; the shell effectively replaces this string with the standard output obtained by evaluating command sequence in a subshell.
Most of the rest of your script has much too few quotes; so it's really unclear why you added them here in particular. http://shellcheck.net/ is a useful service which will point out a few dozen more trivial errors. Briefly, anything which contains a file name should be between double quotes.
Try to put double quotes outside backticks/subtitutions (not INSIDE backticks/substitutions like $("..."))
newfilename="$(....)"
By the way, please consider to use the package perl rename which already does this bulk file rename very well, with Perl regex style which is easier to use. This perl rename command maybe alreay available in your (Mac) distro. See intro pages.
Related
Hello I am trying to get all files with Jane's name to a separate file called oldFiles.txt. In a directory called "data" I am reading from a list of file names from a file called list.txt, from which I put all the file names containing the name Jane into the files variable. Then I'm trying to test the files variable with the files in list.txt to ensure they are in the file system, then append the all the files containing jane to the oldFiles.txt file(which will be in the scripts directory), after it tests to make sure the item within the files variable passes.
#!/bin/bash
> oldFiles.txt
files= grep " jane " ../data/list.txt | cut -d' ' -f 3
if test -e ~data/$files; then
for file in $files; do
if test -e ~/scripts/$file; then
echo $file>> oldFiles.txt
else
echo "no files"
fi
done
fi
The above code gets the desired files and displays them correctly, as well as creates the oldFiles.txt file, but when I open the file after running the script I find that nothing was appended to the file. I tried changing the file assignment to a pointer instead files= grep " jane " ../data/list.txt | cut -d' ' -f 3 ---> files=$(grep " jane " ../data/list.txt) to see if that would help by just capturing raw data to write to file, but then the error comes up "too many arguments on line 5" which is the 1st if test statement. The only way I get the script to work semi-properly is when I do ./findJane.sh > oldFiles.txt on the shell command line, which is me essentially manually creating the file. How would I go about this so that I create oldFiles.txt and append to the oldFiles.txt all within the script?
The biggest problem you have is matching names like "jane" or "Jane's", etc. while not matching "Janes". grep provides the options -i (case insensitive match) and -w (whole-word match) which can tailor your search to what you appear to want without having to use the kludge (" jane ") of appending spaces before an after your search term. (to properly do that you would use [[:space:]]jane[[:space:]])
You also have the problem of what is your "script dir" if you call your script from a directory other than the one containing your script, such as calling your script from your $HOME directory with bash script/findJane.sh. In that case your script will attempt to append to $HOME/oldFiles.txt. The positional parameter $0 always contains the full pathname to the current script being run, so you can capture the script directory no matter where you call the script from with:
dirname "$0"
You are using bash, so store all the filenames resulting from your grep command in an array, not some general variable (especially since your use of " jane " suggests that your filenames contain whitespace)
You can make your script much more flexible if you take the information of your input file (e.g list.txt), the term to search for (e.g. "jane"), the location where to check for existence of the files (e.g. $HOME/data) and the output filename to append the names to (e.g. "oldFile.txt") as command line [positonal] parameters. You can give each default values so it behaves as you currently desire without providing any arguments.
Even with the additional scripting flexibility of taking the command line arguments, the script actually has fewer lines simply filling an array using mapfile (synonymous with readarray) and then looping over the contents of the array. You also avoid the additional subshell for dirname with a simple parameter expansion and test whether the path component is empty -- to replace with '.', up to you.
If I've understood your goal correctly, you can put all the pieces together with:
#!/bin/bash
# positional parameters
src="${1:-../data/list.txt}" # 1st param - input (default: ../data/list.txt)
term="${2:-jane}" # 2nd param - search term (default: jane)
data="${3:-$HOME/data}" # 3rd param - file location (defaut: ../data)
outfn="${4:-oldFiles.txt}" # 4th param - output (default: oldFiles.txt)
# save the path to the current script in script
script="$(dirname "$0")"
# if outfn not given, prepend path to script to outfn to output
# in script directory (if script called from elsewhere)
[ -z "$4" ] && outfn="$script/$outfn"
# split names w/term into array
# using the -iw option for case-insensitive whole-word match
mapfile -t files < <(grep -iw "$term" "$src" | cut -d' ' -f 3)
# loop over files array
for ((i=0; i<${#files[#]}; i++)); do
# test existence of file in data directory, redirect name to outfn
[ -e "$data/${files[i]}" ] && printf "%s\n" "${files[i]}" >> "$outfn"
done
(note: test expression and [ expression ] are synonymous, use what you like, though you may find [ expression ] a bit more readable)
(further note: "Janes" being plural is not considered the same as the singular -- adjust the grep expression as desired)
Example Use/Output
As was pointed out in the comment, without a sample of your input file, we cannot provide an exact test to confirm your desired behavior.
Let me know if you have questions.
As far as I can tell, this is what you're going for. This is totally a community effort based on the comments, catching your bugs. Obviously credit to Mark and Jetchisel for finding most of the issues. Notable changes:
Fixed $files to use command substitution
Fixed path to data/$file, assuming you have a directory at ~/data full of files
Fixed the test to not test for a string of files, but just the single file (also using -f to make sure it's a regular file)
Using double brackets — you could also use double quotes instead, but you explicitly have a Bash shebang so there's no harm in using Bash syntax
Adding a second message about not matching files, because there are two possible cases there; you may need to adapt depending on the output you're looking for
Removed the initial empty redirection — if you need to ensure that the file is clear before the rest of the script, then it should be added back, but if not, it's not doing any useful work
Changed the shebang to make sure you're using the user's preferred Bash, and added set -e because you should always add set -e
#!/usr/bin/env bash
set -e
files=$(grep " jane " ../data/list.txt | cut -d' ' -f 3)
for file in $files; do
if [[ -f $HOME/data/$file ]]; then
if [[ -f $HOME/scripts/$file ]]; then
echo "$file" >> oldFiles.txt
else
echo "no matching file"
fi
else
echo "no files"
fi
done
I am use a if command if filename then move file
this is what I have so far
filename=$*test*
if [ -f "filename" ]
then
mv filename archive/filename
else
echo "no filename exists"
fi
When I run the script I get "no filename exists"
How can I use a wildcard as a variable?
Your script has several problems.
filename=$*test*
$*test* is not valid bash syntax. The $ indicates a variable.
I am looking for test to be a wild card for a part of a filename
If that is the case, you need to drop the leading $. Or possibly what you meant was *${test}*.
mv filename archive/filename
This would move a file named "filename", which is likely not what you want. You probably want:
mv "$filename" archive/
If you are not renaming the file, you do not need to specify the name in the target.
If you change this line:
echo "no filename exists"
to:
echo "no $filename exists"
Include the dollar sign and you will see exactly what file the script tried to look for.
Some additional notes:
Include set -x at the start of your script. This will show you each command expanded so you can see what is actually happening.
Have a look at https://www.shellcheck.net/ which can pick up a lot of syntax errors.
I'm trying to make a script that would test whether my file is exactly as it should be, but I haven't been using bash before:
#!/bin/bash
./myfile <test.in 1>>test.out 2>>testerror.out
if cmp -s "test.out" "pattern.out"
then
echo "Test matches pattern"
else
echo "Test does not match pattern"
fi
if cmp -s "testerror.out" "pattern.err"
then
echo "Errors matches pattern"
else
echo "Errors does not match pattern"
fi
Can I write it in such way that after calling ./script.sh myfile pattern my scripts would run over all files named pattern*.in and check if myfile gives same files as pattern*.out and pattern*.err ? e.g there are files pattern1, pattern2, pattern4 and i want to run test for them, but not for pattern3 that doesn't exist.
Can I somehow go around creating new files? (Assuming i don't need them) If I were doing it from command line, I'd go with something like
< pattern.in ./myfile | diff -s ./pattern.out
but I have no idea how to write it in script file to make it work.
Or maybe i should just use rm everytime?
If I understand you correctly:
for infile in pattern*.in ; do
outfile="${infile%.in}.out"
errfile="${infile%.in}.err"
echo "Working on input $infile with output $outfile and error $errfile"
./myfile <"$infile" >>"$outfile" 2>>"$errfile"
# Your `if`..`fi` blocks here, referencing infile/outfile/errfile
done
The % replacement operator strips a substring off the end of a variable's value. So if $infile is pattern.in, ${infile%.in} is that without the trailing .in, i.e., pattern. The outfile and errfile assignments use this to copy the first part (e.g., pattern1) of the particular .in file being processed.
I'm having an error trying to find a way to replace a string in a directory path with another string
sed: Error tryning to read from {directory_path}: It's a directory
The shell script
#!/bin/sh
R2K_SOURCE="source/"
R2K_PROCESSED="processed/"
R2K_TEMP_DIR=""
echo " Procesando archivos desde $R2K_SOURCE "
for file in $(find $R2K_SOURCE )
do
if [ -d $file ]
then
R2K_TEMP_DIR=$( sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g' $file )
echo "directorio $R2K_TEMP_DIR"
else
# some code executes
:
fi
done
# find $R2K_PROCCESED -type f -size -200c -delete
i'm understanding that the rror it's in this line
R2K_TEMP_DIR=$( sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g' $file )
but i don't know how to tell sh that treats $file variable as string and not as a directory object.
If you want ot replace part of path name you can echo path name and take it to sed over pipe.
Also you must enable globbing by placing sed commands into double quotes instead of single and change separator for 's' command like that:
R2K_TEMP_DIR=$(echo "$file" | sed "s:$R2K_SOURCE:$R2K_PROCESSED:g")
Then you will be able to operate with slashes inside 's' command.
Update:
Even better is to remove useless echo and use "here is string" instead:
R2K_TEMP_DIR=$(sed "s:$R2K_SOURCE:$R2K_PROCESSED:g" <<< "$file")
First, don't use:
for item in $(find ...)
because you might overload the command line. Besides, the for loop cannot start until the process in $(...) finishes. Instead:
find ... | while read item
You also need to watch out for funky file names. The for loop will cough on all files with spaces in them. THe find | while will work as long as files only have a single space in their name and not double spaces. Better:
find ... -print0 | while read -d '' -r item
This will put nulls between file names, and read will break on those nulls. This way, files with spaces, tabs, new lines, or anything else that could cause problems can be read without problems.
Your sed line is:
R2K_TEMP_DIR=$( sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g' $file )
What this is attempting to do is edit your $file which is a directory. What you want to do is munge the directory name itself. Therefore, you have to echo the name into sed as a pipe:
R2K_TEMP_DIR=$(echo $file | sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g')
However, you might be better off using environment variable parameters to filter your environment variable.
Basically, you have a directory called source/ and all of the files you're looking for are under that directory. You simply want to change:
source/foo/bar
to
processed/foo/bar
You could do something like this ${file#source/}. The # says this is a left side filter and it will remove the least amount to match the glob expression after the #. Check the manpage for bash and look under Parameter Expansion.
This, you could do something like this:
#!/bin/sh
R2K_SOURCE="source/"
R2K_PROCESSED="processed/"
R2K_TEMP_DIR=""
echo " Procesando archivos desde $R2K_SOURCE "
find $R2K_SOURCE -print0 | while read -d '' -r file
do
if [ -d $file ]
then
R2K_TEMP_DIR="processed/${file#source/}"
echo "directorio $R2K_TEMP_DIR"
else
# some code executes
:
fi
done
R2K_TEMP_DIR="processed/${file#source/}" removes the source/ from the start of $file and you merely prepend processed/ in its place.
Even better, it's way more efficient. In your original script, the $(..) creates another shell process to run your echo in which then pipes out to another process to run sed. (Assuming you use loentar's solution). You no longer have any subprocesses running. The whole modification of your directory name is internal.
By the way, this should also work too:
R2K_TEMP_DIR="$R2K_PROCESSED/${file#$R2K_SOURCE}"
I just didn't test that.
I am attempting to rename several hundred folders to remove numbers that are prefixed to the folder name. I seem to have gotten most of the way there, but my script does not quite work yet. When I echo the commands created by the script instead of running those commands, everything looks fine. The script:
for file in *
do
echo mv "'"$file"'" $(echo "'"$file"'" | sed 's/[0-9]\{1,3\} //')
done
returns many lines of:
mv '123 filename' 'filename'
mv '99 another name' 'another name'
. . . etc.
If I take one of those lines and use on the command line, the folder is renamed appropriately. If I remove the echo to actually run those mv commands, though, they do not work. Instead, mv prints out the usage reminder that comes when you incorrectly enter the command in some way.
Why do the output commands work individually but not within the for loop? How can I correct this script?
Your quoting looks oddball. The variable should be in double quotes, no more and no less.
Furthermore, you should avoid the brittle $(echo ... | sed ...); this can be accomplished in pure shell:
for f in *; do
g=${f#[0-9]}; g=${g#[0-9]}; g=${g#[0-9]}
echo mv "$f" "${g# }"
done
Run with sh -x if you want to verify that the echo gets correct quoting. Then remove the echo to actually move.
The reason the output worked when copy/pasted is that the second level of quoting was actually necessary when copying the output from echo. A (risky, convoluted) workaround would have been to replace echo with eval.