I have a directory that looks like this:
pages/
folder1/
folder1.filename1.txt
folder1.filename2.txt
folder2/
folder2.filename4.txt
folder2.filename5.txt
folder3/
filename6.txt
I want it to look like this:
pages/
folder1/
filename1.txt
filename2.txt
folder2/
filename3.txt
filename4.txt
folder3/
filename5.txt
With ls * | sed -e s/^[^.]*.// > /tmp/filenames.txt I get a file containing:
filename1.txt
filename2.txt
filename3.txt
filename4.txt
txt
How can I tell sed to ignore filenames of the form [filename].[suffix] and only look at filenames of the form [foldername].[filename].[suffix]?
The final script (as pointed out, the find command would simplify things, but this worked):
for folder in $(ls .)
do
if test -d $folder
then
pushd $folder
ls * | sed 's/.*\.\(.*\..*\)/\1/' > /tmp/filenames.txt
ls * > /tmp/current.txt
exec 3</tmp/current.txt
exec 4</tmp/filenames.txt
while read file <&3; read name <&4;
do
mv "$file" "$name"
done
rm /tmp/current.txt
rm /tmp/filenames.txt
popd
else
echo $folder "not a directory"
fi
done
exit 0
This page is now a community wiki. You can add more elegant solutions below:
for folder in $(ls .)
do
something better
Give this a try:
sed 's/.*\.\(.*\..*\)/\1/'
You should really use find then you wouldn't need the check for "-d folder" or the temp file and execs or the while loop.
You can avoid the temporary file by using process substition:
while read line
do
echo $line
done < <(ls)
Another item of interest: your system may already have a Perl script called rename or prename which will rename files using a regular expression.
You don't need to use sed:
ls * > /tmp/current.txt
exec 3</tmp/current.txt
while read file <&3;
do
replacement=${file#${folder}.}
if [ "$replacement" != "txt" ] ; then
mv "$file" "$replacement"
fi
done
use the following regex:
/\A(.\*?\\.){2,2}.+\Z/
Related
I have a small bug and don't know how to solve it. I want to copy files from a big folder with many files, where the files contain a specific string. For this I use grep, ack or (in this example) ag. When I'm inside the folder it matches without problem, but when I want to do it with a loop over the files in the following script it doesn't loop over the matches. Here my script:
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" | while read -d $'\0' file; do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done
SEARCH_QUERY holds the String I want to find inside the files, INPUT_DIR is the folder where the files are located, OUTPUT_DIR is the folder where the found files should be copied to. Is there something wrong with the while do?
EDIT:
Thanks for the suggestions! I took this one now, because it also looks for files in subfolders and saves a list with all the files.
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" > "output_list.txt"
while read file
do
echo "${file##*/}"
cp "${file}" "${OUTPUT_DIR}/${file##*/}"
done < "output_list.txt"
Better implement it like below with a find command:
find "${INPUT_DIR}" -name "*.*" | xargs grep -l "${SEARCH_QUERY}" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
or another option:
grep -l "${SEARCH_QUERY}" "${INPUT_DIR}/*.*" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
if you do not mind doing it in just one line, then
grep -lr 'ONE\|TWO\|THREE' | xargs -I xxx -P 0 cp xxx dist/
guide:
-l just print file name and nothing else
-r search recursively the CWD and all sub-directories
match these works alternatively: 'ONE' or 'TWO' or 'THREE'
| pipe the output of grep to xargs
-I xxx name of the files is saved in xxx it is just an alias
-P 0 run all the command (= cp) in parallel (= as fast as possible)
cp each file xxx to the dist directory
If i understand the behavior of ag correctly, then you have to
adjust the read delimiter to '\n' or
use ag -0 -l to force delimiting by '\0'
to solve the problem in your loop.
Alternatively, you can use the following script, that is based on find instead of ag.
while read file; do
echo "$file"
cp "$file" "$OUTPUT_DIR/$file"
done < <(find "$INPUT_DIR" -name "*$SEARCH_QUERY*" -print)
I have a folder with few images, all have the same name format:
some-random-name-min.jpg
another-random-name-min.jpg
and-another-random-name-min.jpg
I want to strip the last -min so following this answer
I tried this bash script:
#!/bin/bash
for filename in /home/al/domus-images/portfolio/white snow/*.jpg; do
mv $filname $(echo $filename | sed -e 's/....\.jpg$//');
done
but after running the script nothing happened..any idea what am I missing here?
Thanks
You can use find command to list the files and then to remove -min from file name follow below script.
for filename in `find "/home/al/domus-images/portfolio/white snow/" -name "*.jpg"`;
do
mv $filname $(echo $filename | sed 's/-min.jpg$/.jpg/');
done
This remove last "-min" from file name.
this could be an idea:
#!/bin/bash
for filename in *\-min.jpg
do
newFile=$(echo $filename |sed "s/\-min//g")
printf "before\t:filename[$filename]\tnewFile[$newFile]\n"
mv $filename $newFile
printf "after\t:filename[$filename]\tnewFile[$newFile]\n"
done
output
[shell] ➤ ./myMove
before :filename[and-another-random-name-min.jpg] newFile[and-another-random-name.jpg]
after :filename[and-another-random-name-min.jpg] newFile[and-another-random-name.jpg]
before :filename[another-random-name-min.jpg] newFile[another-random-name.jpg]
after :filename[another-random-name-min.jpg] newFile[another-random-name.jpg]
before :filename[some-random-name-min.jpg] newFile[some-random-name.jpg]
after :filename[some-random-name-min.jpg] newFile[some-random-name.jpg]
this *\-min.jpg is just to be sure to get only the file you need
This might be what you're looking for
#!/bin/bash
for filename in /home/al/domus-images/portfolio/white snow/*-min.jpg
do
mv ${filename} ${filename//-min.jpg/.jpg}
done
Can't get it simpler than this.
rename 's/\-min.jpg$/.jpg/' *-min.jpg
I want to create directories which names should correspond to a list of zipped files in the parent directory. Additionally I want to get rid of the file extension in the resulting name of the directory.
e.g. archive01.gz should result in a directory with name archive01
My script so far:
#!/bin/bash
for file in *.gz; do
echo $file | sed 's/.gz//' | mkdir
done
The error message is:
mkdir: missing operand
However,
echo $file | sed 's/.gz//'
results in the correct name for the directory. How do I pipe it to mkdir?
A better way to do this would be to use parameter substitution instead of a pipe/subshell:
#!/bin/bash
for file in *.gz; do
mkdir ${file%.gz}
done
Try this,
#!/bin/bash
for file in *.gz; do
echo "$file" | sed 's/\.gz//' | xargs mkdir
done
mkdir doesn't work with pipes. Try the following:
mkdir `echo $file | sed 's/.gz//'`
The ` evaluates and then replaces it with the answer. It's called command substitution and alternatively could also be written as:
mkdir $(echo $file | sed 's/.gz//')
You can use command substitution as
#!/bin/bash
for file in *.gz; do
mkdir $(echo $file | sed 's/.gz//')
done
OR
#!/bin/bash
for file in *.gz; do
mkdir `echo $file | sed 's/.gz//'`
done
I am trying to loop through all the files in a directory.
I want to do some stuff on each file (convert it to xml, not included in example), then write the file to a new directory structure.
for file in `find /home/devel/stuff/static/ -iname "*.pdf"`;
do
echo $file;
sed -e 's/static/changethis/' $file > newfile +".xml";
echo $newfile;
done
I want the results to be:
$file => /home/devel/stuff/static/2002/hello.txt
$newfile => /home/devel/stuff/changethis/2002/hello.txt.xml
How do I have to change my sed line?
If you need to rename multiple files, I would suggest to use rename command:
# remove "-n" after you verify it is what you need
rename -n 's/hello/hi/g' $(find /home/devel/stuff/static/ -type f)
or, if you don't have rename try this:
find /home/devel/stuff/static/ -type f | while read FILE
do
# modify line below to do what you need, then remove leading "echo"
echo mv $FILE $(echo $FILE | sed 's/hello/hi/g')
done
Are you trying to change the filename? Then
for file in /home/devel/stuff/static/*/*.txt
do
echo "Moving $file"
mv "$file" "${file/static/changethis}.xml"
done
Please make sure /home/devel/stuff/static/*/*.txt is what you want before using the script.
First, you have to create the name of the new file based on the name of the initial file. The obvious solution is:
newfile=${file/static/changethis}.xml
Second you have to make sure that the new directory exists or create it if not:
mkdir -p $(dirname $newfile)
Then you can do something with your file:
doSomething < $file > $newfile
I wouldn't do the for loop because of the possibility of overloading your command line. Command lines have a limited length, and if you overload it, it'll simply drop off the excess without giving you any warning. It might work if your find returns 100 file. It might work if it returns 1000 files, but it might fail if your find returns 1000 files and you'll never know.
The best way to handle this is to pipe the find into a while read statement as glenn jackman.
The sed command only works on STDIN and on files, but not on file names, so if you want to munge your file name, you'll have to do something like this:
$newname="$(echo $oldname | sed 's/old/new/')"
to get the new name of the file. The $() construct executes the command and puts the results of the command on STDOUT.
So, your script will look something like this:
find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
echo $file;
newfile="$(echo $file | sed -e 's/static/changethis/')"
newfile="$newfile.xml"
echo $newfile;
done
Now, since you're renaming the file directory, you'll have to make sure the directory exists before you do your move or copy:
find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
echo $file;
newfile="$(echo $file | sed -e 's/static/changethis/')"
newfile="$newfile.xml"
echo $newfile;
#Check for directory and create it if it doesn't exist
$dirname=$(dirname "$newfile")
if [ ! -d "$dirname" ]
then
mkdir -p "$dirname"
fi
#Directory now exists, so you can do the move
mv "$file" "$newfile"
done
Note the quotation marks to handle the case there's a space in the file name.
By the way, instead of doing this:
if [ ! -d "$dirname" ]
then
mkdir -p "$dirname"
fi
You can do this:
[ -d "$dirname"] || mkdir -p "$dirname"
The || means to execute the following command only if the test isn't true. Thus, if [ -d "$dirname" ] is a false statement (the directory doesn't exist), you run mkdir.
It's a fairly common shortcut when you see shell scripts.
find ... | while read file; do
newfile=$(basename "$file").xml;
do something to "$file" > "$somedir/$newfile"
done
OUTPUT="$(pwd)";
for file in `find . -iname "*.pdf"`;
do
echo $file;
cp $file $file.xml
echo "file created in directory = {$OUTPUT}"
done
This will create a new file with name whatyourfilename.xml, for hello.pdf the new file created would be hello.pdf.xml, basically it creates a new file with .xml appended at the end.
Remember the above script finds files in the directory /home/devel/stuff/static/ whose file names match the matcher string of the find command (in this case *.pdf), and copies it to your present working directory.
The find command in this particular script only finds files with filenames ending with .pdf If you wanted to run this script for files with file names ending with .txt, then you need to change the find command to this find /home/devel/stuff/static/ -iname "*.txt",
Once I wanted to remove trailing -min from my files. i.e. wanted alg-min.jpg to turn into alg.jpg. so after some struggle, managed to figure something like this:
for f in *; do echo $f; mv $f $(echo $f | sed 's/-min//g');done;
Hope this helps someone willing to REMOVE or SUBTITUDE some part of their file names.
I am trying to create a file that contains all of the code of an app. I have created a file called catlist.txt so that the files are added in the order I need them.
A snippet of my catlist.txt:
app/controllers/application_controller.rb
app/views/layouts/*
app/models/account.rb
app/controllers/accounts_controller.rb
app/views/accounts/*
When I run the command the files that are explicitly listed get added but the wildcard files do not.
cat catlist.txt|xargs cat > fullcode
I get
cat: app/views/layouts/*: No such file or directory
cat: app/views/accounts/*: No such file or directory
Can someone help me with this. If there is an easier method I am open to all suggestions.
Barb
Your problem is that xargs is not the shell, so the wildcard is being interpreted literally as an star. You'll need to have a shell to do the expansion for you like this:
cat catlist.txt | xargs -I % sh -c "cat %" > fullcode
Note that the * is not recursive in your data file. I assume that was what you meant. If you want the entries to be recursive, that's a little trickier and would need something more like DevNull's script, but that will require that you change your data file a bit to not include the stars.
Are you positive those directories exist?
The problem with doing a cat on a list like that (where you're using wildcards) is that the cat isn't recursive. It will only list the contents of that directory; not any subdirectories.
Here's what I would do:
#!/bin/bash.exe
output="out.txt"
if [ -f "$output" ]
then
rm $output
fi
for file in $(cat catlist.txt)
do
if [ -f "$file" ]
then
echo "$file is a file."
cat $file >> $output
elif [ -d "$file" ]
then
echo "$file is a directory."
find $file -type f -exec cat {} >> $output \;
else
echo "huh?"
fi
done
If the entry listed is a directory, it finds all files from that point on and cats them.
use a while read loop to read your file
while read -r file
do
if [ -f "$file" ]; then
yourcode "$file"
fi
# expand asterix
case "$file" in
*"*" )
for f in $file
do
yourcode "$f"
done
esac
done <"catlist.txt"