I have 224 pdf files and I'd like to prefix the files with a number and _
Example:
stackoverflow_nov_2014.pdf
File accounts.csv contains:
2567,stackoverflow
So the goal is to take 2567 and prefix it to the pdf file with an underscore:
2567_stackoverflow_nov_2014.pdf
I think I would want to use read -r in a while loop as explained here:
https://unix.stackexchange.com/questions/147569/rename-a-batch-of-files-after-reading-from-a-source-file
But when attempting this as it's written, the shell gives usage of mv command and nothing changes with the files.
Edit: Adding sample data from sources file (accounts.csv)
11,My_Golf_Shop
2567,stackoverflow
11122,Test_Store
By the way, the sources file (accounts.csv) isn't in the same order as the files in the directory as accounts.csv so somehow there would need to be matching with file name and the accounts.csv that occurs.
Below is the script that should work under the assumption:
1. All the files are under the same folder
2. For a particular prefix, this script will only rename the first found file.
#!/bin/bash
while read -r line; do
num=`echo $line |cut -f 1 -d ","`
prefix=`echo $line |cut -f 2 -d ","`
if [ -n "$num" -a -n "$prefix" ]; then
full_file=$(basename `eval find . -name '$prefix\*' -print -quit` 2>/dev/null )
mv $full_file ${num}_$full_file 2>/dev/null
fi
done < accounts.csv
Related
I have a small bug and don't know how to solve it. I want to copy files from a big folder with many files, where the files contain a specific string. For this I use grep, ack or (in this example) ag. When I'm inside the folder it matches without problem, but when I want to do it with a loop over the files in the following script it doesn't loop over the matches. Here my script:
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" | while read -d $'\0' file; do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done
SEARCH_QUERY holds the String I want to find inside the files, INPUT_DIR is the folder where the files are located, OUTPUT_DIR is the folder where the found files should be copied to. Is there something wrong with the while do?
EDIT:
Thanks for the suggestions! I took this one now, because it also looks for files in subfolders and saves a list with all the files.
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" > "output_list.txt"
while read file
do
echo "${file##*/}"
cp "${file}" "${OUTPUT_DIR}/${file##*/}"
done < "output_list.txt"
Better implement it like below with a find command:
find "${INPUT_DIR}" -name "*.*" | xargs grep -l "${SEARCH_QUERY}" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
or another option:
grep -l "${SEARCH_QUERY}" "${INPUT_DIR}/*.*" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
if you do not mind doing it in just one line, then
grep -lr 'ONE\|TWO\|THREE' | xargs -I xxx -P 0 cp xxx dist/
guide:
-l just print file name and nothing else
-r search recursively the CWD and all sub-directories
match these works alternatively: 'ONE' or 'TWO' or 'THREE'
| pipe the output of grep to xargs
-I xxx name of the files is saved in xxx it is just an alias
-P 0 run all the command (= cp) in parallel (= as fast as possible)
cp each file xxx to the dist directory
If i understand the behavior of ag correctly, then you have to
adjust the read delimiter to '\n' or
use ag -0 -l to force delimiting by '\0'
to solve the problem in your loop.
Alternatively, you can use the following script, that is based on find instead of ag.
while read file; do
echo "$file"
cp "$file" "$OUTPUT_DIR/$file"
done < <(find "$INPUT_DIR" -name "*$SEARCH_QUERY*" -print)
I have the following command that does not work
while read line; do ls $line | head -10 | cp xargs dest_folder; done < my_file.txt
The objective is that I have a file named my_file.txt that contains the name of a folder on every line. I want to copy 10 files from each of the directories mentioned on every line of this file into a dest_folder.
How can I correct the above command?
You cand achive this with the following bash script:
#!/bin/bash
# first argument is the filename which contains ton each line a name of a folder
# teh second argument is the destination folder
while IFS='' read -r line || [[ -n "$line" ]]; do
filesToCopy=($(ls $line |head -n 10))
for i in "${filesToCopy[#]}"
do
:
cp $line/$i $2
done
done < "$1"
Usage:
chmod +x your_bash_script
./your_bash_script my_file.txt dest_folder
Note:
The folder names inside the file should be absolute paths.
You can use xargs with the -I option:
while read line; do ls $line | head -10 | xargs -I % cp % dest_folder; done
In general, you should not use the output of ls programmatically. Use a second loop to iterate over the files in each directory.
while IFS= read -r directory; do
count=0
for f in "$directory"/*; do
(( count++ < 10 )) || break
cp -- "$f" "$dest_folder"
done
done < my_file.txt
This will work for any legal file name found in the directory.
I am trying to move files in folders according to a number in their names.
Files are names like fooNNN_bar.txt I would like to organise them like /NNN/fooNNN_bar.txt
Here is what I have for now. It prints me the folder each file would have to move to. I'm not sure how to collect the number to add it into a mv command. Is this even the correct way to do it?
#!/bin/bash
for filename in foo*.txt;
do
echo "${filename}" | grep -Eo '[0-9]{1,4}';
done
Assuming your grep works as you want:
#!/bin/bash
for filename in foo*.txt; do
num=$(echo "${filename}" | grep -Eo '[0-9]{1,4}')
mkdir -p "$num"
mv "$filename" "$num"
done
I am trying to loop through all the files in a directory.
I want to do some stuff on each file (convert it to xml, not included in example), then write the file to a new directory structure.
for file in `find /home/devel/stuff/static/ -iname "*.pdf"`;
do
echo $file;
sed -e 's/static/changethis/' $file > newfile +".xml";
echo $newfile;
done
I want the results to be:
$file => /home/devel/stuff/static/2002/hello.txt
$newfile => /home/devel/stuff/changethis/2002/hello.txt.xml
How do I have to change my sed line?
If you need to rename multiple files, I would suggest to use rename command:
# remove "-n" after you verify it is what you need
rename -n 's/hello/hi/g' $(find /home/devel/stuff/static/ -type f)
or, if you don't have rename try this:
find /home/devel/stuff/static/ -type f | while read FILE
do
# modify line below to do what you need, then remove leading "echo"
echo mv $FILE $(echo $FILE | sed 's/hello/hi/g')
done
Are you trying to change the filename? Then
for file in /home/devel/stuff/static/*/*.txt
do
echo "Moving $file"
mv "$file" "${file/static/changethis}.xml"
done
Please make sure /home/devel/stuff/static/*/*.txt is what you want before using the script.
First, you have to create the name of the new file based on the name of the initial file. The obvious solution is:
newfile=${file/static/changethis}.xml
Second you have to make sure that the new directory exists or create it if not:
mkdir -p $(dirname $newfile)
Then you can do something with your file:
doSomething < $file > $newfile
I wouldn't do the for loop because of the possibility of overloading your command line. Command lines have a limited length, and if you overload it, it'll simply drop off the excess without giving you any warning. It might work if your find returns 100 file. It might work if it returns 1000 files, but it might fail if your find returns 1000 files and you'll never know.
The best way to handle this is to pipe the find into a while read statement as glenn jackman.
The sed command only works on STDIN and on files, but not on file names, so if you want to munge your file name, you'll have to do something like this:
$newname="$(echo $oldname | sed 's/old/new/')"
to get the new name of the file. The $() construct executes the command and puts the results of the command on STDOUT.
So, your script will look something like this:
find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
echo $file;
newfile="$(echo $file | sed -e 's/static/changethis/')"
newfile="$newfile.xml"
echo $newfile;
done
Now, since you're renaming the file directory, you'll have to make sure the directory exists before you do your move or copy:
find /home/devel/stuff/static/ -name "*.pdf" | while read $file
do
echo $file;
newfile="$(echo $file | sed -e 's/static/changethis/')"
newfile="$newfile.xml"
echo $newfile;
#Check for directory and create it if it doesn't exist
$dirname=$(dirname "$newfile")
if [ ! -d "$dirname" ]
then
mkdir -p "$dirname"
fi
#Directory now exists, so you can do the move
mv "$file" "$newfile"
done
Note the quotation marks to handle the case there's a space in the file name.
By the way, instead of doing this:
if [ ! -d "$dirname" ]
then
mkdir -p "$dirname"
fi
You can do this:
[ -d "$dirname"] || mkdir -p "$dirname"
The || means to execute the following command only if the test isn't true. Thus, if [ -d "$dirname" ] is a false statement (the directory doesn't exist), you run mkdir.
It's a fairly common shortcut when you see shell scripts.
find ... | while read file; do
newfile=$(basename "$file").xml;
do something to "$file" > "$somedir/$newfile"
done
OUTPUT="$(pwd)";
for file in `find . -iname "*.pdf"`;
do
echo $file;
cp $file $file.xml
echo "file created in directory = {$OUTPUT}"
done
This will create a new file with name whatyourfilename.xml, for hello.pdf the new file created would be hello.pdf.xml, basically it creates a new file with .xml appended at the end.
Remember the above script finds files in the directory /home/devel/stuff/static/ whose file names match the matcher string of the find command (in this case *.pdf), and copies it to your present working directory.
The find command in this particular script only finds files with filenames ending with .pdf If you wanted to run this script for files with file names ending with .txt, then you need to change the find command to this find /home/devel/stuff/static/ -iname "*.txt",
Once I wanted to remove trailing -min from my files. i.e. wanted alg-min.jpg to turn into alg.jpg. so after some struggle, managed to figure something like this:
for f in *; do echo $f; mv $f $(echo $f | sed 's/-min//g');done;
Hope this helps someone willing to REMOVE or SUBTITUDE some part of their file names.
I am trying to create a file that contains all of the code of an app. I have created a file called catlist.txt so that the files are added in the order I need them.
A snippet of my catlist.txt:
app/controllers/application_controller.rb
app/views/layouts/*
app/models/account.rb
app/controllers/accounts_controller.rb
app/views/accounts/*
When I run the command the files that are explicitly listed get added but the wildcard files do not.
cat catlist.txt|xargs cat > fullcode
I get
cat: app/views/layouts/*: No such file or directory
cat: app/views/accounts/*: No such file or directory
Can someone help me with this. If there is an easier method I am open to all suggestions.
Barb
Your problem is that xargs is not the shell, so the wildcard is being interpreted literally as an star. You'll need to have a shell to do the expansion for you like this:
cat catlist.txt | xargs -I % sh -c "cat %" > fullcode
Note that the * is not recursive in your data file. I assume that was what you meant. If you want the entries to be recursive, that's a little trickier and would need something more like DevNull's script, but that will require that you change your data file a bit to not include the stars.
Are you positive those directories exist?
The problem with doing a cat on a list like that (where you're using wildcards) is that the cat isn't recursive. It will only list the contents of that directory; not any subdirectories.
Here's what I would do:
#!/bin/bash.exe
output="out.txt"
if [ -f "$output" ]
then
rm $output
fi
for file in $(cat catlist.txt)
do
if [ -f "$file" ]
then
echo "$file is a file."
cat $file >> $output
elif [ -d "$file" ]
then
echo "$file is a directory."
find $file -type f -exec cat {} >> $output \;
else
echo "huh?"
fi
done
If the entry listed is a directory, it finds all files from that point on and cats them.
use a while read loop to read your file
while read -r file
do
if [ -f "$file" ]; then
yourcode "$file"
fi
# expand asterix
case "$file" in
*"*" )
for f in $file
do
yourcode "$f"
done
esac
done <"catlist.txt"