I'm trying to create a folder using txt file and copy the file. I have two file types:
try.txt
Changes/EMAIL/header-20-percent-off.gif
Changes/EMAIL/header-50-percent-off.gif
demo of folder named zip2
zip2/EMAIL/header-20-percent-off.gif
zip2/EMAIL/header-50-percent-off.gif
Code:
mkdir -p dirname `xargs -a try.txt`
cp -R {Dont know how this will work :( }
Actual output:
Changes/EMAIL/header-20-percent-off.gif/
/header-50-percent-off.gif/
Expected output:
Changes/EMAIL/header-20-percent-off.gif
/header-50-percent-off.gif
As you can see for some reason it thinks header-20-percent-off.gif and header-50-percent-off.gif are directories.
Once Changes/Email/ is created I would like to copy the two gif files header-20-percent-off.gif and header-50-percent-off.gif there.
First create folders:
<try.txt xargs -d$'\n' dirname | xargs -d$'\n' mkdir -p
Then copy files. First prepare the stream with proper source and destination directories with sed and then pass to xargs:
sed 's#^Changes/\(.*\)#zip2/\1\n&#' try.txt |
xargs -d$'\n' -n2 cp
But if you are not proficient in bash, just read the stream line by line:
while IFS= read -r dest; do
dir=$(dirname "$dest")
mkdir -p "$dir"
src=$(sed 's#^Changes#zip2#' <<<"$dest")
cp "$src" "$dest"
done < try.txt
Don't use backticks `, they are highly discouraged. Use $(...) for command substitution instead.
Just doing xargs -a try.txt without a command makes little sense, just $(cat try.txt) or better $(<try.txt).
Use -t option with xargs to see what is it doing.
Explicitly specify the delimeter with xargs -d$'\n' - otherwise xargs will parse " ' and \ specially.
I believe with some luck and work you could just use rsync with something along rsync --include-from=try.txt changes/ zip2/.
Related
touch source
$ echo dest.{000000..999999} | tr ' ' '\n' | while read dest ; do echo cp -v source $dest ; done
cp -v source dest.000000
cp -v source dest.000001
cp -v source dest.000002
cp -v source dest.000003
cp -v source dest.000004
cp -v source dest.000005
cp -v source dest.000006
cp -v source dest.000007
cp -v source dest.000008
cp -v source dest.000009
...
Well, this is gonna take forever, mainly because each copy invokes a new cp process.
Let's try with xargs:
$ echo dest.{000000..999999} | xargs -n 1000 cp source
cp: target 'dest.000999' is not a directory
Yeah, right, when giving multiple arguments, cp assumes that n-1 arguments are source files, and the nth argument is a destination directory.
I need a command that works differently:
mycp source dest1 dest2 dest3 ...
How could I achieve this, without invoking a new process for each copy?
(based on the suggestion by Cyrus)
This works:
function multi-cp () {
local source="$1"
shift
tee "${#}" < "$source" > /dev/null
}
echo dest.{000000..999999} | xargs -n 1000 | while read -r destinations ; do
multi-cp source $destinations
done
We use while because xargs can not call functions (there are ways around this, but they have other problems). We still use xargs to split the arguments in manageable chunks.
This assumes that the arguments have no spaces (which is the case, since we are in control).
I want to look through 100K+ text files from a directory and copy to another directory only the ones which contain at least one word from a list.
I tried doing an if statement with grep and cp but I have no idea how to make it to work this way.
for filename in *.txt
do
grep -o -i "cultiv" "protec" "agricult" $filename|wc -w
if [ wc -gt 0 ]
then cp $filename ~/desktop/filepath
fi
done
Obviously this does not work but I have no idea how to store the wc result and then compare it to 0 and only act on those files.
Use the -l option to have grep print all the filenames that match the pattern. Then use xargs to pass these as arguments to cp.
grep -l -E -i 'cultiv|protec|agricult' *.txt | xargs cp -t ~/desktop/filepath --
The -t option is a GNU cp extension, it allows you to put the destination directory first so that it will work with xargs.
If you're using a version without that option, you need to use the -J option to xargs to substitute in the middle of the command.
grep -l -E -i 'cultiv|protec|agricult' *.txt | xargs -J {} cp -- {} ~/desktop/filepath
I have a small bug and don't know how to solve it. I want to copy files from a big folder with many files, where the files contain a specific string. For this I use grep, ack or (in this example) ag. When I'm inside the folder it matches without problem, but when I want to do it with a loop over the files in the following script it doesn't loop over the matches. Here my script:
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" | while read -d $'\0' file; do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done
SEARCH_QUERY holds the String I want to find inside the files, INPUT_DIR is the folder where the files are located, OUTPUT_DIR is the folder where the found files should be copied to. Is there something wrong with the while do?
EDIT:
Thanks for the suggestions! I took this one now, because it also looks for files in subfolders and saves a list with all the files.
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" > "output_list.txt"
while read file
do
echo "${file##*/}"
cp "${file}" "${OUTPUT_DIR}/${file##*/}"
done < "output_list.txt"
Better implement it like below with a find command:
find "${INPUT_DIR}" -name "*.*" | xargs grep -l "${SEARCH_QUERY}" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
or another option:
grep -l "${SEARCH_QUERY}" "${INPUT_DIR}/*.*" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
if you do not mind doing it in just one line, then
grep -lr 'ONE\|TWO\|THREE' | xargs -I xxx -P 0 cp xxx dist/
guide:
-l just print file name and nothing else
-r search recursively the CWD and all sub-directories
match these works alternatively: 'ONE' or 'TWO' or 'THREE'
| pipe the output of grep to xargs
-I xxx name of the files is saved in xxx it is just an alias
-P 0 run all the command (= cp) in parallel (= as fast as possible)
cp each file xxx to the dist directory
If i understand the behavior of ag correctly, then you have to
adjust the read delimiter to '\n' or
use ag -0 -l to force delimiting by '\0'
to solve the problem in your loop.
Alternatively, you can use the following script, that is based on find instead of ag.
while read file; do
echo "$file"
cp "$file" "$OUTPUT_DIR/$file"
done < <(find "$INPUT_DIR" -name "*$SEARCH_QUERY*" -print)
I am trying to move all my video files that are in my pictures directory to my movies Directory. This is on a Mac by the way.
I thought I could simple Recurse through all my picture directories with an "ls -R"
Then I pipe that to grep -i ".avi" This give me all the movie files.
Now I pipe these values to "mv -n $1 ~/Movies" this I am hoping would move the files to the Movies folder.
I have a few Problems.
1. The "ls -R" does not list the path when listing the files. So I think I may fail to move the file.
2. I can not seem to get the file name to assign to the $1 in the mv command.
All together my command looks like this: Note I am running this from ~/Pictures
ls -R | grep -i ".avi" | mv -n $1 ~/Movies
So right now I am not sure which part is failing but I do get this error:
usage: mv [-f | -i | -n] [-v] source target
mv [-f | -i | -n] [-v] source ... directory
If I remove the 'mv' command I get a listing of avi files with out the path. Example Below:
4883.AVI
4884.AVI
4885.AVI
4886.AVI
4887.AVI
...
Any one have any ideas on how I can get the path in the 'ls' or how to pass a value in between the '|' commands.
Thanks.
It's better if you use the find command:
$ find -name "*.avi" -exec mv {} ~/Movies \;
you should create simple copy.sh like this
#!/bin/bash
cp $1 ~/Movies/
An run command ./copy.sh "$(ls | grep avi)"
The bash for loop can help you find all the avi files easily
shopt -s nullglob
for file in *.avi
do
mv "$file" "$file" ~/Movies/"$file"
done
you can achieve this in many ways, one of it in my openion:
ls -R | grep -i ".avi" | while read movie
do
echo " moving $movie"
mv $movie ~/Movies/
done
Use backticks
mv `ls *.avi` ~/Movies
I have a folder with files named as
input (1).txt
input (2).txt
input (3).txt
...
input (207).txt
How do I rename them to
input_1.in
input_2.in
input_3.in
...
input_207.in
I am trying this
for f in *.txt ; do mv $f `echo $f | sed -e 's/input\ (\(\d*\))\.txt/input_\1.in/'` ; done
But it gives me
mv: target `(100).txt' is not a directory
mv: target `(101).txt' is not a directory
mv: target `(102).txt' is not a directory
...
Where did I go wrong?
I have put in the quotes now, but I get this now
mv: `input (90).txt' and `input (90).txt' are the same file
It is somehow trying to rename the file to the same name. How is that happening?
That is because bash for split the element with space ' ' so you are commanding it to move 'input' to '(1)'.
The way to solve this is to tell bash to split by new line using IFS variable.
Like this:
IFS=$'\n'
Then do your command.
However, I suggest you to use find to do this instead using -exec command.
For example:
find *.txt -exec mv "{}" `echo "{}" | sed -e 's/input\ (\([0-9]*\))\.txt/input_\1.in/'` \;
NOTE: I write this from memory and I did test this so let try and adjust it.
Hope this helps.
You're forgetting to quote your arguments.
... mv "$f" "$(echo "$f" | ... )" ; done
no need to call external commands
#!/bin/bash
shopt -s nullglob
shopt -s extglob
for file in *.txt
do
newfile="${file//[)]/}"
newfile="${file// [(]/_}"
mv "$file" "${newfile%.txt}.in"
done
As you've already fixed, you need to quote the $f argument to mv.
As to your second problem, sed doesn't support \d. You could use [0-9] instead.
for f in *.txt ; do mv "$f" `echo $f | sed -e 's/input\ (\(\d*\))\.txt/input_\1.in/'` ; done
If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:
seq 1 207 | parallel -q mv 'input ({}).txt' input_{}.in
Watch the intro video for GNU Parallel to learn more:
http://www.youtube.com/watch?v=OpaiGYxkSuQ