I am trying to move files in folders according to a number in their names.
Files are names like fooNNN_bar.txt I would like to organise them like /NNN/fooNNN_bar.txt
Here is what I have for now. It prints me the folder each file would have to move to. I'm not sure how to collect the number to add it into a mv command. Is this even the correct way to do it?
#!/bin/bash
for filename in foo*.txt;
do
echo "${filename}" | grep -Eo '[0-9]{1,4}';
done
Assuming your grep works as you want:
#!/bin/bash
for filename in foo*.txt; do
num=$(echo "${filename}" | grep -Eo '[0-9]{1,4}')
mkdir -p "$num"
mv "$filename" "$num"
done
Related
I have a small bug and don't know how to solve it. I want to copy files from a big folder with many files, where the files contain a specific string. For this I use grep, ack or (in this example) ag. When I'm inside the folder it matches without problem, but when I want to do it with a loop over the files in the following script it doesn't loop over the matches. Here my script:
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" | while read -d $'\0' file; do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done
SEARCH_QUERY holds the String I want to find inside the files, INPUT_DIR is the folder where the files are located, OUTPUT_DIR is the folder where the found files should be copied to. Is there something wrong with the while do?
EDIT:
Thanks for the suggestions! I took this one now, because it also looks for files in subfolders and saves a list with all the files.
ag -l "${SEARCH_QUERY}" "${INPUT_DIR}" > "output_list.txt"
while read file
do
echo "${file##*/}"
cp "${file}" "${OUTPUT_DIR}/${file##*/}"
done < "output_list.txt"
Better implement it like below with a find command:
find "${INPUT_DIR}" -name "*.*" | xargs grep -l "${SEARCH_QUERY}" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
or another option:
grep -l "${SEARCH_QUERY}" "${INPUT_DIR}/*.*" > /tmp/file_list.txt
while read file
do
echo "$file"
cp "${file}" "${OUTPUT_DIR}/${file}"
done < /tmp/file_list.txt
rm /tmp/file_list.txt
if you do not mind doing it in just one line, then
grep -lr 'ONE\|TWO\|THREE' | xargs -I xxx -P 0 cp xxx dist/
guide:
-l just print file name and nothing else
-r search recursively the CWD and all sub-directories
match these works alternatively: 'ONE' or 'TWO' or 'THREE'
| pipe the output of grep to xargs
-I xxx name of the files is saved in xxx it is just an alias
-P 0 run all the command (= cp) in parallel (= as fast as possible)
cp each file xxx to the dist directory
If i understand the behavior of ag correctly, then you have to
adjust the read delimiter to '\n' or
use ag -0 -l to force delimiting by '\0'
to solve the problem in your loop.
Alternatively, you can use the following script, that is based on find instead of ag.
while read file; do
echo "$file"
cp "$file" "$OUTPUT_DIR/$file"
done < <(find "$INPUT_DIR" -name "*$SEARCH_QUERY*" -print)
I would like to know how to display the filename along with the lines matching a specfic word of a tar file.
Command wise :
zcat file | grep "stuff" -r # shows what I want
zcat *.gz | grep "stuff" -ar # this fails
You can use zgrep:
For single file, you can use the following command to display filename:
zgrep "stuff" file.gz /dev/null
For multiple files:
zgrep "stuff" *.gz
Maybe this related answer can help. It uses tar to untar (you would need to add -z) and pipes each file of the archive to awk for "grepping" inside it.
I'm not quite sure what the question is but if you are looking for tar files on your system then just do something like this. This will recursively search your current directory and any child directories for .tar files. Hope this helps.
find -name "*.tar"
If zcat file | grep "stuff" -r shows what you want, you can do this for multiple files:
for name in *.gz ; do zcat "$name" | grep -a "stuff" | sed -e "s/^/${name}: /" ; done
This command uses globbing (*) to expand to a list of .gz files in your working directory, then calls zcat for extraction, grep for the search and sed for prefixing with the filename on each of the files.
Note that if you are working with gzipped tarballs, most people give them a .tgz or .tar.gz instead of just .gz extension.
This will output nameOfFileInTar:LineNumber:Match. Invoke with greptar.sh tarfile.tar pattern
If you don't want the line number, remove the -n option. If you only want the line number, add |cut -f1 -d: after the grep
#!/bin/bash
TARFILE=$1
PATTERN=$2
tar ztf $TARFILE | while read -r FILE
do
res=$(tar zxf $TARFILE $FILE -O | grep -n $2 )
if [[ $? == 0 ]]; then
echo "$res" | while read -r line; do
echo $FILE:$line;
done
fi
done
I want to create directories which names should correspond to a list of zipped files in the parent directory. Additionally I want to get rid of the file extension in the resulting name of the directory.
e.g. archive01.gz should result in a directory with name archive01
My script so far:
#!/bin/bash
for file in *.gz; do
echo $file | sed 's/.gz//' | mkdir
done
The error message is:
mkdir: missing operand
However,
echo $file | sed 's/.gz//'
results in the correct name for the directory. How do I pipe it to mkdir?
A better way to do this would be to use parameter substitution instead of a pipe/subshell:
#!/bin/bash
for file in *.gz; do
mkdir ${file%.gz}
done
Try this,
#!/bin/bash
for file in *.gz; do
echo "$file" | sed 's/\.gz//' | xargs mkdir
done
mkdir doesn't work with pipes. Try the following:
mkdir `echo $file | sed 's/.gz//'`
The ` evaluates and then replaces it with the answer. It's called command substitution and alternatively could also be written as:
mkdir $(echo $file | sed 's/.gz//')
You can use command substitution as
#!/bin/bash
for file in *.gz; do
mkdir $(echo $file | sed 's/.gz//')
done
OR
#!/bin/bash
for file in *.gz; do
mkdir `echo $file | sed 's/.gz//'`
done
I have 224 pdf files and I'd like to prefix the files with a number and _
Example:
stackoverflow_nov_2014.pdf
File accounts.csv contains:
2567,stackoverflow
So the goal is to take 2567 and prefix it to the pdf file with an underscore:
2567_stackoverflow_nov_2014.pdf
I think I would want to use read -r in a while loop as explained here:
https://unix.stackexchange.com/questions/147569/rename-a-batch-of-files-after-reading-from-a-source-file
But when attempting this as it's written, the shell gives usage of mv command and nothing changes with the files.
Edit: Adding sample data from sources file (accounts.csv)
11,My_Golf_Shop
2567,stackoverflow
11122,Test_Store
By the way, the sources file (accounts.csv) isn't in the same order as the files in the directory as accounts.csv so somehow there would need to be matching with file name and the accounts.csv that occurs.
Below is the script that should work under the assumption:
1. All the files are under the same folder
2. For a particular prefix, this script will only rename the first found file.
#!/bin/bash
while read -r line; do
num=`echo $line |cut -f 1 -d ","`
prefix=`echo $line |cut -f 2 -d ","`
if [ -n "$num" -a -n "$prefix" ]; then
full_file=$(basename `eval find . -name '$prefix\*' -print -quit` 2>/dev/null )
mv $full_file ${num}_$full_file 2>/dev/null
fi
done < accounts.csv
Could someone please help with this script. I need to use grep to loop to through the filenames that need to be changed.
#!/bin/bash
file=
for file in $(ls $1)
do
grep "^.old" | mv "$1/$file" "$1/$file.old"
done
bash can handle regular expressions without using grep.
for f in "$1"/*; do
[[ $f =~ \.old ]] && continue
# Or a pattern instead
# [[ $f == *.old* ]] && continue
mv "$f" "$f.old"
done
You can also move the name checking into the pattern itself:
shopt -s extglob
for f in "$1/"!(*.old*); do
mv "$f" "$f.old"
done
If I understand your question correctly, you want to make rename a file (i.e. dir/file.txt ==> dir/file.old) only if the file has not been renamed before. The solution is as follow.
#!/bin/bash
for file in "$1/"*
do
backup_file="${file%.*}.old"
if [ ! -e "$backup_file" ]
then
echo mv "$file" "$backup_file"
fi
done
Discussion
The script currently does not actual make back up, it only displays the action. Run the script once and examine the output. If this is what you want, then remove the echo from the script and run it again.
Update
Here is the no if solution:
ls "$1/"* | grep -v ".old" | while read file
do
echo mv "$file" "${file}.old"
done
Discussion
The ls command displays all files.
The grep command filter out those files that has the .old extension so they won't be displayed.
The while loop reads the file names that do not have the .old extension, one by one and rename them.