I would appreciate any help, relatively new here
I have the following directory structure
Main_dir
|-Barcode_subdirname_01\(many further subfolders)\filename.pdf
|-Barcode_subdirname_02\(many further subfolders)\filename.csv
There are 1000s of files within many subfolders
The first level sub directories have the barcode associated to all files within. eg 123456_dirname
I want to copy all files within all subfoders to the main_dir and
rename the files subdirname_barcode_filename.extension (based only on the first subdirectory name and barcode)
I've been attempting to write a bash script to do this from the main_dir but have hit the limit of my coding ability (i'm open to any other way that'll work).
firstly identifying the first level sub folders
find -maxdepth 1 -type d |
then cut out the first 2 parts deliminated by the underscores
cut -d\_ -f1 > barcode
then find the files within the subfolders, rename and move
find -type f -print0 |
while IFS= read -r filenames; do
newname="${barcode/sudirname/filename\/}"
mv "filename" "main_dir"/"newname"
done
I can't get it to work and may be headed in the wrong direction.
You can use rename with sed like substitute conventions, for example
$ rename 's~([^_]+)_([^_]+)_.*/([^/.]+\..*)~$1_$2_$3~' barcode_subdir_01/a/b/c/file2.csv
will rename file to
barcode_subdir_file2.csv
I used ~ instead of the more common / separator to make it more clear.
You can test the script with -n option to show the renamed files without actually doing the action.
Related
I have a set of files that are organised across a large number of sub-directories, for example:
/Image/Square001/Data
/Image/Square002/Data
/Image/Square003/Data
and so on.
All the files within these directories have a similar type of name but have different numbers within the name.
Triangle_284852_Data_222844_222846_20220302_034926.tiff
Triangle_284908_Data_222841_222843_20220302_035350.tiff
Triangle_284908_Data_222845_222843_20220302_035350.tiff
I had like to rename all of these files to include another number. The correct number needed has already been appended to a given filename within a text file (temp_names.txt)
Triangle_284852_Data_222844_222846_20220302_034926_1.tiff
Triangle_284908_Data_222841_222843_20220302_035350_63.tiff
Triangle_284908_Data_222845_222843_20220302_035350_12.tiff
I am trying to write a bash script to rename the files in the directory to match those in temp_names.txt
I think I need to use the find command, set up a while loop for each file to be matched based on their similarity and be renamed, but I am having trouble getting this right.
In the past, I have used something like:
for file in ./Image/Square*/Data/*.tiff
do
read line
mv "${file}" "${line}"
done < temp_names.txt
When the files are all in the same sub-directory and are read in the same order as they appear in my text file. However, this is not the case for these files and I need a new approach. I have been trying some variants of starting like this:
find /Images/Square*/Data/ -type f -name .tiff |
while IFS= read -d '' file_name; do
But I am unsure whether I should be using rename or mv in the next line, how to match the common parts between the filename in the directory and in the text file, and then how to add a changeable string (the number at the end) to the file name.
This is a script to do your requirement:
#!/bin/bash
find . -type f -name "Triangle_*.tiff" -print0 | while IFS= read -r -d '' file
do
# extract the filename, without extension
filename=$(basename "${file%.*}")
# Find that in the temp_names.txt file
newfilename=$(grep "$filename" temp_names.txt)
# extract the path of the file
filepath=$(dirname "$file")
# rename the file
mv -v "$file" "$filepath/$newfilename"
done
The find is recursive, so it will handle all files.
From each file, remove its extension, then grep for the name in the list of new filenames.
Since the file could be in any sub-directory, the path is extracted from the file found by find.
The mv changes the name of the file to the new name, under the same directory as the original file.
I put option -v to the mv to get some log of what is done. But you can remove it if not required.
Try to copy files recursively from directory (d1) to directory (d2) exclude some files and folders using lists:
filepaths to exclude stored as files.txt
and directories list to exlude stored as dirs.txt.
Already read some SOF articles like
Copy folder recursively, excluding some folders
BASH copy all files except one
tried rsync
rsync -avr --exclude=*/exclude_filename_* d1/ d2/
but wildcards don't suit either due to a large number of different file names.
Maybe I have to use WHILE loop with rsync..? Still looking for solution.
Does anybody know how to do this?
This is not the best way to do it, but if rsync is not working, you can use find to composite a list and then remove files from that list.
find . -name "*" | grep -v "<filename you dont want>" | xargs -i cp --parents {} /output/dir/name
The only thing to keep in mind here is that your current directory must be the base directory of the files you want to copy, in order to preserve the parent structure.
note:
Add another | grep -v "<filename you dont want>" to remove a second file. Alternatively, you can use wildcard matching in one grep grep -v "<file1>\|<file2>"
I have a list of files ~100,000 txt files with the following pattern name: file_[combination of letters and numbers]_[numbers from 1 to 400].txt, three examples would be:
file_ab34_1.txt, file_ab35_1.txt, file_bg12_2.txt, file_bg12_2.txt. What I want to do automatically is move all the files with _1 to a subfolder named /1 and all the files with _2 to a subfolder /2 and so on.
I would need a bash script to do it automatically and not one by one
Please back up your files before trying this answer.
I would try rename, like this:
rename --dry-run 's|(.*)_(\d+).txt|$2/$1_$2.txt|' *.txt
'file_ab34_1.txt' would be renamed to '1/file_ab34_1.txt'
'file_ab35_1.txt' would be renamed to '1/file_ab35_1.txt'
'file_bg12_2.txt' would be renamed to '2/file_bg12_2.txt'
The --dry-run just shows you what it would do without actually doing anything - great for testing before using.
It is basically Perl, and it is doing a substitution on the filename. The bones of it is to substitute like this:
s|something|something else|
Every time there are parentheses on the left side (called capture groups), they capture some aspect of the left side and it is then available as a numbered item to put in the replacement, right hand side, where $1 represents whatever was captured in the first set of parentheses and $2 represents whatever was in the second set of parentheses and so on.
You will likely need -p option to create the output directories, so:
rename -p ....
If you get errors about the argument list being too long, you will probably need to use find and xargs, along these lines (untested):
find . -name \*.txt -maxdepth 1 -print0 | xargs -0 -n 1000 rename ....
You can do this with just mv with shell globbing to get the files:
mv -t /1 file_*_1.txt
mv -t /2 file_*_2.txt
I would like to paste a number of files in a unique file named, for example, "output.txt".
Nevertheless I would like to assign the name of the folder to the name of the output file so that it will be: "output_<name of the Folder>.txt".
I have thousands of folders so that the "paste" command will run in a for loop I'm able to write. Can anyone help me please?
The question is incredibly unclear. I'm going to interpret it to mean that you have a large number of directories that all contain a file named 'output.txt', and you want to move those files to a single directory with the original path embedded in the name. Assuming that the root of the directory tree containing all the files is /path/to/source and you want to move them to /path/to/destination:
find /path/to/source -name output.txt -exec sh -c 'd=$(dirname "$1" |
tr / _); cp "$1" "/path/to/destination/output_$d.txt"' sh {} \;
Relative paths will work fine as well as absolute paths.
I too am unclear about what you want, but mktemp(1) has TEMPLATES which might help.
I'm trying to get an example working:
Here is what I'm trying to do:
a) There are 7 files in a folder with the name and timestamp appended.
Examples : Windows_<timestamp>.csv and Linux_<timestamp>.csv so on and so forth.
I want to first a) Move the files that I'm about to re-name to a new folder as is and then rename the current file.
I've tried looking at Rename multiple files by replacing a particular pattern in the filenames using a shell script but that script isn't working for me. I believe I have to modify something in there, but I cant seem to get it work.
Can anyone please help me? I'm really stuck here.
Thanks!
#!/bin/bash
# find all the files that are created today and end with an extension.
# a) First find all the files created today, and filter only the files we care about.
# b) Move all these files into a new folder.
# c) Iterate all the files in this new folder.
# d) Re-name all the files in the destination folder by replacing the _
sourceFolderName="/home/abhididdigi/Desktop/TADDM"
targetFolderName="/home/abhididdigi/Desktop/TADDM_ServiceNow/"
#find all the files created today and only the CSV ones.
find $sourceFolderName -type f -mtime 0 -name '*.csv'|
while read filename
do
# TODO: Find only those files that we care about
cp $filename $targetFolderName
done
#targetFileName=${filename%_*}|sed 's#.*/##';
#Rename the files now, removing the timestamp from underscore, so that it is ready to consume.
for filename in $targetFolderName*; do
mv -v "${filename}" ${filename%_*}.`echo "${filename}" | awk -F. '{print $2}'`
done