I have the following code below, which is an attempt to create a symbolic link for each file matching the pattern *let.txt in the working folder, which has the same name as the original file but with underscores instead of spaces. I need to keep the original files untouched hence the use of symlinks.
The error I get is
ln: failed to access ‘*let.txt’: Too many levels of symbolic links
So I see the search string is getting passed very literally into tempstring, I don't know why. How do I correct my code?
for file in *let.txt; do
tempstring="${file// /_}"
ln -s "$file" $tempstring
done
Try putting quotes around all your variables when 1 variable is 1 parameter:
for file in *let.txt; do
tempstring="${file// /_}"
ln -s "$file" "$tempstring"
done
Tested and works on GNU bash, version 5.0.11(1)-release (x86_64-apple-darwin18.6.0)
Related
Trying to write a bash script to copy a large number of files from an external drive into separate directories based on a subject id.
I've included the script I've written below.
I get the following error:
cat: /Volumes/Seagate: No such file or directory
cat: Backup: No such file or directory
cat: Plus: No such file or directory
cat: Drive/Subject_List.txt: No such file or directory
When I try to copy a single file at a time using the terminal, it copies using the exact command I've put in this script. I'm not sure why it's not recognizing the directory when I try and use it in the script below. Any help is greatly appreciated!
#!/bin/bash
#A bash script to copy the structural and functional files into the HCP_Entropy folder
#subject list
SUBJECT_LIST="/Volumes/Seagate/Backup/Plus/Drive/Subject_List.txt
for j in $(cat ${SUBJECT_LIST}); do
echo ${j}
cp /Volumes/Seagate\ Backup\ Plus\ Drive/HCP_DATA/Structural/{j}/unprocessed/3T/T1w_MPR1/${j}_3T_T1w_MPR1.nii.gz /Users/myname/Box/HCP_Entropy/BEN/${j}/anat
done
the line
$SUBJECT_LIST=/Volumes/Seagate\ Backup\ Plus\ Drive/Subject_List.txt
is bogus.
to assign values to a variable, you must not add the $ specifier.
a token starting with $ will be expanded, so $SUBJECT_LIST=... will first be expanded to =... (since you haven't assigned anything to the SUBJECT_LIST variable yet it is empty).
the proper way would be:
SUBJECT_LIST="/Volumes/Seagate Backup Plus Drive/Subject_List.txt"
(this uses quotes instead of escaping each space, which i find much more readable)
you also need to quote variables in case they contain spaces, else they might be interpreted by the command (cp) as multiple arguments.
for j in $(cat "${SUBJECT_LIST}"); do
# ...
done
and of course, you should check whether the source file actually exists, just like the destination directory.
indir="/Volumes/Seagate Backup Plus Drive"
SUBJECT_LIST="${indir}/Subject_List.txt"
cat "${SUBJECT_LIST}" | while read j; do
infile="${indir}/HCP_DATA/Structural/${j}/unprocessed/3T/T1w_MPR1/${j}_3T_T1w_MPR1.nii.gz"
outdir="/Users/myname/Box/HCP_Entropy/BEN/${j}/anat"
mkdir -p "${outdir}"
if [ -e "${infile}" ]; then
cp -v "${infile}" "${outdir}"
else
echo "missing file ${infile}" 1>&2
fi
done
grep -R is returning a file called ..ThisFile. The double dot is not separated from the filename with a /.
I know all about ./ and ../ to signify "this directory" and "the directory above". What does this mean?
$ grep -R fish > grepresults
$ cat grepresults
..SomeFile
I looked in, above, and below the current directory for SomeFile, and I sure don't see it. Maybe I messed it. I don't know what to expect. I don't know what the .. is telling me.
It is a normal file. File and directory names are allowed to include dots, also at the beginning. Some tools, like ls, do however hide files and directories starting with a dot by default. The -a command line flag to ls disables this behavior.
I'm writing a small piece of code that checks for .mov files in a specific folder over 4gb and writes it to a log.txt file by name (without an extension). I'm then reading the names into a while loop line by line which signals some archiving and copying commands.
Consider a file named abcdefg.mov (new) and a corresponding folder somewhere else named abcdefg_20180525 (<-*underscore timestamp) that also contains a file named abcedfg.mov (old).
When reading in the filename from the log.txt, I strip the extension to store the variable "abcdefg" ($in1) and i'm using that variable to locate a folder elsewhere that contains that matching string at the beginning.
My problem is with how the mv command seems to support a wild card in the "source" string, but not in the "destination" string.
For example i can write;
mv -f /Volumes/Myshare/SourceVideo/$in1*/$in1.mov /Volumes/Myshare/Archive
However a wildcard on the destination doesn't work in the same way. For example;
mv -f /Volumes/Myshare/Processed/$in1.mov Volumes/Myshare/SourceVideo/$in1*/$in1.mov
Is there an easy fix here that doesn't involve using another method?
Cheers for any help.
mv accepts a single destination path. Suppose that $in1 is abcdfg, and that $in1* expands to abcdefg_20180525 and abcdefg_20180526. Then the command
mv -f /dir1/$in1 /dir2/$in1*/$in1.mov
will be equivalent to:
mv -f /dir1/abcdefg.mov /dir2/abcdefg_20180526/abcdefg.mov
mv -f /dir1/abcdefg.mov /dir2/abcdefg_20180526/abcdefg.mov
mv -f /dir2/abcdefg_20180525/abcdefg.mov /dir2/abcdefg_20180526/abcdefg.mov
Moreover, because the destination file is the same in all three cases, the first two files will be overwritten by the third.
You should create a precise list and do a precise copy instead of using wild cards.
This is what I would probably do, generate a list of results in a file with FULL path information, then read those results in another function. I could have used arrays but I wanted to keep it simple. At the bottom of this script is a function call to scan for files of EXT mp4 (case insensitive) then writes the results to a file in tmp. then the script reads the results from that file in another function and performs some operation (mv etc.). Note, if functions are confusing , you can just remove the function name { } and name calls and it becomes a normal script again. functions are really handy, learn to love them!
#!/usr/bin/env bash
readonly SIZE_CHECK_LIMIT_MB="10M"
readonly FOLDER="/tmp"
readonly DESTINATION_FOLDER="/tmp/archive"
readonly SAVE_LIST_FILE="/tmp/$(basename $0)-save-list.txt"
readonly EXT="mp4"
readonly CASE="-iname" #change to -name for exact ext type upper/lower
function find_files_too_large() {
> ${SAVE_LIST_FILE}
find "${FOLDER}" -maxdepth 1 -type f "${CASE}" "*.${EXT}" -size +${SIZE_CHECK_LIMIT_MB} -print0 | while IFS= read -r -d $'\0' line ; do
echo "FOUND => $line"
echo "$line" >> ${SAVE_LIST_FILE}
done
}
function archive_large_files() {
local read_file="${SAVE_LIST_FILE}"
local write_folder="$DESTINATION_FOLDER"
if [ ! -s "${read_file}" ] || [ ! -f "${read_file}" ] ;then
echo "No work to be done ... "
return
fi
while IFS= read -r line ;do
echo "mv $line $write_folder" ;sleep 1
done < "${read_file}"
}
# MAIN (this is where the script starts) We just call two functions.
find_files_too_large
archive_large_files
it might be easier, i think, to change the filenames to the folder name initially. So abcdefg.mov would be abcdefg_timestamp.mov. I can always strip the timestamp from the filename easy enough after its copied to the right location. I was hoping i had a small syntax issue but i think there is no easy way of doing what i thought i could...
I think you have a basic misunderstanding of how wildcards work here. The mv command doesn't support wildcards at all; the shell expands all wildcards into lists of matching files before they get passed to the mv command as wildcards. Furthermore, the mv command doesn't know if the list of arguments it got came from wildcards or not, and the shell doesn't know anything about what the command is going to do with them. For instance, if you run the command grep *, the grep command just gets a list of names of files in the current directory as arguments, and will treat the first of them as a regex pattern ('cause that's what the first argument to grep is) to search the rest of the files for. If you ran mv * (note: don't do this!), it will interpret all but the last filename as sources, and the last one as a destination.
I think there's another source of confusion as well: when the shell expands a string containing a wildcard, it tries to match the entire thing to existing files and/or directories. So when you use Volumes/Myshare/SourceVideo/$in1*/$in1.mov, it looks for an already-existing file in a matching directory; AIUI the file isn't there yet, there's no match. What it does in that case is pass the raw (unexpanded) wildcard-containing string to mv as an argument, which looks for that exact name, doesn't find it, and gives you an error.
(BTW, should there be a "/" at the front of that pattern? I assume so below.)
If I understand the situation correctly, you might be able to use this:
mv -f /Volumes/Myshare/Processed/$in1.mov /Volumes/Myshare/SourceVideo/$in1*/
Since the filename isn't supplied in the second string, it doesn't look for existing files by that name, just directories with the right prefix; mv will automatically retain the filename from the source.
However, I'll echo #Sergio's warning about chaos from multiple matches. In this case, it won't overwrite files (well, it might, but for other reasons), but if it gets multiple matching target directories it'll move all but the last one into the last one (along with the file you meant to move). You say you're 100% certain this won't be a problem, but in my experience that means that there's at least a 50% chance that something you'd never have thought of will go ahead and make it happen anyway. For instance, is it possible that $in1 could wind up empty, or contain a space, or...?
Speaking of spaces, I'd also recommend double-quoting all variable references. You want the variables inside double-quotes, but the wildcards outside them (or they won't be expanded), like this:
mv -f "/Volumes/Myshare/Processed/$in1.mov" "/Volumes/Myshare/SourceVideo/$in1"*/
I'm trying append to word "dicom" to the front of many filenames in a set of folders. The folders all begin with "s" (referred to by "s*" in the script below), and each contain many files (specified by "*" below)--I'd like all of these files to be changed using this bash script. I tried to run this:
for file in /Volumes/USB_AIB/DICOMFunCurrentBatch/MOVr1unzip/s*/*
do
mv $file dicom${file%%}
done
but got thousands of lines of the following error (once for each file within each folder--this is just an example of one of them):
mv: rename /Volumes/USB_AIB/DICOMFunCurrentBatch/MOVr1unzip/s307_1/29217684 to dicom/Volumes/USB_AIB/DICOMFunCurrentBatch/MOVr1unzip/s307_1/29217684: No such file or directory
Any ideas on how to fix it?
I don't you have a valid path as dicom/Volumes/USB_AIB/DICOMFunCurrentBatch/MOVr1unzip/s307_1/, why do you add dicom at the beginning?
maybe you want to append dicom to the end of the $file?
mv "$file" "${file}_dicom"
or something like that.
the following variable expansion ${file%%} is strange because it does nothing:
${parameter%%word} : remove the longest matching suffix pattern.
to move the file into a directory the path should exists, to create the path:
mkdir -p "$(dirname "${newfilename}")"
Maybe what you are trying to do:
for file in /Volumes/USB_AIB/DICOMFunCurrentBatch/MOVr1unzip/s*/*
do
mv "$file" "${file%/*}/dicom${file##*/}"
done
I am very new with linux usage maybe this is my first time so i hope some detailed help please.
I have more than 500 files in multiple directories on my server (Linux) I want to change their extensions to .xml using bash script
I used a lot of codes but none of them work some codes i used :
for file in *.txt
do
mv ${file} ${file/.txt}/.xml
done
or
for file in *.*
do
mv ${file} ${file/.*}/.xml
done
i do not know even if the second one is valid code or not i tried to change the txt extension beacuse the prompt said no such file '.txt'
I hope some good help for that thank you
Explanation
For recursivity you need Bash >=4 and to enable ** (i.e. globstar) ;
First, I use parameter expansion to remove the string .txt, which must be anchored at the end of the filename (%) :
the # anchors the pattern (plain word or glob) to the beginning,
and the % anchors it to the end.
Then I append the new extension .xml
Be extra cautious with filename, you should always quote parameters expansion.
Code
This should do it in Bash (note that I only echothe old/new filename, to actually rename the files, use mv instead of echo) :
shopt -s globstar # enable ** globstar/recursivity
for i in **/*.txt; do
[[ -d "$i" ]] && continue; # skip directories
echo "$i" "${i/%.txt}.xml";
done
If its a matter of a one or two sub-directories, you can use the rename command:
rename .txt .xml *.txt
This will rename all the .txt to .xml files in the directory from which the command is executed.
If all the files are in same directory, it can be done using a single command. For example you want to convert all jpg files to png, go to the related directory location and then use command
rename .jpg .png *
I wanted to rename "file.txt" to "file.jpg.txt", used rename easy peezy:
rename 's/.txt$/.jpg.txt/' *.txt
man rename will tell you everything you need to know.
Got to love Linux, there's a tool for everything :-)
passing command line argument for dir path
#!/bin/sh
cd $1
names_1=`ls`
for file in ${names_1}
do
mv ${file} ${file}.jpg
done