Description of Task & problem
I have dumped a list of files which match certain criteria to a text file. The command i used is:
find . -name "*.logic" | grep -v '.bak' | grep -v 'Project File Backup' > logic_manifest.txt
Filenames with spaces in are proving difficult to automatically open, such as :
./20160314 _ Pop/20160314 _ Pop.logic
I have replaced the spaces with '\ ' to escape them but the open command complains:
The file /Users/daniel/Music/Logic/20160314\ _\ Pop/20160314\ _\ Pop.logic does not exist.
When I copy that parsed path, type open in the terminal and paste it in, the file opens successfully.
My BASH script:
#!/bin/bash
clear
# file full of file paths, gathered using the find command
#logic_manifest.txt
# For keeping track of which line of the file I'm using
COUNTER=0
it=1
while IFS='' read -r line || [[ -n "$line" ]]; do
# Increment iterator
COUNTER=`expr $COUNTER + $it`
# replace spaces with a black-slash and space
line=${line// /<>}
line=${line//<>/'\ '}
# print the file name and the line it is on
echo "Line: $COUNTER $line"
#open the file
open "$line"
# await key press before moving on to next iterator
read input </dev/tty
done < "$1"
Encapsulating the filename in speech-marks has not helped
line=${line// /<>}
line=${line//<>/'\ '}
line="\"$line\""
The file /Users/daniel/Music/Logic/"./20160314\ _\ Pop/20160314\ _\
Pop.logic" does not exist.
Nor did passing "\${line}" to open
Question
What do I need to do to enable the open command to launch the files successfully?
Renaming the directories and filenames is not a viable option at this time.
Spaces in filenames are bad, I know, I put it down to moments of madness
There is absolutely no need whatsoever to replace any characters in line.
This simpler loop should open files just fine:
while IFS='' read -r line; do
((COUNTER++))
echo "Line: $COUNTER $line"
open "$line"
read input </dev/tty
done < "$1"
That's it. Moreover:
Spaces in filenames are bad, I know, I put it down to moments of madness.
There's nothing wrong with spaces in filenames.
You just have to use proper quoting, that's all.
That is, if the file names didn't have spaces and other special characters in them, then you could write open $line and it would work.
Since they contain spaces, you must enclose the variable in double-quotes, as in open "$line".
Actually it's strongly recommended to enclose variables in double-quotes when used in command line arguments.
Related
I've written a script to go through all the files in the directory the script is located in, identify if a file name contains a certain string and then modify the filename. When I run this script, the files that are supposed to be modified are disappearing. It appears my usage of the mv command is incorrect and the files are likely going to an unknown directory.
#!/bin/bash
string_contains="dummy_axial_y_position"
string_dontwant="dummy_axial_y_position_time"
file_extension=".csv"
for FILE in *
do
if [[ "$FILE" == *"$string_contains"* ]];then
if [[ "$FILE" != *"$string_dontwant"* ]];then
filename= echo $FILE | head -c 15
combined_name="$filename$file_extension"
echo $combined_name
mv $FILE $combined_name
echo $FILE
fi
fi
done
I've done my best to go through the possible errors I've made in the MV command but I haven't had any success so far.
There are a couple of problems and several places where your script can be improved.
filename= echo $FILE | head -c 15
This pipeline runs echo $FILE adding the variable filename having the null string as value in its environment. This value of the variable is visible only to the echo command, the variable is not set in the current shell. echo does not care about it anyway.
You probably want to capture the output of echo $FILE | head -c 15 into the variable filename but this is not the way to do it.
You need to use command substitution for this purpose:
filename=$(echo $FILE | head -c 15)
head -c outputs only the first 15 characters of the input file (they can be on multiple lines but this does not happen here). head is not the most appropriate way for this. Use cut -c-15 instead.
But for what you need (extract the first 15 characters of the value stored in the variable $FILE), there is a much simpler way; use a form of parameter expansion called "substring expansion":
filename=${FILE:0:15}
mv $FILE $combined_name
Before running mv, the variables $FILE and $combined_name are expanded (it is called "parameter expansion"). This means that the variable are replaced by their values.
For example, if the value of FILE is abc def and the value of combined_name is mnp opq, the line above becomes:
mv abc def mnp opq
The mv command receives 4 arguments and it attempts to move the files denoted by the first three arguments into the directory denoted by the fourth argument (and it probably fails).
In order to keep the values of the variables as single words (if they contain spaces), always enclose them in double quotes. The correct command is:
mv "$FILE" "$combined_name"
This way, in the example above, the command becomes:
mv "abc def" "mnp opq"
... and mv is invoked with two arguments: abc def and mnp opq.
combined_name="$filename$file_extension"
There isn't any problem in this line. The quotes are simply not needed.
The variables filename and file_extension are expanded (replaced by their values) but on assignments word splitting is not applied. The value resulted after the replacement is the value assigned to variable combined_name, even if it contains spaces or other word separator characters (spaces, tabs, newlines).
The quotes are also not needed here because the values do not contain spaces or other characters that are special in the command line. They must be quoted if they contain such characters.
string_contains="dummy_axial_y_position"
string_dontwant="dummy_axial_y_position_time"
file_extension=".csv"
It is not not incorrect to quote the values though.
for FILE in *
do
if [[ "$FILE" == *"$string_contains"* ]];then
if [[ "$FILE" != *"$string_dontwant"* ]]; then
This is also not wrong but it is inefficient.
You can use the expression from the if condition directly in the for statement (and get rid of the if statement):
for FILE in *"$string_contains"*; do
if [[ "$FILE" != *"$string_dontwant"* ]]; then
...
If you have read and understood the above (and some of the linked documentation) you will be able to figure out yourself where were your files moved :-)
I write a bash script that ask user to input file in full path:
printf ' Please type the path of ISO file:\n'
read -p ' ' berkas
if [[ -f $berkas ]]
then
printf " $berkas found\n"
else
printf " Could not find $berkas\n"
fi
But it failed if file path is single quoted (e.g if i drag file from Nautilus window to script windows):
Please type the path of ISO file:
'/home/iza/Software/Windows/ISO/Windows7_Ultimate_x64_SP1.iso'
Could not find '/home/iza/Software/Windows/ISO/Windows7_Ultimate_x64_SP1.iso'
What's wrong with the code? I would love if workaround is not involving sed or awk.
Thanx :)
You can remove the quotes from the start and end of the variable in bash using:
berkas=${berkas%\'} # remove single quote from end of variable if it exists
berkas=${berkas#\'} # remove single quote from start of variable if it exists
If no quotes are present, nothing will be removed.
It's generally considered good practice to put double quotes around your variables, for example
[[ -f "$berkas" ]]
Also, rather than using printf and appending a \n newline character manually, you can just use echo:
echo ' Please type the path of ISO file:'
read -r -p ' ' berkas
if [[ "$berkas" = \'*\' ]]; then
berkas=${berkas%\'}
berkas=${berkas#\'}
fi
if [[ -f "$berkas" ]]
then
echo " $berkas found"
else
echo " Could not find $berkas"
fi
As suggested in the comments (thanks Etan), I have added a check that the variable starts and ends with a single quote, which makes the script slightly safer. I also added the -r switch to read, which you almost always want to use.
disclaimer
It is worth mentioning that this approach indiscriminately removes quotes from the start and/or end of the variable, so if it possible that your filenames legitimately may contain quotes in those positions, this will not work.
I have a source file that is a combination of multiple files that have been merged together. My script is supposed to separate them into the original individual files.
Whenever I encounter a line that starts with "FILENM", that means that it's the start of the next file.
All of the detail lines in the files are fixed width; so, I'm currently encountering a problem where a line that starts with leading whitespaces is truncated when it's not supposed to be truncated.
How do I enhance this script to retain the leading whitespaces?
while read line
do
lineType=`echo $line | cut -c1-6`
if [ "$lineType" == "FILENM" ]; then
fileName=`echo $line | cut -c7-`
else
echo "$line" >> $filePath/$fileName
fi
done <$filePath/sourcefile
The leading spaces are removed because read splits the input into words. To counter this, set the IFS variable to empty string. Like this:
OLD_IFS="$IFS"
IFS=
while read line
do
...
done <$filePath/sourcefile
IFS="$OLD_IFS"
To preserve IFS variable you could write while in the following way:
while IFS= read line
do
. . .
done < file
Also to preserve backslashes use read -r option.
I'm having an error trying to find a way to replace a string in a directory path with another string
sed: Error tryning to read from {directory_path}: It's a directory
The shell script
#!/bin/sh
R2K_SOURCE="source/"
R2K_PROCESSED="processed/"
R2K_TEMP_DIR=""
echo " Procesando archivos desde $R2K_SOURCE "
for file in $(find $R2K_SOURCE )
do
if [ -d $file ]
then
R2K_TEMP_DIR=$( sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g' $file )
echo "directorio $R2K_TEMP_DIR"
else
# some code executes
:
fi
done
# find $R2K_PROCCESED -type f -size -200c -delete
i'm understanding that the rror it's in this line
R2K_TEMP_DIR=$( sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g' $file )
but i don't know how to tell sh that treats $file variable as string and not as a directory object.
If you want ot replace part of path name you can echo path name and take it to sed over pipe.
Also you must enable globbing by placing sed commands into double quotes instead of single and change separator for 's' command like that:
R2K_TEMP_DIR=$(echo "$file" | sed "s:$R2K_SOURCE:$R2K_PROCESSED:g")
Then you will be able to operate with slashes inside 's' command.
Update:
Even better is to remove useless echo and use "here is string" instead:
R2K_TEMP_DIR=$(sed "s:$R2K_SOURCE:$R2K_PROCESSED:g" <<< "$file")
First, don't use:
for item in $(find ...)
because you might overload the command line. Besides, the for loop cannot start until the process in $(...) finishes. Instead:
find ... | while read item
You also need to watch out for funky file names. The for loop will cough on all files with spaces in them. THe find | while will work as long as files only have a single space in their name and not double spaces. Better:
find ... -print0 | while read -d '' -r item
This will put nulls between file names, and read will break on those nulls. This way, files with spaces, tabs, new lines, or anything else that could cause problems can be read without problems.
Your sed line is:
R2K_TEMP_DIR=$( sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g' $file )
What this is attempting to do is edit your $file which is a directory. What you want to do is munge the directory name itself. Therefore, you have to echo the name into sed as a pipe:
R2K_TEMP_DIR=$(echo $file | sed 's/"$R2K_SOURCE"/"$R2K_PROCESSED"/g')
However, you might be better off using environment variable parameters to filter your environment variable.
Basically, you have a directory called source/ and all of the files you're looking for are under that directory. You simply want to change:
source/foo/bar
to
processed/foo/bar
You could do something like this ${file#source/}. The # says this is a left side filter and it will remove the least amount to match the glob expression after the #. Check the manpage for bash and look under Parameter Expansion.
This, you could do something like this:
#!/bin/sh
R2K_SOURCE="source/"
R2K_PROCESSED="processed/"
R2K_TEMP_DIR=""
echo " Procesando archivos desde $R2K_SOURCE "
find $R2K_SOURCE -print0 | while read -d '' -r file
do
if [ -d $file ]
then
R2K_TEMP_DIR="processed/${file#source/}"
echo "directorio $R2K_TEMP_DIR"
else
# some code executes
:
fi
done
R2K_TEMP_DIR="processed/${file#source/}" removes the source/ from the start of $file and you merely prepend processed/ in its place.
Even better, it's way more efficient. In your original script, the $(..) creates another shell process to run your echo in which then pipes out to another process to run sed. (Assuming you use loentar's solution). You no longer have any subprocesses running. The whole modification of your directory name is internal.
By the way, this should also work too:
R2K_TEMP_DIR="$R2K_PROCESSED/${file#$R2K_SOURCE}"
I just didn't test that.
I am writing a simple shell script to make automated backups, and I am trying to use basename to create a list of directories and them parse this list to get the first and the last directory from the list.
The problem is: when I use basename in the terminal, all goes fine and it gives me the list exactly as I want it. For example:
basename -a /var/*/
gives me a list of all the directories inside /var without the / in the end of the name, one per line.
BUT, when I use it inside a script and pass a variable to basename, it puts single quotes around the variable:
while read line; do
dir_name=$(echo $line)
basename -a $dir_name/*/ > dir_list.tmp
done < file_with_list.txt
When running with +x:
+ basename -a '/Volumes/OUTROS/backup/test/*/'
and, therefore, the result is not what I need.
Now, I know there must be a thousand ways to go around the basename problem, but then I'd learn nothing, right? ;)
How to get rid of the single quotes?
And if my directory name has spaces in it?
If your directory name could include spaces, you need to quote the value of dir_name (which is a good idea for any variable expansion, whether you expect spaces or not).
while read line; do
dir_name=$line
basename -a "$dir_name"/*/ > dir_list.tmp
done < file_with_list.txt
(As jordanm points out, you don't need to quote the RHS of a variable assignment.)
Assuming your goal is to populate dir_list.tmp with a list of directories found under each directory listed in file_with_list.txt, this might do.
#!/bin/bash
inputfile=file_with_list.txt
outputfile=dir_list.tmp
rm -f "$outputfile" # the -f makes rm fail silently if file does not exist
while read line; do
# basic syntax checking
if [[ ! ${line} =~ ^/[a-z][a-z0-9/-]*$ ]]; then
continue
fi
# collect targets using globbing
for target in "$line"/*; do
if [[ -d "$target" ]]; then
printf "%s\n" "$target" >> $outputfile
fi
done
done < $inputfile
As you develop whatever tool will process your dir_list.tmp file, be careful of special characters (including spaces) in that file.
Note that I'm using printf instead of echo so that targets whose first character is a hyphen won't cause errors.
This might work
while read; do
find "$REPLY" >> dir_list.tmp
done < file_with_list.txt