#!/bin/bash
cp ./Source/* ./Working/ 2> /dev/null
echo "Done"
for filename in *.zip;do unzip “$filename”;
done
In the above script, I am trying to copy all the files from source to working and unzip the files in working folder but I am geting igetting an error unexpected end of file
It looks like you have different kinds of double quotes in "$filename", make sure both are ASCII double quotes (decimal 34, hex 22). Try analyzing your script with
od -c scriptname
You have up to two problems here. First you quotes are not standard. You probably copy pasted from MS Word or something that automatically converts quotes.
The second problem you may have is that your filenames may have spaces in it. This can cause all sorts of problems in scripts if you do not expect it. There are a few workarounds but the easiest is probably to change the IFS:
OLDIFS=$IFS
IFS="$(echo -e '\t\n')" # get rid of the space character in the IFS
... do stuff ...
IFS=$OLDIFS
Related
A script save.sh uses 'cp' and outputs its cp errors to an errors file. Mostly these errors are due to the origin filesystem being EXT4 and the destination filesystem being NTFS or FAT and doesnt accept some specia characters.
Another script onerrors.sh reads the error file so as to best manage files that could not be copied : it copies them toward a crafted filename file that's OK for FAT and NTFS = where "bad" characters have been replaced with '_'.
That works fine for allmost all of the errors, but not 100%.
In this error file, the special characters in the filenames seem to be multiple times escaped :
simple quotes ' appear as '\''.
\n (real newline in filenames!) appear as '$'\n'' (7 glyphs !)
I want to unescape these so as to get the filename.
I convert quotes back to ' with line=${line//\'\\\'\'/\'}. That's OK.
But how can i convert the escaped newline back to a real unescaped \n in $line variable = how can i replace the '$'\n'' to unescaped \n in variable ?
The issue is not in recognising the pattern but in inserting a real newline. I've not been able to do it using same variable expansion syntax.
What other tool is advised or other way of doing it ?
The question is:
how can i replace the '$'\n'' to unescaped \n in variable ?
That's simple:
var="def'$'\n''abc"
echo "${var//\'$\'\\n\'\'/$'\n'}"
I think I remember, that using ANSI C quoting inside variable expansion happened to be buggy in some version of bash. Use a temporary variable in such cases.
What other tool is advised or other way of doing it ?
For string replacement in shell, the most popular tools are sed (which the name literally comes from "String EDitor") and awk. Writing a parser is better done in full-blown programming languages, like Python, C, C++ and similar.
The only way to decode cp output correctly, is to see cp source code, see how it quotes the filenames, and decode it in the same way. Note that it may change between cp flavors and versions, so such a tool may need to query cp version and will be not portable.
Note that parsing cp output is a very very very very bad idea. cp output is in no way standardized, may change anytime and is meant for humans to read. Instead, strongly consider rewriting save.sh to copy file by file and in case of cp returning non-zero exit status, write the filename yourself in an "errors file" as a zero separated stream.
# save.sh
find .... -print0 |
while IFS= read -d '' -r file; do
if ! cp "$file" "$dst"; then
printf "%s\0" "$file" > errorsfile
fi
done
# onerrors.sh
while IFS= read -d '' -r file; do
echo "do something with $file"
done < errorsfile
I am writing a Bash script that creates a CMakeLists.txt file for a project.
The problem arises at this portion:
echo "file(GLOB projSRC src/*.cpp)" >> CMakeLists.txt
After that, I need the program to output ${SOURCES} into CMakeLists.txt
I don't mean a variable in the script named SOURCES, I mean it should actually write the plaintext ${SOURCES}.
What I mean is, the final file should look this like:
arbitrary_command(target PROJECT sources ${SOURCES})
and not like:
arbitrary_command(target PROJECT sources [insert however bash messes it up here])
How can I do this in my Bash script?
Use single quotes, not double quotes, for a literal string:
echo 'file(GLOB ${projSRC} src/*.cpp)' >> CMakeLists.txt
That said, you might consider using a heredoc (or even a quoted heredoc) in this case, to write the entire file as one command:
cat >CMakeLists.txt <<'EOF'
everything here will be emitted to the file exactly as written
${projSRC}, etc
even over multiple lines
EOF
...or, if you want some substitutions, an unquoted heredoc (that is, one where the sigil -- EOF in these examples -- isn't quoted at the start):
foo="this"
cat >CMakeLists.txt <<EOF
here, parameter expansions will be honored, like ${foo}
but can still be quoted: \${foo}
EOF
You can also have multiple commands writing output to a single redirection, to avoid paying the cost of opening your output file more than once:
foo=this
{
echo "Here's a line, which is expanded due to double quotes: ${foo}"
echo 'Here is another line, with no expansion due to single quotes: ${foo}'
} >CMakeLists.txt
May be I don't understand your question...
But
echo \${SOURCES}
will print
${SOURCES}
for you.
I'm trying to pull a list of files over ssh with rsync, but I can't get it to work with filenames that have spaces on it! One example file is this:
/home/pi/Transmission_Downloads/FUNDAMENTOS_JAVA_E_ORIENTAÇÃO_A_OBJETOS/2. Fundamentos da linguagem/estruturas-de-controle-if-else-if-e-else-v1.mp4
and I'm trying to transfer it using this shell code.
cat $file_name | while read LINE
do
echo $LINE
rsync -aP "$user#$server:$LINE" $local_folder
done
and the error I'm getting is this:
receiving incremental file list
rsync: link_stat "/home/pi/Transmission_Downloads/FUNDAMENTOS_JAVA_E_ORIENTAÇÃO_A_OBJETOS/2." failed: No such file or directory (2)
rsync: link_stat "/home/pi/Fundamentos" failed: No such file or directory (2)
rsync: link_stat "/home/pi/da" failed: No such file or directory (2)
rsync: change_dir "/home/pi//linguagem" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [Receiver=3.1.0]
I don't get it why does it print OK on the screen, but parses the file name/path incorrectly! I know spaces are actually backslash with spaces, but don't know how to solve this. Sed (find/replace) didn't help either, and I also tried this code without success
while IFS='' read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
rsync -aP "$user#$server:$line" $local_folder
done < $file_name
What should I do to fix this, and why this is happening?
I read the list of files from a .txt file (each file and path on one line), and I'm using ubuntu 14.04. Thanks!
rsync does space splitting by default.
You can disable this using the -s (or --protect-args) flag, or you can escape the spaces within the filename
The shell is correctly passing the filename to rsync, but rsync interprets spaces as separating multiple paths on the same server. So in addition to double-quoting the variable expansion to make sure rsync sees the string as a single argument, you also need to quote the spaces within the filename.
If your filenames don't have apostrophes in them, you can do that with single quotes inside the double quotes:
rsync -aP "$user#$server:'$LINE'" "$local_folder"
If your filenames might have apostrophes in them, then you need to quote those (whether or not the filenames also have spaces). You can use bash's built-in parameter substitution to do that (as long as you're on bash 4; older versions, such as the /bin/bash that ships on OS X, have issues with backslashes and apostrophes in such expressions). Here's what it looks like:
rsync -aP "$user#$server:'${LINE//\'/\'\\\'\'}'" "$local_folder"
Ugly, I know, but effective. Explanation follows after the other options.
If you're using an older bash or a different shell, you can use sed instead:
rsync -aP "$user#$server:'$(sed "s/'/'\\\\''/g" <<<"$LINE")'" "$local_folder"
... or if your shell also doesn't support <<< here-strings:
rsync -aP "$user#$server:'$(echo "$LINE" | sed "s/'/'\\\\''/g")'" "$local_folder"
Explanation: we want to replace all apostrophes with.. something that becomes a literal apostrophe in the middle of a single-quoted string. Since there's no way to escape anything inside single quotes, we have to first close the quotes, then add a literal apostrophe, and then re-open the quotes for the rest of the string. Effectively, that means we want to replace all occurrences of an apostrophe (') with the sequence (apostrophe, backslash, apostrophe, apostrophe): '\''. We can do that with either bash parameter expansion or sed.
In bash, ${varname/old/new} expands to the value of the variable $varname with the first occurrence of the string old replaced by the string new. Doubling the first slash ( ${varname//old/new} ) replaces all occurrences instead of just the first one. That's what we want here. But since both apostrophe and backslash are special to the shell, we have to put a(nother) backslash in front of every one of those characters in both expressions. That turns our old value into \', and our new one into \'\\\'\'.
The sed version is a little simpler, since apostrophes aren't special. Backslashes still are, so we have to put a \\ in the string to get a \ back. Since we want apostrophes in the string, it's easier to use a double-quoted string instead of a single-quoted one, but that means we need to double all the backslashes again to make sure the shell passes them on to sed unmolested. That's why the shell command has \\\\: that gets handed to sed as \\, which it outputs as \.
I am trying to create a "watch" folder where I will be able to copy files 2 sets of files with the same name, but different file extensions. I have a program that need to reference both files, but since they have the same name, only differing by extension I figure I might be able to do something like this with a cron job
cronjob.sh:
#/bin/bash
ls *.txt > processlist.txt
for filename in 'cat processlist.txt'; do
/usr/local/bin/runcommand -input1=/home/user/process/$filename \
-input2=/home/user/process/strsub($filename, -4)_2.stl \
-output /home/user/process/done/strsub($filename, -4)_2.final;
echo "$filename finished processing"
done
but substr is a php command, not bash. What would be the right way of doing this?
strsub($filename, -4)
in Bash is
${filename:(-4)}
See Shell Parameter Expansion.
Your command can look like
/usr/local/bin/runcommand "-input1=/home/user/process/$filename" \
"-input2=/home/user/process/${filename:(-4)}_2.stl" \
"-output /home/user/process/done/${filename:(-4)}_2.final"
Note: Prefer quoting your arguments with variables around double-quotes to prevent word splitting and possible pathname expansion. This would be helpful to filenames with spaces.
It would also be better to directly pass your glob pattern as an argument to for to properly distribute tokens without getting split with word splitting.
for filename in *.txt; do
So Konsolebox's solution was almost right, but the issue was that when you do ${filename:(-4)} it only returns the last 4 letters of the variable instead of trimming the last 4 off. When I did was change it to ${filename%.txt} where the %.txt matches to the text I want to find and remove, and then just tagged .mp3 on at the end to change the extension.
His other suggestion of using this for loop also was much better than mine:
for filename in *.txt; do
The only other modification was putting the full command all on one line in the end. I divided it up here to make sure it was all easily visible.
This is pretty basic, I guess I'm missing something really obvious...
The following sequence should explain it:
$ cat read_file_names.sh
#!/bin/bash
for i in $# ; do
echo "$i"
done
$ touch "filename has many spaces"
$ ./read_file_names.sh filename\ has\ many\ spaces
filename
has
many
spaces
ideally, the command line will have quotes around the filename as in:
$ ./read_file_names.sh "filename\ has\ many\ spaces"
The problem is that when allowing bash to auto-complete the filename (by hitting tab), the file name is left unquoted. Instead, it has a backslash-space "\ " to signal a space. I understand I can add quotes manually, but that would be tedious and a poor user experience.
I'm looking for a solution which assigns the entire file name to the for-loop variable, so that the output looks something like this:
$ ./read_file_names.sh filename\ has\ many\ spaces
filename has many spaces
The backslashes are working. It's your debugging printer that's wrong:
for i in $# ; do
That needs to be:
for i in "$#"; do
Otherwise, the argument string is inserted unquoted into the for expression and then word-split.