Why are my here-docs (<<-) giving me a syntax error? - bash

methods() {
cat <<-!
start
stop
restart
reload
status
methods
!
}
Is this correct I am getting error
syntax error: unexpected end of file

For here-docs in ancient shells, you have to match the tag exactly. That means:
methods() {
cat <<!
start
stop
restart
reload
status
methods
!
}
Yes, at the start of the line, although you could do tricky things like cat <<'^I!' to set the marker to a single tab followed by !.
Now bash (and possibly earlier shells) fixed that with the <<- variant which strips off all leading tabs from your data lines and the end marker before processing. That way, you could still indent nicely:
methods() {
cat <<-!
start
stop
restart
reload
status
methods
!
}
But, note the proviso: it strips tabs, not whitespace in general. If you have spaces (or any non-tab character, printable or otherwise) anywhere before that ! character, it won't work.
If you're using vi, you can enter :set list to see the non-printable characters a bit better, otherwise xd or od -xcb can give you a hex dump of your file.

Related

'>' Terminal mode activated when I accidentally type an extra apostrophe

What is this mode please may someone enlighten me? Merci
I looked on the interwebs and couldn't find diddly squat at what mode it is.
Unless they're escaped, quotes (') are used to surround a string and prevent the shell from parsing it. When you have a linebreak after an opening quote, you're telling the shell that you want to input a multilined string. The > is the continuation of the that string on another line.
Let's see an example that illustrates this:
mureinik#computer /tmp
$ echo 'this is the first line
> and this is the second
> and now we will terminate the string on the third, with a closing quote -> '
this is the first line
and this is the second
and now we will terminate the string on the third, with a closing quote ->

Way to create multiline comments in Bash?

I have recently started studying shell script and I'd like to be able to comment out a set of lines in a shell script. I mean like it is in case of C/Java :
/* comment1
comment2
comment3
*/`
How could I do that?
Use : ' to open and ' to close.
For example:
: '
This is a
very neat comment
in bash
'
Multiline comment in bash
: <<'END_COMMENT'
This is a heredoc (<<) redirected to a NOP command (:).
The single quotes around END_COMMENT are important,
because it disables variable resolving and command resolving
within these lines. Without the single-quotes around END_COMMENT,
the following two $() `` commands would get executed:
$(gibberish command)
`rm -fr mydir`
comment1
comment2
comment3
END_COMMENT
Note: I updated this answer based on comments and other answers, so comments prior to May 22nd 2020 may no longer apply. Also I noticed today that some IDE's like VS Code and PyCharm do not recognize a HEREDOC marker that contains spaces, whereas bash has no problem with it, so I'm updating this answer again.
Bash does not provide a builtin syntax for multi-line comment but there are hacks using existing bash syntax that "happen to work now".
Personally I think the simplest (ie least noisy, least weird, easiest to type, most explicit) is to use a quoted HEREDOC, but make it obvious what you are doing, and use the same HEREDOC marker everywhere:
<<'###BLOCK-COMMENT'
line 1
line 2
line 3
line 4
###BLOCK-COMMENT
Single-quoting the HEREDOC marker avoids some shell parsing side-effects, such as weird subsitutions that would cause crash or output, and even parsing of the marker itself. So the single-quotes give you more freedom on the open-close comment marker.
For example the following uses a triple hash which kind of suggests multi-line comment in bash. This would crash the script if the single quotes were absent. Even if you remove ###, the FOO{} would crash the script (or cause bad substitution to be printed if no set -e) if it weren't for the single quotes:
set -e
<<'###BLOCK-COMMENT'
something something ${FOO{}} something
more comment
###BLOCK-COMMENT
ls
You could of course just use
set -e
<<'###'
something something ${FOO{}} something
more comment
###
ls
but the intent of this is definitely less clear to a reader unfamiliar with this trickery.
Note my original answer used '### BLOCK COMMENT', which is fine if you use vanilla vi/vim but today I noticed that PyCharm and VS Code don't recognize the closing marker if it has spaces.
Nowadays any good editor allows you to press ctrl-/ or similar, to un/comment the selection. Everyone definitely understands this:
# something something ${FOO{}} something
# more comment
# yet another line of comment
although admittedly, this is not nearly as convenient as the block comment above if you want to re-fill your paragraphs.
There are surely other techniques, but there doesn't seem to be a "conventional" way to do it. It would be nice if ###> and ###< could be added to bash to indicate start and end of comment block, seems like it could be pretty straightforward.
After reading the other answers here I came up with the below, which IMHO makes it really clear it's a comment. Especially suitable for in-script usage info:
<< ////
Usage:
This script launches a spaceship to the moon. It's doing so by
leveraging the power of the Fifth Element, AKA Leeloo.
Will only work if you're Bruce Willis or a relative of Milla Jovovich.
////
As a programmer, the sequence of slashes immediately registers in my brain as a comment (even though slashes are normally used for line comments).
Of course, "////" is just a string; the number of slashes in the prefix and the suffix must be equal.
I tried the chosen answer, but found when I ran a shell script having it, the whole thing was getting printed to screen (similar to how jupyter notebooks print out everything in '''xx''' quotes) and there was an error message at end. It wasn't doing anything, but: scary. Then I realised while editing it that single-quotes can span multiple lines. So.. lets just assign the block to a variable.
x='
echo "these lines will all become comments."
echo "just make sure you don_t use single-quotes!"
ls -l
date
'
what's your opinion on this one?
function giveitauniquename()
{
so this is a comment
echo "there's no need to further escape apostrophes/etc if you are commenting your code this way"
the drawback is it will be stored in memory as a function as long as your script runs unless you explicitly unset it
only valid-ish bash allowed inside for instance these would not work without the "pound" signs:
1, for #((
2, this #wouldn't work either
function giveitadifferentuniquename()
{
echo nestable
}
}
Here's how I do multiline comments in bash.
This mechanism has two advantages that I appreciate. One is that comments can be nested. The other is that blocks can be enabled by simply commenting out the initiating line.
#!/bin/bash
# : <<'####.block.A'
echo "foo {" 1>&2
fn data1
echo "foo }" 1>&2
: <<'####.block.B'
fn data2 || exit
exit 1
####.block.B
echo "can't happen" 1>&2
####.block.A
In the example above the "B" block is commented out, but the parts of the "A" block that are not the "B" block are not commented out.
Running that example will produce this output:
foo {
./example: line 5: fn: command not found
foo }
can't happen
Simple solution, not much smart:
Temporarily block a part of a script:
if false; then
while you respect syntax a bit, please
do write here (almost) whatever you want.
but when you are
done # write
fi
A bit sophisticated version:
time_of_debug=false # Let's set this variable at the beginning of a script
if $time_of_debug; then # in a middle of the script
echo I keep this code aside until there is the time of debug!
fi
in plain bash
to comment out
a block of code
i do
:||{
block
of code
}

How does : <<'END' work in bash to create a multi-line comment block?

I found a great answer for how to comment in bash script (by #sunny256):
#!/bin/bash
echo before comment
: <<'END'
bla bla
blurfl
END
echo after comment
The ' and ' around the END delimiter are important, otherwise things inside the block like for example $(command) will be parsed and executed.
This may be ugly, but it works and I'm keen to know what it means. Can anybody explain it simply? I did already find an explanation for : that it is no-op or true. But it does not make sense to me to call no-op or true anyway....
I'm afraid this explanation is less "simple" and more "thorough", but here we go.
The goal of a comment is to be text that is not interpreted or executed as code.
Originally, the UNIX shell did not have a comment syntax per se. It did, however, have the null command : (once an actual binary program on disk, /bin/:), which ignores its arguments and does nothing but indicate successful execution to the calling shell. Effectively, it's a synonym for true that looks like punctuation instead of a word, so you could put a line like this in your script:
: This is a comment
It's not quite a traditional comment; it's still an actual command that the shell executes. But since the command doesn't do anything, surely it's close enough: mission accomplished! Right?
The problem is that the line is still treated as a command beyond simply being run as one. Most importantly, lexical analysis - parameter substitution, word splitting, and such - still takes place on those destined-to-be-ignored arguments. Such processing means you run the risk of a syntax error in a "comment" crashing your whole script:
: Now let's see what happens next
echo "Hello, world!"
#=> hello.sh: line 1: unexpected EOF while looking for matching `''
That problem led to the introduction of a genuine comment syntax: the now-familiar # (which was first introduced in the C shell created at BSD). Everything from # to the end of the line is completely ignored by the shell, so you can put anything you like there without worrying about syntactic validity:
# Now let's see what happens next
echo "Hello, world!"
#=> Hello, world!
And that's How The Shell Got Its Comment Syntax.
However, you were looking for a multi-line (block) comment, of the sort introduced by /* (and terminated by */) in C or Java. Unfortunately, the shell simply does not have such a syntax. The normal way to comment out a block of consecutive lines - and the one I recommend - is simply to put a # in front of each one. But that is admittedly not a particularly "multi-line" approach.
Since the shell supports multi-line string-literals, you could just use : with such a string as an argument:
: 'So
this is all
a "comment"
'
But that has all the same problems as single-line :. You could also use backslashes at the end of each line to build a long command line with multiple arguments instead of one long string, but that's even more annoying than putting a # at the front, and more fragile since trailing whitespace breaks the line-continuation.
The solution you found uses what is called a here-document. The syntax some-command <<whatever causes the following lines of text - from the line immediately after the command, up to but not including the next line containing only the text whatever - to be read and fed as standard input to some-command. Here's an alternate shell implementation of "Hello, world" which takes advantage of this feature:
cat <<EOF
Hello, world
EOF
If you replace cat with our old friend :, you'll find that it ignores not only its arguments but also its input: you can feed whatever you want to it, and it will still do nothing (and still indicate that it did that nothing successfully).
However, the contents of a here-document do undergo string processing. So just as with the single-line : comment, the here-document version runs the risk of syntax errors inside what is not meant to be executable code:
#!/bin/sh -e
: <<EOF
(This is a backtick: `)
EOF
echo 'In modern shells, $(...) is preferred over backticks.'
#=> ./demo.sh: line 2: bad substitution: no closing "`" in `
The solution, as seen in the code you found, is to quote the end-of-document "sentinel" (the EOF or END or whatever) on the line introducing the here document (e.g. <<'EOF'). Doing this causes the entire body of the here-document to be treated as literal text - no parameter expansion or other processing occurs. Instead, the text is fed to the command unchanged, just as if it were being read from a file. So, other than a line consisting of nothing but the sentinel, the here-document can contain any characters at all:
#!/bin/sh -e
: <<'EOF'
(This is a backtick: `)
EOF
echo 'In modern shells, $(...) is preferred over backticks.'
#=> In modern shells, $(...) is preferred over backticks.
(It is worth noting that the way you quote the sentinel doesn't matter - you can use <<'EOF', <<E"OF", or even <<EO\F; all have the same result. This is different from the way here-documents work in some other languages, such as Perl and Ruby, where the content is treated differently depending on the way the sentinel is quoted.)
Notwithstanding any of the above, I strongly recommend that you instead just put a # at the front of each line you want to comment out. Any decent code editor will make that operation easy - even plain old vi - and the benefit is that nobody reading your code will have to spend energy figuring out what's going on with something that is, after all, intended to be documentation for their benefit.
It is called a Here Document. It is a code block that lets you send a list of commands to another command or program
The string following the << is the marker determining the end of the block. If you send commands to no-op, nothing happens, which is why you can use it as a comment block.
That's heredoc syntax. It's a way of defining multi-line string literals.
As the answer at your link explains, the single quotes around the END disables interpolation, similar to the way single-quoted strings disable interpolation in regular bash strings.

How to strip ANSI escape sequences from a variable?

Weird question. When I set a variable in Bash to display as a certain color, I don't know how to reset it. Here is an example:
First define the color code:
YELLOW=$(tput setaf 3)
RESET=$(tput sgr0)
Now set the error message variable and color part of it.
ERROR="File not found: "$YELLOW"Length.db$RESET"
This sets the variable ERROR as the error message to be returned from a function that will eventually be displayed on the terminal. The error will be all white with the exception of the file name. The file name is highlighted yellow for the user.
This works great except when logging with rsyslog. When the error message gets logged, it comes out something like this:
File not found: #033[33mLength.db#033(B#033[m
This obviously makes log files very difficult to read. At first I figured I could process using sed the error message immediately after outputting to the terminal but before logging, but there is nothing to search for and replace. ie, I thought I could use sed to do something similar to this:
ERROR=$(echo "$ERROR" | sed -r 's%\#033\[33m%%')
But those characters are not present when you echo the variable (which makes sense since you dont see it on the terminal). So im stuck. I dont know how to reset the color of the variable after setting it. I also tried to reverse the process somehow using $RESET but maybe my syntax is wrong or something.
You almost had it. Try this instead:
ERROR=$(echo "$ERROR" | sed 's%\o033\[33m%%g')
Note, however, that the use of the \oNNN escape sequence in sed is a GNU extension, and thus not POSIX compliant. If that is an issue, you should be able to do something more like:
ERROR=$(echo "$ERROR" | sed 's%'$(echo -en "\033")'\[33m%%g')
Obviously, this will only work for this one specific color (yellow), and a regex to remove any escape sequence (such as other colors, background colors, cursor positioning, etc) would be somewhat more complicated.
Also note that the -r is not required, since nothing here is using the extended regular expression syntax. I'm guessing you already know that, and that you just included the -r out of habit, but I mention it anyway just for the sake of clarity.
Here is a pure Bash solution:
ERROR="${ERROR//$'\e'\[*([0-9;])m/}"
Make it a function:
# Strips ANSI codes from text
# 1: The text
# >: The ANSI stripped text
function strip_ansi() {
shopt -s extglob # function uses extended globbing
printf %s "${1//$'\e'\[*([0-9;])m/}"
}
See:
Bash Shell Parameter Expansion
Bash Pattern-Matching

cat a file inside a shell function

I am wondering how I can implement something like the following:
test(){
cat>file<<'EOF'
abc
EOF
}
Many thanks.
Qiang:
Remove the spaces in front of EOF (so it's on a line by itself and not indented).
From bash(1):
If the redirection operator is <<-, then all leading tab
characters are stripped from input lines and the line
containing delimiter. This allows here-documents within
shell scripts to be indented in a natural fashion.
It says tab, and in my testing, tab works, but spaces do not:
#!/bin/bash
cat>file <<-END
hello
world
hello
END
echo done
(All those indents are tabs; the funny thing about the four-leading-spaces markup for code means only four spaces show up in the rendered text, too.)
Your code should work just fine, is there anything specific you are looking for?
#!/bin/sh
input() {
cat > file <<EOF
input
line
another line
EOF
}
input
EDIT: Changed function input to input()

Resources