How to write fancy-indented multi-line brace expansion in Bash? - bash

I'm dealing with a line such :
mkdir -p "$DEST_ROOT_PATH/"{"$DEST_DIR1","$DEST_DIR2", ..., "$DEST_DIRN"}
This line is quite long. I want to cut it so its width will fit into a 80 columns line. I tried to escape an end of line with a backslash, but space alignement breaks the expansion :
$ echo "ha"{a,b,\
> c}
ha{a,b, c}

You could use this disgusting hack.
echo "ha"{a,b,\
> ` `c}
It opens a subshell with nothing in it, but gets processed before the expansion so the expansion just sees an empty space

This is the normal behaviour. From the Bash reference manual:
3.5.1 Brace expansion
Brace expansion is a mechanism by which arbitrary strings may be
generated. This mechanism is similar to filename expansion (see
Filename Expansion), but the filenames generated need not exist.
Patterns to be brace expanded take the form of an optional preamble, followed by either a series of comma-separated strings or a sequence
expression between a pair of braces, followed by an optional
postscript. The preamble is prefixed to each string contained within
the braces, and the postscript is then appended to each resulting
string, expanding left to right.
Brace expansion does not allow spaces in between elements that get placed between \ and the next element in the following line.
And why? Because it gets removed when being processed:
3.1.2.1 Escape Character
A non-quoted backslash ‘\’ is the Bash escape character. It preserves
the literal value of the next character that follows, with the
exception of newline. If a \newline pair appears, and the backslash
itself is not quoted, the \newline is treated as a line continuation
(that is, it is removed from the input stream and effectively
ignored).
So when you say
something + \ + <new line> + another_thing
Bash converts it into
something + another_thing
What can you do, then?
Add a backslash and then start writing from the very beginning on the next line:
mkdir -p "$DEST_ROOT_PATH/"{"$DEST_DIR1",\
"$DEST_DIR2",\
...,\
"$DEST_DIRN"}
Some examples
When you say:
$ echo "ha"{a,b\
> c}
ha{a,b c}
And then move the arrow up you'll see this is the command that was performed:
$ echo "ha"{a,b c}
So just say:
$ echo "ha"{a,b\
> c}
haa habc
And you'll see this when moving up:
$ echo "ha"{a,b,c}
Another example:
$ cat touch_files.sh
touch X{1,\
2,3}
$ bash touch_files.sh
$ ls X*
X1 X2 X3

Thus I accepted #123's answer, here's the one I choosed :
mkdir -p "$DEST_ROOT_PATH/"{"$DEST_DIR1","$DEST_DIR2"}
mkdir -p "$DEST_ROOT_PATH/"{"$DEST_DIR3","$DEST_DIR4"}
There are not a lot of destination directories here, so I think it's a good balance between the fancy-and-disgusting hack and the frustrating backslash which breaks the indentation.

I would do it is follows (though it only addresses your particular task of creating multiple directories and doesn't answer the question as stated in the title):
for d in \
"$DEST_DIR1" \
"$DEST_DIR2" \
... \
"$DEST_DIRn" \
;
do
mkdir -p "$DEST_ROOT_PATH/$d"
done
The advantage of this approach is that maintaining the list is a little easier.
In general, you should stop sticking to syntactic sugar when you notice that it starts causing inconveniences.

Related

Deeply nesting commands

To escape characters in bash, Why the syntax is confusing when nesting commands deeply?, I know that there is an alternate approach with $() to nest commands, Just curious, why it is as such when nesting commands using backticks!
For example:
echo `echo \`echo \\\`echo inside\\\`\``
Gives output: inside
But
echo `echo \`echo \\`echo inside\\`\``
Fails with,
bash: command substitution: line 1: unexpected EOF while looking for matching ``'
bash: command substitution: line 2: syntax error: unexpected end of file
bash: command substitution: line 1: unexpected EOF while looking for matching ``'
bash: command substitution: line 2: syntax error: unexpected end of file
echo inside\
My question is that why the number of backslashes required for second level nesting is 3 and why it is not 2. In the above example given, one backslash is used for one level deep and three are used for second-level nesting commands to preserve the literal meaning of the backtick.
The basic problem is that there's no distinction between an open-backtick and a close-backtick. So if the shell sees something like this:
somecommand ` something1 ` something2 ` something3 `
...there's no intrinsic way to tell if that's two separate backticked commands (something1 and something3), with a literal string ("something2") in between; or a nested backtick expression, with something2 being run first and its output passed to something1 as an argument (along with the literal string "something3"). In order to avoid ambiguity, the shell syntax picks the first interpretation, and requires that if you want the second interpretation you need to escape the inner level of backticks:
somecommand ` something1 ` something2 ` something3 ` # Two separate expansions
somecommand ` something1 \` something2 \` something3 ` # Nested expansions
And that means adding another level of parsing-and-removing escapes, which means you need to escape any escapes you didn't want parsed at that point, and the whole thing gets quickly out of hand.
The $( ) syntax, on the other hand, is not ambiguous, because the opening and closing markers are not the same. Compare the two possibilities:
somecommand $( something1 ) something2 $( something3 ) # Two separate expansions
somecommand $( something1 $( something2 ) something3 ) # Nested expansions
There's no ambiguity there, so no need for escapes or other syntactic weirdness.
The reason the number of escapes grows so fast with the number of levels is again to avoid ambiguity. And it's not something specific to command expansions with backticks; this escape inflation shows up anytime you have a string going through multiple levels of parsing, each of which applies (and removes) escapes.
Suppose the shell runs across two escapes and a backtick (\\`) as it parses a line. Should it parse that as a doubly-escaped backtick, or a singly-escaped escape (backslash) character followed by a not-escaped-at-all backtick? If it runs across three escapes and a backtick (\\\`), is that a triply-escaped backtick, a doubly-escaped escape followed a not-escaped-at-all backtick, or a singly-escaped escape followed by a singly-escaped backtick?
The shell (like most things that deal with escapes) avoids the ambiguity by not treating stacked escapes as a special thing. When it runs into an escape character, that applies only to the thing immediately after it; if the thing immediately after it is another escape, then it escapes that one character and has no effect on whatever's after it. Thus \\` is an escaped escape, followed by a not-escaped-at-all backtick. That means you can't just add another escape to the front, you have to add an escape in front of each and every escape-worthy character in the string (including escapes from lower levels).
So, let's start with a simple backtick, and work through escaping it to various levels:
First level is easy, just escape it: \'.
For the second level, we have to escape that escape (\\) and then separately escape the backtick itself (\`), giving a total of three backticks: \\\`.
For the third level, we have to individually escape each of those three escapes (so 3x\\) and once again escape the backtick itself (\`), giving a total of seven backticks: \\\\\\\`.
It continues like that, more than doubling the number of escapes for each level. From 7 it goes to 15, then 31, then 63, then... There's a good reason people try to avoid situations with deeply nested escapes.
Oh, and as I mentioned, the shell isn't the only thing that does this, and that can complicate matters because different levels can have different escaping syntaxes, and some things may not need escaping at some of the levels. For example, suppose the thing being escaped is the regular expression \s. To add a level to that, you'd only need one additional escape (\\s) because the "s" doesn't need to be escaped by itself. Additional levels of escaping on that would give 4, 8, 16, 32 etc escapes.
TLDR; Yo, dawg, I heard you like escapes...
P.s. You can use the shell's -v option to make it print commands before executing them. With nested commands like this, it'll print each of the commands as it un-nests them, so you can watch the stack escaped escapes collapse as the layers get stripped off:
$ set -v
$ echo "this is `echo "a literal \`echo "backtick: \\\\\\\`" \`" `"
echo "this is `echo "a literal \`echo "backtick: \\\\\\\`" \`" `"
echo "a literal `echo "backtick: \\\`" `"
echo "backtick: \`"
this is a literal backtick: `
(For even more fun, try this after set -vx -- the -x option will print the commands after parsing, so after you see it drill into the nested commands, you'll then see what happens as it unwinds back out to the final top-level command.)
There is nothing confusing per se in the syntax that you have shown. You just need to breakdown each of the levels one by one.
The GNU bash man page says
When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \.
Command substitutions may be nested. To nest when using the backquoted form, escape the inner backquotes with backslashes.
So with that in context, the nested substitution has one \ to escape the back-quote and one more to escape the escape character (now read the above quote that \ loses its special meaning except when followed by another \). So that's the reason the second level of escaping needs two additional backslashes to escape the original character
echo `echo \`echo \\\`echo inside\\\`\``
# ^^^^ ^^^^
becomes
echo `echo \`echo inside\``
# ^^ ^^
which in turn becomes
echo `echo inside`
# ^ ^
which eventually becomes
echo inside

Is this some kind of regex within bash variable instantiation?

What do those two assignations (i and C omitting the first one to void) do? Is it some kind of regex for the variable? I tried with bash, but so far there were no changes in the output of my strings after instantiating them with "${i//\\/\\\\}" or "\"${i//\"/\\\"}\""
C=''
for i in "$#"; do
i="${i//\\/\\\\}"
C="$C \"${i//\"/\\\"}\""
done
${i//\\/\\\\} is a slightly complicated-looking parameter expansion:
It expands the variable $i in the following way:
${i//find/replace} means replace all instances of "find" with "replace". In this case, the thing to find is \, which itself needs escaping with another \.
The replacement is two \, which each need escaping.
For example:
$ i='a\b\c'
$ echo "${i//\\/\\\\}"
a\\b\\c
The next line performs another parameter expansion:
find " (which needs to be escaped, since it is inside a double-quoted string)
replace with \" (both the double quote and the backslash need to be escaped).
It looks like the intention of the loop is to build a string C, attempting to safely quote/escape the arguments passed to the script. This type of approach is generally error-prone, and it would probably be better to work with the input array directly. For example, the arguments passed to the script can be safely passed to another command like:
cmd "$#" # does "the right thing" (quotes each argument correctly)
if you really need to escape the backslashes, you can do that too:
cmd "${#//\\/\\\\}" # replaces all \ with \\ in each argument
It's bash parameter expansions
it replace all backslashes by double backslashes :"${i//\\/\\\\}
it replace all \" by \\" : ${i//\"/\\\"}
Check http://wiki.bash-hackers.org/syntax/pe

bash print words on multiple lines in a single line

I am writing a shell script for which I write a header that has 30 (growing) column names. Right now, I have a echo statement that works and looks like this
echo "Colum_Name1, Column_Name2,Column_Name30"
While this works the readability sucks for me. if i want to add a column, its a bit of a nightmare to look at the screen and understand whether it is already in there. of course, I search my way out. Is it possible to do something like this with echo or printf and get the CSV in one line?
echo " Column_Name1,
Column_Name2,
Column_Name30"
and get the output as
Column_Name1,Column_Name2,Column_Name30
You can add backslash as the line continuation:
echo " Column_Name1,"\
"Column_Name2,"\
"Column_Name30"
From the bash manual:
The backslash character ‘\’ may be used to remove any special meaning
for the next character read and for line continuation.
Decouple the definition of the header and printing it, and use an array to store the column names.
headers=(
Column_Name1
Column_Name2
Column_Name30
)
(IFS=","; printf '%s\n' "${headers[*]}")
The elements of the array are joined by the first character of IFS when ${headers[*]} is expanded. The subshell is used so you don't have to worry about restoring the previous value of IFS.
Convenience solution, using paste:
If you don't mind the (probably negligible) overhead of invoking an external utility (paste) to build your string, you can combine it with a (literal, in this case) here-doc:
paste -s -d, - <<'EOF'
Column_Name1
Column_Name2
Column_Name30
EOF
yields
Column_Name1,Column_Name2,Column_Name30
The above acts like a single-quoted string, due to the opening delimiter, 'EOF', being quoted.
Omit the enclosing '...' to treat the string like a double-quoted string, i.e., with expansions being performed (allowing the inclusion of variable references, command substitutions, and arithmetic expansions).
If you take care to use actual leading tabs (\t) in your here-doc (multiple spaces do not work), you can even introduce indentation, by prepending - to the opening delimiter:
# !! Only works with actual *tabs* as the leading whitespace.
paste -s -d, - <<-'EOF'
Column_Name1
Column_Name2
Column_Name30
EOF
More efficient solution, using line continuation:
POSIX-compatible shells support line continuation even inside double-quoted strings, "..." (but not inside single-quoted ones, '...').
That means that any \<newline> sequence inside a double-quoted string is removed:
echo "\
Column_Name1,\
Column_Name2,\
Column_Name3\
"
Given that a here-document with an unquoted opening delimiter is treated like a double-quoted string, you can do the following:
cat <<EOF
Column_Name1,\
Column_Name2,\
Column_Name30
EOF
Note:
Using <<-EOF with to-be-stripped leading tabs (\t) for readability is not an option here, because the line continuations will still include them.
To take advantage of line continuation, it is invariably the interpolating (expanding) here-doc variety that must be used; therefore, you may need to \-escape $ instances to ensure their literal use.
Both commands again yield the desired single-line string:
Column_Name1,Column_Name2,Column_Name30
echo "foo bar" | (IFS=" "; xargs -n 1 echo)
yields
foo
bar

Delete all comments in a file using sed

How would you delete all comments using sed from a file(defined with #) with respect to '#' being in a string?
This helped out a lot except for the string portion.
If # always means comment, and can appear anywhere on a line (like after some code):
sed 's:#.*$::g' <file-name>
If you want to change it in place, add the -i switch:
sed -i 's:#.*$::g' <file-name>
This will delete from any # to the end of the line, ignoring any context. If you use # anywhere where it's not a comment (like in a string), it will delete that too.
If comments can only start at the beginning of a line, do something like this:
sed 's:^#.*$::g' <file-name>
If they may be preceded by whitespace, but nothing else, do:
sed 's:^\s*#.*$::g' <file-name>
These two will be a little safer because they likely won't delete valid usage of # in your code, such as in strings.
Edit:
There's not really a nice way of detecting whether something is in a string. I'd use the last two if that would satisfy the constraints of your language.
The problem with detecting whether you're in a string is that regular expressions can't do everything. There are a few problems:
Strings can likely span lines
A regular expression can't tell the difference between apostrophies and single quotes
A regular expression can't match nested quotes (these cases will confuse the regex):
# "hello there"
# hello there"
"# hello there"
If double quotes are the only way strings are defined, double quotes will never appear in a comment, and strings cannot span multiple lines, try something like this:
sed 's:#[^"]*$::g' <file-name>
That's a lot of pre-conditions, but if they all hold, you're in business. Otherwise, I'm afraid you're SOL, and you'd be better off writing it in something like Python, where you can do more advanced logic.
This might work for you (GNU sed):
sed '/#/!b;s/^/\n/;ta;:a;s/\n$//;t;s/\n\(\("[^"]*"\)\|\('\''[^'\'']*'\''\)\)/\1\n/;ta;s/\n\([^#]\)/\1\n/;ta;s/\n.*//' file
/#/!b if the line does not contain a # bail out
s/^/\n/ insert a unique marker (\n)
ta;:a jump to a loop label (resets the substitute true/false flag)
s/\n$//;t if marker at the end of the line, remove and bail out
s/\n\(\("[^"]*"\)\|\('\''[^'\'']*'\''\)\)/\1\n/;ta if the string following the marker is a quoted one, bump the marker forward of it and loop.
s/\n\([^#]\)/\1\n/;ta if the character following the marker is not a #, bump the marker forward of it and loop.
s/\n.*// the remainder of the line is comment, remove the marker and the rest of line.
Since there is no sample input provided by asker, I will assume a couple of cases and Bash is the input file because bash is used as the tag of the question.
Case 1: entire line is the comment
The following should be sufficient enough in most case:
sed '/^\s*#/d' file
It matches any line has which has none or at least one leading white-space characters (space, tab, or a few others, see man isspace), followed by a #, then delete the line by d command.
Any lines like:
# comment started from beginning.
# any number of white-space character before
# or 'quote' in "here"
They will be deleted.
But
a="foobar in #comment"
will not be deleted, which is the desired result.
Case 2: comment after actual code
For example:
if [[ $foo == "#bar" ]]; then # comment here
The comment part can be removed by
sed "s/\s*#*[^\"']*$//" file
[^\"'] is used to prevent quoted string confusion, however, it also means that comments with quotations ' or " will not to be removed.
Final sed
sed "/^\s*#/d;s/\s*#[^\"']*$//" file
To remove comment lines (lines whose first non-whitespace character is #) but not shebang lines (lines whose first characters are #!):
sed '/^[[:space:]]*#[^!]/d; /#$/d' file
The first argument to sed is a string containing a sed program consisting of two delete-line commands of the form /regex/d. Commands are separated by ;. The first command deletes comment lines but not shebang lines. The second command deletes any remaining empty comment lines. It does not handle trailing comments.
The last argument to sed is a file to use as input. In Bash, you can also operate on a string variable like this:
sed '/^[[:space:]]*#[^!]/d; /#$/d' <<< "${MYSTRING}"
Example:
# test.sh
S0=$(cat << HERE
#!/usr/bin/env bash
# comment
# indented comment
echo 'FOO' # trailing comment
# last line is an empty, indented comment
#
HERE
)
printf "\nBEFORE removal:\n\n${S0}\n\n"
S1=$(sed '/^[[:space:]]*#[^!]/d; /#$/d' <<< "${S0}")
printf "\nAFTER removal:\n\n${S1}\n\n"
Output:
$ bash test.sh
BEFORE removal:
#!/usr/bin/env bash
# comment
# indented comment
echo 'FOO' # trailing comment
# last line is an empty, indented comment
#
AFTER removal:
#!/usr/bin/env bash
echo 'FOO' # trailing comment
Supposing "being in a string" means "occurs between a pair of quotes, either single or double", the question can be rephrased as "remove everything after the first unquoted #". You can define the quoted strings, in turn, as anything between two quotes, excepting backslashed quotes. As a minor refinement, replace the entire line with everything up through just before the first unquoted #.
So we get something like [^\"'#] for the trivial case -- a piece of string which is neither a comment sign, nor a backslash, nor an opening quote. Then we can accept a backslash followed by anything: \\. -- that's not a literal dot, that's a literal backslash, followed by a dot metacharacter which matches any character.
Then we can allow zero or more repetitions of a quoted string. In order to accept either single or double quotes, allow zero or more of each. A quoted string shall be defined as an opening quote, followed by zero or more of either a backslashed arbitrary character, or any character except the closing quote: "\(\\.\|[^\"]\)*" or similarly for single-quoted strings '\(\\.\|[^\']\)*'.
Piecing all of this together, your sed script could look something like this:
s/^\([^\"'#]*\|\\.\|"\(\\.\|[^\"]\)*"\|'\(\\.\|[^\']\)*'\)*\)#.*/\1/
But because it needs to be quoted, and both single and double quotes are included in the string, we need one more additional complication. Recall that the shell allows you to glue together strings like "foo"'bar' gets replaced with foobar -- foo in double quotes, and bar in single quotes. Thus you can include single quotes by putting them in double quotes adjacent to your single-quoted string -- '"foo"'"'" is "foo" in single quotes next to ' in double quotes, thus "foo"'; and "' can be expressed as '"' adjacent to "'". And so a single-quoted string containing both double quotes foo"'bar can be quoted with 'foo"' adjacent to "'bar" or, perhaps more realistically for this case 'foo"' adjacent to "'" adjacent to another single-quoted string 'bar', yielding 'foo'"'"'bar'.
sed 's/^\(\(\\.\|[^\#"'"'"']*\|"\(\\.\|[^\"]\)*"\|'"'"'\(\\.\|[^\'"'"']\)*'"'"'\)*\)#.*/\1/p' file
This was tested on Linux; on other platforms, the sed dialect may be slightly different. For example, you may need to omit the backslashes before the grouping and alteration operators.
Alas, if you may have multi-line quoted strings, this will not work; sed, by design, only examines one input line at a time. You could build a complex script which collects multiple lines into memory, but by then, switching to e.g. Perl starts to make a lot of sense.
As you have pointed out, sed won't work well if any parts of a script look like comments but actually aren't. For example, you could find a # inside a string, or the rather common $# and ${#param}.
I wrote a shell formatter called shfmt, which has a feature to minify code. That includes removing comments, among other things:
$ cat foo.sh
echo $# # inline comment
# lone comment
echo '# this is not a comment'
[mvdan#carbon:12] [0] [/home/mvdan]
$ shfmt -mn foo.sh
echo $#
echo '# this is not a comment'
The parser and printer are Go packages, so if you'd like a custom solution, it should be fairly easy to write a 20-line Go program to remove comments in the exact way that you want.
sed 's:^#\(.*\)$:\1:g' filename
Supposing the lines starts with single # comment, Above command removes all comments from file.

How to split strings over multiple lines in Bash?

How can i split my long string constant over multiple lines?
I realize that you can do this:
echo "continuation \
lines"
>continuation lines
However, if you have indented code, it doesn't work out so well:
echo "continuation \
lines"
>continuation lines
This is what you may want
$ echo "continuation"\
> "lines"
continuation lines
If this creates two arguments to echo and you only want one, then let's look at string concatenation. In bash, placing two strings next to each other concatenate:
$ echo "continuation""lines"
continuationlines
So a continuation line without an indent is one way to break up a string:
$ echo "continuation"\
> "lines"
continuationlines
But when an indent is used:
$ echo "continuation"\
> "lines"
continuation lines
You get two arguments because this is no longer a concatenation.
If you would like a single string which crosses lines, while indenting but not getting all those spaces, one approach you can try is to ditch the continuation line and use variables:
$ a="continuation"
$ b="lines"
$ echo $a$b
continuationlines
This will allow you to have cleanly indented code at the expense of additional variables. If you make the variables local it should not be too bad.
Here documents with the <<-HERE terminator work well for indented multi-line text strings. It will remove any leading tabs from the here document. (Line terminators will still remain, though.)
cat <<-____HERE
continuation
lines
____HERE
See also http://ss64.com/bash/syntax-here.html
If you need to preserve some, but not all, leading whitespace, you might use something like
sed 's/^ //' <<____HERE
This has four leading spaces.
Two of them will be removed by sed.
____HERE
or maybe use tr to get rid of newlines:
tr -d '\012' <<-____
continuation
lines
____
(The second line has a tab and a space up front; the tab will be removed by the dash operator before the heredoc terminator, whereas the space will be preserved.)
For wrapping long complex strings over many lines, I like printf:
printf '%s' \
"This will all be printed on a " \
"single line (because the format string " \
"doesn't specify any newline)"
It also works well in contexts where you want to embed nontrivial pieces of shell script in another language where the host language's syntax won't let you use a here document, such as in a Makefile or Dockerfile.
printf '%s\n' >./myscript \
'#!/bin/sh` \
"echo \"G'day, World\"" \
'date +%F\ %T' && \
chmod a+x ./myscript && \
./myscript
You can use bash arrays
$ str_array=("continuation"
"lines")
then
$ echo "${str_array[*]}"
continuation lines
there is an extra space, because (after bash manual):
If the word is double-quoted, ${name[*]} expands to a single word with
the value of each array member separated by the first character of the
IFS variable
So set IFS='' to get rid of extra space
$ IFS=''
$ echo "${str_array[*]}"
continuationlines
In certain scenarios utilizing Bash's concatenation ability might be appropriate.
Example:
temp='this string is very long '
temp+='so I will separate it onto multiple lines'
echo $temp
this string is very long so I will separate it onto multiple lines
From the PARAMETERS section of the Bash Man page:
name=[value]...
...In the context where an assignment statement is assigning a value to a shell variable or array index, the += operator can be used to append to or add to the variable's previous value. When += is applied to a variable for which the integer attribute has been set, value is evaluated as an arithmetic expression and added to the variable's current value, which is also evaluated. When += is applied to an array variable using compound assignment (see Arrays below), the variable's value is not unset (as it is when using =), and new values are appended to the array beginning at one greater than the array's maximum index (for indexed arrays) or added as additional key-value pairs in an associative array. When applied to a string-valued variable, value is expanded and appended to the variable's value.
You could simply separate it with newlines (without using backslash) as required within the indentation as follows and just strip of new lines.
Example:
echo "continuation
of
lines" | tr '\n' ' '
Or if it is a variable definition newlines gets automatically converted to spaces. So, strip of extra spaces only if applicable.
x="continuation
of multiple
lines"
y="red|blue|
green|yellow"
echo $x # This will do as the converted space actually is meaningful
echo $y | tr -d ' ' # Stripping of space may be preferable in this case
This isn't exactly what the user asked, but another way to create a long string that spans multiple lines is by incrementally building it up, like so:
$ greeting="Hello"
$ greeting="$greeting, World"
$ echo $greeting
Hello, World
Obviously in this case it would have been simpler to build it one go, but this style can be very lightweight and understandable when dealing with longer strings.
Line continuations also can be achieved through clever use of syntax.
In the case of echo:
# echo '-n' flag prevents trailing <CR>
echo -n "This is my one-line statement" ;
echo -n " that I would like to make."
This is my one-line statement that I would like to make.
In the case of vars:
outp="This is my one-line statement" ;
outp+=" that I would like to make." ;
echo -n "${outp}"
This is my one-line statement that I would like to make.
Another approach in the case of vars:
outp="This is my one-line statement" ;
outp="${outp} that I would like to make." ;
echo -n "${outp}"
This is my one-line statement that I would like to make.
Voila!
I came across a situation in which I had to send a long message as part of a command argument and had to adhere to the line length limitation. The commands looks something like this:
somecommand --message="I am a long message" args
The way I solved this is to move the message out as a here document (like #tripleee suggested). But a here document becomes a stdin, so it needs to be read back in, I went with the below approach:
message=$(
tr "\n" " " <<-END
This is a
long message
END
)
somecommand --message="$message" args
This has the advantage that $message can be used exactly as the string constant with no extra whitespace or line breaks.
Note that the actual message lines above are prefixed with a tab character each, which is stripped by here document itself (because of the use of <<-). There are still line breaks at the end, which are then replaced by tr with spaces.
Note also that if you don't remove newlines, they will appear as is when "$message" is expanded. In some cases, you may be able to workaround by removing the double-quotes around $message, but the message will no longer be a single argument.
Depending on what sort of risks you will accept and how well you know and trust the data, you can use simplistic variable interpolation.
$: x="
this
is
variably indented
stuff
"
$: echo "$x" # preserves the newlines and spacing
this
is
variably indented
stuff
$: echo $x # no quotes, stacks it "neatly" with minimal spacing
this is variably indented stuff
Following #tripleee 's printf example (+1):
LONG_STRING=$( printf '%s' \
'This is the string that never ends.' \
' Yes, it goes on and on, my friends.' \
' My brother started typing it not knowing what it was;' \
" and he'll continue typing it forever just because..." \
' (REPEAT)' )
echo $LONG_STRING
This is the string that never ends. Yes, it goes on and on, my friends. My brother started typing it not knowing what it was; and he'll continue typing it forever just because... (REPEAT)
And we have included explicit spaces between the sentences, e.g. "' Yes...". Also, if we can do without the variable:
echo "$( printf '%s' \
'This is the string that never ends.' \
' Yes, it goes on and on, my friends.' \
' My brother started typing it not knowing what it was;' \
" and he'll continue typing it forever just because..." \
' (REPEAT)' )"
This is the string that never ends. Yes, it goes on and on, my friends. My brother started typing it not knowing what it was; and he'll continue typing it forever just because... (REPEAT)
Acknowledgement for the song that never ends
However, if you have indented code, it doesn't work out so well:
echo "continuation \
lines"
>continuation lines
Try with single quotes and concatenating the strings:
echo 'continuation' \
'lines'
>continuation lines
Note: the concatenation includes a whitespace.
This probably doesn't really answer your question but you might find it useful anyway.
The first command creates the script that's displayed by the second command.
The third command makes that script executable.
The fourth command provides a usage example.
john#malkovich:~/tmp/so$ echo $'#!/usr/bin/env python\nimport textwrap, sys\n\ndef bash_dedent(text):\n """Dedent all but the first line in the passed `text`."""\n try:\n first, rest = text.split("\\n", 1)\n return "\\n".join([first, textwrap.dedent(rest)])\n except ValueError:\n return text # single-line string\n\nprint bash_dedent(sys.argv[1])' > bash_dedent
john#malkovich:~/tmp/so$ cat bash_dedent
#!/usr/bin/env python
import textwrap, sys
def bash_dedent(text):
"""Dedent all but the first line in the passed `text`."""
try:
first, rest = text.split("\n", 1)
return "\n".join([first, textwrap.dedent(rest)])
except ValueError:
return text # single-line string
print bash_dedent(sys.argv[1])
john#malkovich:~/tmp/so$ chmod a+x bash_dedent
john#malkovich:~/tmp/so$ echo "$(./bash_dedent "first line
> second line
> third line")"
first line
second line
third line
Note that if you really want to use this script, it makes more sense to move the executable script into ~/bin so that it will be in your path.
Check the python reference for details on how textwrap.dedent works.
If the usage of $'...' or "$(...)" is confusing to you, ask another question (one per construct) if there's not already one up. It might be nice to provide a link to the question you find/ask so that other people will have a linked reference.

Resources