In Git Bash, when I run the below command:
echo "foo" | clip
and paste the output into a text editor, I see a new line character has been appended to the output string:
foo
<newline>
How can I prevent it from adding the trailing newline character or trim the same after getting appended?
Directly from echo(1) man page:
-n do not output the trailing newline
so just add the -n option to the command: echo -n "foo" | clip.
Related
I am new to shell script. I am sourcing a file, which is created in Windows and has carriage returns, using the source command. After I source when I append some characters to it, it always comes to the start of the line.
test.dat (which has carriage return at end):
testVar=value123
testScript.sh (sources above file):
source test.dat
echo $testVar got it
The output I get is
got it23
How can I remove the '\r' from the variable?
yet another solution uses tr:
echo $testVar | tr -d '\r'
cat myscript | tr -d '\r'
the option -d stands for delete.
You can use sed as follows:
MY_NEW_VAR=$(echo $testVar | sed -e 's/\r//g')
echo ${MY_NEW_VAR} got it
By the way, try to do a dos2unix on your data file.
Because the file you source ends lines with carriage returns, the contents of $testVar are likely to look like this:
$ printf '%q\n' "$testVar"
$'value123\r'
(The first line's $ is the shell prompt; the second line's $ is from the %q formatting string, indicating $'' quoting.)
To get rid of the carriage return, you can use shell parameter expansion and ANSI-C quoting (requires Bash):
testVar=${testVar//$'\r'}
Which should result in
$ printf '%q\n' "$testVar"
value123
use this command on your script file after copying it to Linux/Unix
perl -pi -e 's/\r//' scriptfilename
Pipe to sed -e 's/[\r\n]//g' to remove both Carriage Returns (\r) and Line Feeds (\n) from each text line.
for a pure shell solution without calling external program:
NL=$'\n' # define a variable to reference 'newline'
testVar=${testVar%$NL} # removes trailing 'NL' from string
I am new to shell script. I am sourcing a file, which is created in Windows and has carriage returns, using the source command. After I source when I append some characters to it, it always comes to the start of the line.
test.dat (which has carriage return at end):
testVar=value123
testScript.sh (sources above file):
source test.dat
echo $testVar got it
The output I get is
got it23
How can I remove the '\r' from the variable?
yet another solution uses tr:
echo $testVar | tr -d '\r'
cat myscript | tr -d '\r'
the option -d stands for delete.
You can use sed as follows:
MY_NEW_VAR=$(echo $testVar | sed -e 's/\r//g')
echo ${MY_NEW_VAR} got it
By the way, try to do a dos2unix on your data file.
Because the file you source ends lines with carriage returns, the contents of $testVar are likely to look like this:
$ printf '%q\n' "$testVar"
$'value123\r'
(The first line's $ is the shell prompt; the second line's $ is from the %q formatting string, indicating $'' quoting.)
To get rid of the carriage return, you can use shell parameter expansion and ANSI-C quoting (requires Bash):
testVar=${testVar//$'\r'}
Which should result in
$ printf '%q\n' "$testVar"
value123
use this command on your script file after copying it to Linux/Unix
perl -pi -e 's/\r//' scriptfilename
Pipe to sed -e 's/[\r\n]//g' to remove both Carriage Returns (\r) and Line Feeds (\n) from each text line.
for a pure shell solution without calling external program:
NL=$'\n' # define a variable to reference 'newline'
testVar=${testVar%$NL} # removes trailing 'NL' from string
I am new to shell script. I am sourcing a file, which is created in Windows and has carriage returns, using the source command. After I source when I append some characters to it, it always comes to the start of the line.
test.dat (which has carriage return at end):
testVar=value123
testScript.sh (sources above file):
source test.dat
echo $testVar got it
The output I get is
got it23
How can I remove the '\r' from the variable?
yet another solution uses tr:
echo $testVar | tr -d '\r'
cat myscript | tr -d '\r'
the option -d stands for delete.
You can use sed as follows:
MY_NEW_VAR=$(echo $testVar | sed -e 's/\r//g')
echo ${MY_NEW_VAR} got it
By the way, try to do a dos2unix on your data file.
Because the file you source ends lines with carriage returns, the contents of $testVar are likely to look like this:
$ printf '%q\n' "$testVar"
$'value123\r'
(The first line's $ is the shell prompt; the second line's $ is from the %q formatting string, indicating $'' quoting.)
To get rid of the carriage return, you can use shell parameter expansion and ANSI-C quoting (requires Bash):
testVar=${testVar//$'\r'}
Which should result in
$ printf '%q\n' "$testVar"
value123
use this command on your script file after copying it to Linux/Unix
perl -pi -e 's/\r//' scriptfilename
Pipe to sed -e 's/[\r\n]//g' to remove both Carriage Returns (\r) and Line Feeds (\n) from each text line.
for a pure shell solution without calling external program:
NL=$'\n' # define a variable to reference 'newline'
testVar=${testVar%$NL} # removes trailing 'NL' from string
Inside a Makefile I run a shell command which I want to pass a NULL byte as argument. The following attempt fails:
echo $(shell /bin/echo -n $$'\x00' | ruby -e "puts STDIN.read.inspect")
It generates:
echo "$\\x00"
Instead I expected:
echo "\u0000"
How do I properly escape such a NULL byte?
echo disables interpretation of backslash escapes by default. You need to supply the -e option to enable it.
$ echo -ne "\x00" | ruby -e "puts STDIN.read.inspect"
"\u0000"
Due to the execve(2) semantics it is not possible to pass a string containing a null byte as argument. Each argument string is terminated by null byte, therefore making it impossible to distinguish between the contained null byte and the end of the string.
These uses of echo are totally non-portable. Use printf, it's much easier to use for anything other than the simplest strings, and much more portable.
$ cat makefile
all:
printf '\0' > foo.out
od -a foo.out
$ make
printf '\0' > foo.out
od -a foo.out
0000000 nul
0000001
You can't use NUL as argument in bash
You can't use $'\0' as an argument, store it as a variable or using command substitution $(printf '\0') since bash (and most shells?) use C-strings that are null terminated. The leading string before NUL is interpreted as the string and the trailing string discarded.
You can only input using pipes - printf '\0' | cat -v or letting the resulting program use a file for input.
Use another means of input
Most programs that work on input with line strings NUL strings (xargs, cut, ...) typically have a -z flag. This is primarily used when dealing with paths as a character may contain ANY character EXCEPT NUL.
Programs like find and git ls-files support outputting this format, usually in the form of a -print0 or -0 flag.
Programs like sed, tr, bash et. al. use special escape characters like \0, \x0, \x00 to generate NUL bytes.
Massage the input
OP originally seems to have wanted to know how to use cut with a NUL delimiter. The problem is typically that something is separated using \n, where \n is a valid part of the values and not a line-separator (typically in paths).
Say you have a situation where you group files, each separated by a NUL character, and the groups separated by \n.
# Temporary mock output with NUL columns and newline rows
printf 'foo\0bar\nbar\0\nFOO\0BAR\0\n' > /tmp/$$.output
A work-around is to get creative with a combination of sed, awk or tr to massage the output to something that suits our input/commands.
our.sed
#!/usr/bin/sed -nf
# beginning
:x
# match \0\n
/\x0$/ {
# Change \0\n to \n
s|\x0$|\n|g
# print
p
# delete
d
}
# match \n with no leading \0
/[^\x0]$/ {
# change \0 to \1
s|\x0|\x1|g
# read next line
N
# branch to beginning
bx
}
In this scenario we map:
\0\n => \n
\0 not followed by \n => \1
While a valid character in a filename, it's unlikely to pose a problem.
# Change NUL to another unlikely to be used control character
sed -f our.sed /tmp/$$.output |\
cut -d $'\x1' -f 2
output
bar
BAR
If anyone else came here looking how to escape a null via a shell command in ruby backticks:
irb(main):024:0> `curl --silent http://some-website-or-stream.com | sed 's/\\x0//g' 1>&2`
=> ""
I am new to shell script. I am sourcing a file, which is created in Windows and has carriage returns, using the source command. After I source when I append some characters to it, it always comes to the start of the line.
test.dat (which has carriage return at end):
testVar=value123
testScript.sh (sources above file):
source test.dat
echo $testVar got it
The output I get is
got it23
How can I remove the '\r' from the variable?
yet another solution uses tr:
echo $testVar | tr -d '\r'
cat myscript | tr -d '\r'
the option -d stands for delete.
You can use sed as follows:
MY_NEW_VAR=$(echo $testVar | sed -e 's/\r//g')
echo ${MY_NEW_VAR} got it
By the way, try to do a dos2unix on your data file.
Because the file you source ends lines with carriage returns, the contents of $testVar are likely to look like this:
$ printf '%q\n' "$testVar"
$'value123\r'
(The first line's $ is the shell prompt; the second line's $ is from the %q formatting string, indicating $'' quoting.)
To get rid of the carriage return, you can use shell parameter expansion and ANSI-C quoting (requires Bash):
testVar=${testVar//$'\r'}
Which should result in
$ printf '%q\n' "$testVar"
value123
use this command on your script file after copying it to Linux/Unix
perl -pi -e 's/\r//' scriptfilename
Pipe to sed -e 's/[\r\n]//g' to remove both Carriage Returns (\r) and Line Feeds (\n) from each text line.
for a pure shell solution without calling external program:
NL=$'\n' # define a variable to reference 'newline'
testVar=${testVar%$NL} # removes trailing 'NL' from string