I am using Windows and MobaXterm.
I created a .bash_profile file in the ~ directory and the following line
alias sbp="source ~/.bash_profile"
is the only code in that file.
However, when I was trying to do sbp, I got an error.
This works on my Mac and it used to work on my old Windows computer (but that one has some water damage so it broke down). Why does this not work now?
Thanks in advance!
From the way that error message is garbled I'm pretty sure that the .bash_profile file you created has DOS/Windows-style line endings, consisting of a carriage return character followed by a newline character. Unix tools expect unix-style line endings consisting of just a newline; if they see DOS/Windows-style endings, they'll treat the carriage return as part of the content of the line. In this case, bash will treat the carriage return as part of the alias definition, and therefore part of the filename to filename to source. Try running alias sbp | cat -vt to print the alias with invisible characters shown; my guess is it'll print alias sbp='source ~/.bash_profile^M' (where the ^M is cat -vt's way of representing the carriage return).
Solution: convert the file to unix format, and either switch to a text editor that knows how to save in unix format, or change your settings in the current editor to do it. For conversion, there are a number of semi-standard tools like dos2unix and fromdos. If you don't have any of those, this answer has some other options.
BTW, the reason the error message is garbled is that the CR gets printed as part of the error message, and the terminal treats that as an instruction to go back to the beginning of the line; it then prints the rest of the message over top of the beginning of the message. It's a little like this:
-bash: /home/dir/path/.bash_profile
: No such file or directory
...but with the second line printed over the first, so it comes out as:
: No such file or directory_profile
Related
I want to download a huge file from s3 to a server so I'm using nohup.
Problem is, the process outputs "self-updating reports", which are cool in the terminal but are horrible when written to a file as a single long line.
My questions are:
How should I handle this output to avoid a 84MB text file of just one line?
Given that I have such a file, how can I read its "bottom line" effectively?
thanks!
The "self-updating" text is just a carriage return character, which when printed to the terminal moves the cursor back to the beginning of line, allowing you to overwrite any previous text on the line.
To remove everything up through the last carriage return character,
awk '{ sub(/.*\r/, ""); }1' file
or the same with any regex tool (sed 's/.*\r//' file would work if your sed recognizes the nonstandard \r escape).
Some tools have an option to turn off progress updates; if you are using the standard aws s3 cp, try adding the option --no-progress.
I have following snippet:
host="https://example.com"
port="80"
url="${host}:${port}"
echo $url
the output is:
:80ps://example.com
How can I escape the colon here. I also tried:
url="${host}\:${port}"
but it did not work.
Expected output is:
https://example.com:80
You've most likely run into what I call the Linefeed-Limbo.
If I copy the code you provided from StackOverflow and run it on my machine (bash version 4.4.19(1)), then it outputs correctly
user#host:~$ cat script.sh
host="https://example.com"
port="80"
url="${host}:${port}"
echo $url
user#host:~$ bash script.sh
https://example.com:80
What is Linefeed-Limbo?
Different operating systems use different ASCII symbols to represent when a new line occurs in a text, such as a script. This Wikipedia article gives a good introduction.
As you can see, Unix and Unix-like systems use the single character \n, also called a "Line Feed". Windows, as well as other systems, use \r\n, so a "carriage return" followed by a "line feed".
What happens now is when you write a script on Windows on an editor such as notepad, what you write is host="example.com"\r\n. When you copy this file into Linux, Linux interprets the \r as if it were part of the script, since only \n is considered a new line. And indeed, when I change my newline style to DOS-style, I get the exact output you get.
How can I fix this?
You have several options to fix this issue.
Converting the script (with dos2unix)
Since all you need to do is replacing every instance of \r\n with \n, you could use any text-editing software you want. However, if you like simple solutions, then dos2unix (and its sister unix2dos) might be what you looking for:
user#host:~$ dos2unix script.sh
dos2unix: converting file script.sh to Unix format...
That's it. Run your file now and you will see it behaves well.
Encoding the source-file correctly
By using a more advanced text editor such as Notepad++, you can define which style of newline you would like to use.
By changing the newline-type to whichever system you intend to run your script on, you will not run into any problems like this anymore.
Bonus round: Why does it output :80ps://example.com?
To understand why your output is like this, you have to look at what your script is doing, and what \r means.
Try thinking of your terminal as an old-fashioned typewriter. Returning the carriage means you start writing on the left again. Making a "new line" means sliding the paper. These two things are seperate, and I think that's why some systems decided to use these two characters as a logical "new line".
But I digress. Let's look at the first line, host="https://example.com"\r.
What this means when printed is "Print https://example.com, then put the carriage back at the start". When you then print :80\r, it doesn't start after ".com", it starts at the beginning of the line, because that's where you (unknowingly) told the cursor to go. it then overwites the first few characters, resulting in ":80ps://example.com" to be written. Keep in mind that after 80, you again placed a carriage return symbol, so any new text you would have written ends up overwriting the beginning again.
It works for me, try to remove carriage returns in variables and then try.
new_host=$(echo "$host" | tr -d '\r')
new_port=$(echo "$port" | tr -d '\r')
new_url="${new_host}:${new_port}"
When running bash scripts in cygwin from the Windows command prompt, I always get a error on line 1, even when the script runs correctly
C:\ImageMagickWatchFolder>bash .\whitebalance
.\whitebalance: line 1: #!/bin/bash: No such file or directory
... (rest of script output)
(whitebalance is the filename of the script)
I've tried replacing line 1 of the script with
#!/cygdrive/c/cygwin64/bash
#!bash
#!C:\cygwin64\bin\bash
#!C:\\cygwin64\\bin\\bash
among others, and just eliminating it entirely, all to no avail. (Some give No such file or directory and some give command not found). In particular, with first line empty, it gives
C:\ImageMagickWatchFolder>bash .\whitebalance
.\whitebalance: line 1: $'\357\273\277': command not found
I also tried switching line ending styles from unix to windows but that just made things worse.
To the first comment below:
C:\ImageMagickWatchFolder>head -n 2 whitebalance | cat -A
M-oM-;M-?#!/bin/bash$
coords=""$
Figured it out. I had been saving the script in Notepad++ in UTF-8-BOM encoding. When changed to UTF-8 or ANSI, the error no longer occurred.
It's impossible to tell from here, but one common way this happens is because of line endings. Make sure your script does not have \r\n (Windows-style) line endings. bash will ignore these but the shebang will still not work: it will try to find /bin/bash\r, and fail.
I'm attempting to apply a patch to a file with Windows line endings on a Linux system and I'm getting conflicts due to the carriage returns in the file.
The -l option (ignore whitespace) isn't ignoring the EOL characters. Is there anyway to get patch to ignore windows style line endings?
Try using the --binary option, from the manpage (emphasis mine)
--binary
Write all files in binary mode, except for standard output and /dev/tty. When reading, disable the heuristic for transforming CRLF line endings into LF line endings. (On POSIX -conforming systems, reads and writes never transform line endings. On Windows, reads and writes do transform line endings by default, and patches should be generated by diff --binary when line endings are significant.)
I don't fully understand the above, but it worked for me on a Linux machine to apply a Unix patch onto a DOS file.
Here's a link http://www.chemie.fu-berlin.de/chemnet/use/info/diff/diff_2.html
The -w' and--ignore-all-space' options ignore difference even if one file has white space >where the other file has none. White space characters include tab, newline, vertical tab, >form feed, carriage return, and space
Run diff like: diff -w file1.txt file2.txt
I work around this using the following commands to convert all files of interest to unix line endings.
dos2unix `grep Index\: mixed-line-ending.patch | sed -e 's/Index\://'`
dos2unix mixed-line-ending.patch
patch -p0 < mixed-line-ending.patch
I had this problem with a diff that was manually copied and pasted from git diff console output, into a patch file with LFs. To get that patch file to work again - to be able to be applied on the actual files that were using CRs and LFs - several things had to be done manually:
find all instances of "^M" and drop them
add CR to all lines within the hunks - but not the meta format lines (## etc)
on all lines within hunks that were empty, add the missing space in the first column
joe syntax highlighting was very helpful there, because it colored hunks properly as soon as I fixed them.
Tell patch to ignore white space:
-l, --ignore-whitespace
Causes the pattern matching to be done loosely, in case the tabs
and spaces have been munged in your input file. Any sequence of
whitespace in the pattern line will match any sequence in the
input file. Normal characters must still match exactly. Each
line of the context must still match a line in the input file.
This ignores mismatches of EOLs too -- at least, on FreeBSD, using patch version 2.0-12u11.
I once write a simple sed command like this
s/==/EQU/
while I run it in command line:
sed 's/==/EQU' filename
it works well, replace the '==' with 'EQU', but while I write the command to a script file named replace.sed, run it in this way:
sed -f replace.sed filename
there is a error, says that
sed: file replace.sed line 1: unknwon option to 's'
What I want to ask is that is there any problem with my script file replace.sed while it run in windows?
The unknown option is almost invariably a rogue character after the trailing / (which is missing from your command line version, by the way so it should complain about an unterminated command).
Have a look at you replace.sed again. You may have a funny character at the end, which could include the ' if you forgot to delete it, or even a CTRL-M DOS-style line ending, though CygWin seems to handle this okay - you haven't specified which sed you're using (that may help).
Okay, based on your edit, it looks like one of my scattergun of suggestions was right :-) You had CTRL-M at the end of the line because of the CR/LF line endings:
At the end of each line in the *.sed file, there was a 'CR\LF' pair, and that the problem, but you cannot see it by default, I use notepad to delete them manually and fix the problem. But I have not find a way to delete it automatically or do not contain the 'new-line' style while edit a new text file in windows.
You may want to get your hands on a more powerful editor like Notepad++ or gVim (my favourite) but, in fact, you do have a tool that can get rid of those characters :-) It's called sed.
sed 's/\015//g' replace.sed >replace2.sed
should get rid of all the CR characters from your file and give you a replace2.sed that you can use for your real job.