I have a Python project which is a bunch of Docker containers and terraform scripts. My workstation is Windows 10 so I have installed Ubuntu in WSL to develop it. The problem I am facing is there are a lot of shell scripts (.sh) that I need to run as the build process. In Git it always checks out the files as Windows style and commit files as Unix Style. As a result, the Ubuntu bash shell does not like these shell scripts and I have to run dos2unix before running a file. Is there a one liner that allows me to run these scripts without modifying them?
If the scripts don't call another script you could
tr "\r\n" "\n" script.sh | bash
But in the other case I think you need to change your repository config to tell git to don't convert files with .sh extension. Look at this answer https://stackoverflow.com/a/39461324/5032541
If you want to change git configuration to remove auto conversion, take a look on official documentation. But it's a bad idea.
The main problem imo is checking in the wrong file endings into git.
An easy way to fix all shell scripts at once is:
find . -type f -name "*.sh" -exec dos2unix {} \;
It will recursively find all files with the .sh extension and perform dos2unix on it.
I would recommend setting your git repo to checkout linux style rather than windows line endings.
I do a lot of node/docker/linux work but have a windows 10 laptop, this is what I have done.
git config --global core.autocrlf input
See this link for some more details. This way you dont need to continuously run scripts to make sure you have the correct line endings. Also a good thing to do is set you editor to default to linux line endings.
Related
I installed Git For Windows 2.7.2 a few days ago. Now I have some problems with using optional tools Git For Windows provides from cmd.exe. In the previous version of Git For Windows(or say msysgit), I could configure these tools by modifying Git/etc/git-completion.bash. For example:
alias ls='ls --show-control-chars --color=auto'
I used this way to make ls display file names that contained Chinese characters normally. Now it seems this way doesn't work. In fact there is no git-completion.bash under Git/etc/. There is a git-completion.bash under the folder Git/mingw64/share/git/completion. I tried to copy it to Git/etc and add the alias above, which didn't work either. These tools only works fine in Git Bash. So how should I configure these tools together with git to use them from cmd.exe?
This answer explains why you no longer have this functionality. In short, msysgit provided a unix shell emulator, mingw. Git for Windows is git compiled in a Windows environment.
Therefore, ls is simply an alias for dir in a Windows shell, not mingw's ls. If you want to create some Windows aliases, you can use doskey. Here's an answer for that.
Alternatively, I would suggest that you just start using PowerShell, where you'll be able to set up the $profile variable with some powerful commands like these.
Synopsis
Pass arguments to git bash using a .bat file
OR
Use git some other way by running a batch file.
Disclaimer
There are a lot of answers to similar questions out there, but having tried and failed most or all of them, I hope you don't smack me with a DUPLICATE stamp at first sight.
My system
Windows 7, 64 bit
git version 1.9.4.msysgit.2
The challenge
I want to rationalize my workflow from navigating to a number of git repos and running git status, git add --all etc, to simply run a batch.
Other posts certainly suggests it's possible. I think most of them use Unix, but this one at least got me up and running on Windows so I could test it out. However, I'm having the same problems as OP in that post when it comes to passing the commands to the git bash, and to a complete beginner like me it seems a bit complicated to use the suggestions from #Erik further down in the same post if you want to include more commands in the work flow.
Following the suggestions from #inf3rno in the same post, it seems I'm able to change folders but not able to use git commands like this:
set bash=C:\Program Files (x86)\Git\bin\bash.exe
"%bash%" --login -i -c "exec "%1""
cd c:\repos\research
git status
pause
Does anyone have suggestions as to how I can change the code above to get it working?
And by the way, what's the deal with #!bin\bash from other posts like this one? I assumed I had to use #!Program Files (x86)\Git\bin\bash, but I had no luck with that either...
Anyway, thank you for any suggestions!
First, uninstall git 1.9.4 (msygit, which is now obsolete): git-for-windows will offer a much recent bash (2013 vs. 2005).
Simply unzip PortableGit-2.6.1-64-bit.7z.exe anywhere you want, add C:\path\PortableGit-2.6.1-64-bit\bin to your %PATH% and you are good to go.
Second, inf3rno's answer is about executing any bash script ("%bash%" --login -i -c "exec "%1"": the %1 is the path/name of the bash script to be executed)
The right shebang to use in your bash scripts would be #!/bin/bash (see "What is the preferred Bash shebang?")
With the latest git 2.6, that would be:
c:\prgs\git\PortableGit-2.6.1-64-bit\bin\bash.exe --login -i -c "exec ./myscript"
Since that folder is supposed to be in your %PATH%:
bash --login -i -c "exec ./myscript"
With myscript being a file using Unix-style eol (LF), not Windows-style (CRLF)
Note also that any bash script (even on Windows) called git-myscript can be directly called with:
git myscript
I described in 2012 another approach in "Running a batch file in git shell" for executing git command.
But for a pure bash script, you will want to go with bash --login -i -c "exec ./myscript".
For writing bash scripts with unix eol style, you can choose various editor from Notepad++, SublimeText 3 or Atom.io.
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed last year.
I'm trying to install YouCompleteMe from here.
When I execute:
./install.sh --clang-completer
I get this error:
env: bash\r: No such file or directory
I don't know what's wrong with environment variables. Here's my bash path:
which bash
/bin/bash
Do I need to change it to /usr/bash? If yes, then how should I do that? I tried changing ~/.bashrc file, but it didn't work.
The error message suggests that the script you're invoking has embedded \r characters, which in turn suggests that it has Windows-style \r\n line endings instead of the \n-only line endings bash expects.
As a quick fix, you can remove the \r chars. as follows:
sed $'s/\r$//' ./install.sh > ./install.Unix.sh
Note: The $'...' string is an ANSI-C quoted string supported in bash, ksh, and zsh. It is used to ensure that the \r expands to an actual CR character before sed sees the script, because not all sed implementations themselves support \r as an escape sequence.
and then run
./install.Unix.sh --clang-completer
However, the larger question is why you've ended up with \r\n-style files - most likely, other files are affected, too.
Perhaps you're running Git on Windows, where a typical configuration is to convert Unix-style \n-only line breaks to Windows-style \r\n line breaks on checking files out and re-converting to \n-only line breaks on committing.
While this makes sense for development on Windows, it gets in the way of installation scenarios like these.
To make Git check out files with Unix-style file endings on Windows - at least temporarily - use:
git config --global core.autocrlf false
Then run your installation commands involving git clone again.
To restore Git's behavior later, run git config --global core.autocrlf true.
>vim gradlew
:set fileformat=unix
:wq
>./gradlew clean build
It is happening due to windows line endings. To fix the issue follow below steps
For MAC:
brew install dos2unix # Installs dos2unix Mac
find . -type f -exec dos2unix {} \; # recursively removes windows related stuff
For Linux:
sudo apt-get install -y dos2unix # Installs dos2unix Linux
sudo find . -type f -exec dos2unix {} \; # recursively removes windows related stuff
And make sure your git config is set as follows:
git config --global core.autocrlf input
input makes sure to convert CRLF to LF when writing to the object database
Quick command for converting line ending:
dos2unix thescript.sh
Your file has Windows line endings. Change to Unix line endings.
If you are on MAC, and using VS Code, you can switch from CRLF to LF and save the file again. This will replace all CRLF with LF.
Ran into something similar. You can use dos2unix install.sh to convert the line endings. Multiple files via find [pattern] | xargs dos2unix
In my case I had a wrong git configuration. The git documentation states:
If you’re programming on Windows and working with people who are not
(or vice-versa), you’ll probably run into line-ending issues at some
point
I'm using Mac OS and I exactly have this issue in one of my projects. To solve it I turned autocrlf to true which was wrong.
You can check the autocrlf state of your git configuration like this:
git config core.autocrlf
So if this returns true and the problem occurs within a git repository you'll have to change that configuration to
git config --global core.autocrlf input
on a Mac / Unix system. For Windows only projects you can use
git config --global core.autocrlf false
In my case I deleted the git repository and cloned it again and after that everything worked again as expected.
Find out more at https://www.git-scm.com/book/en/v2/Customizing-Git-Git-Configuration
I used to have this problem when I tried to downgrade flutter
this solved my issue
rm -rf flutter
git config --global core.autocrlf false
git clone git#github.com:flutter/flutter.git
flutter channel stable
This link helped me solving the issue.
https://github.com/tiangolo/uwsgi-nginx-flask-docker/issues/127
I edited my .sh file, replacing all CRLF with LF
In my case:
This error occurs when I downloaded and unzip the WINDOWS version into MAC
and then added the windows version path to .bash_profile or .zprofile
so the solution for me was to remove the paths from (.bash_profile and .zprofile)
then download the mac version by opening the terminal and type:
mkdir src
cd src
git clone https://github.com/flutter/flutter.git -b stable
export PATH="$PATH:pwd/flutter/bin"
flutter doctor
I am using VM (in my case simply boot2docker) to run docker containers on Windows host. For convinience, my source files are mapped from host file system, so text files are by default using Windows-style CRLF line endings instead of Unix-style LF endings.
When I try to run some .sh file from docker container, I'll get an error
bash: ./script.sh: /bin/bash^M: bad interpreter: No such file or directory
Is there a way how could I somehow tell bash/sh interpreter to automatically convert \r\n to \n and run a file?
Sure, I could do some piplelining like this cat script.sh | tr -d "\r" | sh or even create an alias for that, but it would not cover situation where one script includes another.
The only acceptable solution I have found so far, is to set Git to checkout source files in UNIX format.
You can commit a .gitattributes file in your repo in the root folder to automatically tell Git to apply LF line endings to all .sh files when checking out, even on Windows. This way you or other developers don't have to remember to configure Git on each machine where you clone your repo.
# Set line endings to LF, even on Windows. Otherwise, execution within Docker fails.
# See https://help.github.com/articles/dealing-with-line-endings/
*.sh text eol=lf
For some scripts the solution ln -s /bin/bash /bin/bash^M will fail.
So create a script /bin/bash^M that will dos2unix the arguments they get and run them with "$#".
Edit:
I tried it with the following bash^M script:
#!/bin/bash
PROG=$1
shift
dos2unix $PROG > /dev/null 2>&1
$PROG "$#"
This will fail when your script includes other scripts with a dot, in that case you should avoid this work-around and go for the Unix format.
You said you can checkout in Unix format, or you can self change the files.
And commit into git, it is an improvement to have unix files stored in git in unix format. When you really want to edit them under windows, change the settings of your editor or dos2unix afterwards.
I imagine you could write a script such as
#!/bin/bash
#bash_run_unixified
#unixify and run in bash
dos2unix < "$1" | /bin/bash
and assuming you saved it in /urs/local/bin/bash_run_unixified with appropriate permissions (sudo chmod o+r,o+x), then you could prefix your scripts with
#!/usr/local/bin/bash_run_unixified
#This script has CRLFs in it
If you don't want such unusual shebang lines, you could
$ ln -s /usr/local/bin/{bash_run_unixified,bash}
and then use
(As Walter A. notes, it might be a good idea to link it to bash^M too)
#!/usr/bin/env bash
as your shebang line (/usr/local/bin/ is normally closer to the beginning of your $PATH than /bin so the env command should select the bash that links to bash_run_unixified).
(Word of caution: I haven't tested any of this)
The best solution I have is to make sure to use editors (like Notepad++ or Sublime Text) able to save directly with the proper eol style.
That way, I don't have to dos2unix any file in my boot2docker session.
Visual Studio code is also good at changing the line ending to linux or windows file endings
I've just started with Git, and I can't figure out how to enable command line for Git. I see many posts suggesting the use of msysgit to enable the Git command line, and I also see many other tools that can work around it. But currently I just get the Git client tools for Windows from GitHub. Then I can use the git command in my windows command console. I don't know if it includes the msysgit in it.
The git client tools will include msysgit. You might need to add the git bin directory into your path for ssh to work correctly with cmd (and powershell)
Also, I would highly recommend "posh git" which is a powershell module that gives you some tab completion and a git prompt in powershell, if you are a windows person, its likely you are more comfortable scripting in PS than in bash, and posh git is great for that.
find instructions here on how to install it.
Note on Windows usage: you should use Git with the provided msysGit
shell (Unix style), it allows to use the complex lines of command
given in this book. If you need, for some reason, to use the native
Windows shell / command line console, you have to use double quotes
instead of simple quotes (for parameters with spaces in them) and you
must quote the parameters ending with the circumflex accent (^) if
they are last on the line, as it is a continuation symbol in Windows.
http://git-scm.com/book/en/Getting-Started-Installing-Git
Are you using the msygit shell?