How to run jq from gitbash in windows? - bash

I have gitbash in Windows. I am trying to run jq but its giving me error.
$ ./jq-win64.exe
jq
parse error: Invalid numeric literal at line 2, column 0
Intention: I want to use jq to parse json.

Easiest solution and always latest version:
run this curl in your gitbash:
curl -L -o /usr/bin/jq.exe https://github.com/stedolan/jq/releases/latest/download/jq-win64.exe
or manually save the jq-win64.exe in link above as jq.exe to your /usr/bin (which is in your git bash installation folder)
(if you are behind a proxy add the -x proxyhost:port)

Using jq-win64.exe from github.com/stedolan/jq/releases, I get
vonc#voncav MINGW64 /d/prgs/dl
$ ./jq-win64.exe --version
jq-1.6
vonc#voncav MINGW64 /d/prgs/dl
$ echo '{"foo": 0}' | ./jq-win64.exe .
{
"foo": 0
}
So it does work, but it then depends on the json document you are parsing with it.
If that json document is not well-formed, that would generate the error you see.
In your bash session, you can define (or add to your ~/.bashrc) an alias:
alias jq=/path/to/jq-win64.exe
That way, you don't need to use ./jq, but directly jq.
$ echo '{"foo": 0}' | jq
In my case:
vonc#voncav:/$ alias jq=/mnt/d/dwnl/jq-win64.exe
vonc#voncav:/$ echo '{"foo": 0}' | jq
{
"foo": 0
}

I hate answers that say you need another to use another tool to download, but using https://chocolatey.org/ made this incredibly simple for me.
From an elevated command-prompt, run:
choco install jq
jq magically works from bash going forward.

Instead of using chocolatey (with elevated rights) you could also use scoop.sh:
scoop install jq

Below steps worked for me in git bash on windows 10
curl -L -o jq-win64.exe https://github.com/stedolan/jq/releases/latest/download/jq-win64.exe
export PATH=$PATH:"/C/Users/devops/Downloads/jq-win64.exe
jq --version
jq-1.6

For windows Powershell && cmd
Download executable -> jq 1+ executables for 64-bit
Create folder in Program Files -> JQ
Copy .exe into program folder
Set environment variable to that folder
Restart terminal
Set alias
Set-Alias -Name jq -Value .\jq-win64.exe
Run command
echo '{"foo": 0}' | jq
Output
{
"foo": 0
}
enjoy!

I just downloaded the binary to %HOMEPATH%/bin/jq-win64 and it worked right away via jq

In my case, I have fixed it in Windows10 while practicing a scenario of "Techworld with Nana" course, as per the below steps:
1) open "cmd" prompt (Run as administrator)
2) curl -L -o /usr/bin/jq-win64.exe
Download it here
3) You can add C:\Program Files\Git\usr\bin path to your environment variables. If you follow this step, ignore the below steps.
[OR]
3) Go to the directory where jq is downloaded: C:\Program Files\Git\usr\bin and copy the file to C:\ drive
4) Add the path C:\ to "path" environment variable. This PC-->Properties-->Advanced system settings --> Environment Variables -->Edit-->New. Then add the path (where you copied jq.exe, i.e., C:) and Save it.
5) Now open a new "cmd" prompt, jq --version

Related

Installing/running JQ on git bash

I'm trying to implement a git hook that edits some JSON every time I push.
I have JQ installed on my Mac using homebrew "brew install jq", but when the git hook runs my .sh I get the error
jq: command not found
My latest attempts have been to use curl to download the jq library, point to it, and run jq that way:
jq=/usr/local/Cellar/jqz
curl -L -o $jq https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
Unfortunately, this is also returning the same 'command not found' error.
Sidenote: jq=/usr/bin/jq gives me a permission error when I try to write to it
jq=/usr/local/Cellar/jqz
curl -L -o $jq https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
It looks like you are storing the binary with the name jqz. No surprise that it cannot be executed as jq; you would have to invoke it as jqz.
I don't know if /usr/local/Cellar is part of your PATH?
The canonical way would be:
jq='/usr/local/bin/jq'
curl -L -o "$jq" https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
you could also store it in the bin directory of your home folder: `jq="$HOME/bin" which should be added automatically to your PATH on most installations (might require a logout & login).

Problems in running a script.sh in Ubuntu app Windows10

I'm trying to learn how to code. I have to say that I'm using the Ubuntu app on the Windows system, so I don't know if my problems are related to this system.
I established these variables in the terminal
FOLDER="/mnt/c/Users/franc/Desktop/nuova"
species=mm10
fragmentsize=200
window=200
gap=200
output="/mnt/c/Users/franc/Desktop/nuova/sicer"
and then I wrote this loop
#!/bin/bash
for fq in $FOLDER/*.bam
do
bedtools bamtobed -i "$fq" > "${fq%.bam}.bed"
sicer -t ${fq%.bam}.bed \
-s $species \
-f $fragmentsize \
-w $window \
-g $gap \
-o $output
echo "DONE"
done
Basically I want the files in the FOLDER to be transformed in "${fq%.bam}.bed" and then I want to run sicer tool on these new files.
If I copy and paste these commands, on the terminal, everything goes fine but if I save the loop as script.sh and I try to run the script I obtain different errors.
Of course, I made the script executable with chmod +x and I also changed the syntax of the script using awk '{ sub("\r$", ""); print }' myscript.sh > myscript1.sh since I edited it in Windows(otherwise ubuntu fails to open it).
But when I launch the script containing the loop, it says that it is not able to open the files in the FOLDER (Failed to open BAM file /*.bam or BAD permission denied). I tried both to open it just giving the command ./myscript1.sh or also using sudo ./myscript1.sh.
What I'm missing? I have in some way link the variable I establish in the terminal to a new variable in the script saved?
thanks
Francesca
You need to export the variables so that they'll be inherited by the shell process running the script.
export FOLDER="/mnt/c/Users/franc/Desktop/nuova"
export species=mm10
export fragmentsize=200
export window=200
export gap=200
export output="/mnt/c/Users/franc/Desktop/nuova/sicer"

Linux: Curl installed but -bash: :curl: command not found

Running Debian Stretch on an r710. Using the non-free/contrib build for driver support.
When I try to use packages that I've installed (curl, zpool, etc), I have to include the path to the package... Which is a pain when I don't always know where packages install to.
Two questions:
How do I remedy the path issue in the short term?
How do I amend Debian so that when packages are installed, their paths update/install automatically?
Just install it by:
apt install curl
or sudo apt install curl
Find where the command is stored by
which <command>
Either you can try run curl from the output above for example /usr/bin/curl then try execute this:
/usr/bin/curl
For a temporary fix until you solve the real problem you can do:
cd /usr/local/bin; ln -s $(which curl) curl
Or you can just set an alias:
echo "alias curl='$(which curl)'" >> ~/.bashrc; . ~/.bashrc
Troubleshoot your problem:
Check so PATH folder has the correct paths exported:
printf "%s\n" $PATH
Modify current PATH
Use the export command to add new paths and see if that works you can then update your ~/.bashrc or ~/.bash_profile, but first you can try in shell without adding it permanent to $PATH
export PATH=$PATH:/missed/bin/folder
To format your PATH variable for easy viewing in future you can add below function to your .bashrc
function path(){
old=$IFS
IFS=:
printf "%s\n" $PATH
IFS=$old
}

skip installing confirm('yes' or 'no') in Dockerfile [duplicate]

How do I install the anaconda / miniconda without prompts on Linux command line?
Is there a way to pass -y kind of option to agree to the T&Cs, suggested installation location etc. by default?
can be achieved by bash miniconda.sh -b (thanks #darthbith)
The command line usage for this can only be seen with -h flag but not --help, so I missed it.
To install the anaconda to another place, use the -p option:
bash anaconda.sh -b -p /some/path
AFAIK pyenv let you install anaconda/miniconda
(after successful instalation)
pyenv install --list
pyenv install miniconda3-4.3.30
For a quick installation of miniconda silently I use a wrapper
script script that can be executed from the terminal without
even downloading the script. It takes the installation destination path
as an argument (in this case ~/miniconda) and does some validation too.
curl -s https://gist.githubusercontent.com/mherkazandjian/cce01cf3e15c0b41c1c4321245a99096/raw/03c86dae9a212446cf5b095643854f029b39c921/miniconda_installer.sh | bash -s -- ~/miniconda
Silent installation can be done like this, but it doesn't update the PATH variable so you can't run it after the installation with a short command like conda:
cd /tmp/
curl -LO https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -u
Here -b means batch/silent mode, and -u means update the existing installation of Miniconda at that path, rather than failing.
You need to run additional commands to initialize PATH and other shell init scripts, e.g. for Bash:
source ~/miniconda3/bin/activate
conda init bash

Azure CLI in Git Bash

I am trying to use a bash (sh) script on windows to run a test deployment. I am running the script from the gitbash console so that I have a copy of bash, but doing so means that the azure clie is not available (i.e. azure command is not found). Does anyone know how I can get the Azure cli working in GitBash (I am assuming I just install it somewhere else) or should I change to a different way of using bash
Sometimes commands in windows git bash need .cmd appended. Also, another way of installing the Azure-Cli is through Chocolatey https://chocolatey.org/
Try this command after Azure-Cli is installed:
az.cmd --version
Echoing mscrivo you can run the line below in CMD not PowerShell (elevated/admin)
echo "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" $1 $2 $3 $4 $5 $6 $7 $8 $9 ${10} ${11} ${12} ${13} ${14} ${15} > "C:\Program Files\Git\mingw64\bin\az"
Now you should be able to run in Git bash:
az --version
artberri noted the best solution in a comment:
Add the following to your %USERPROFILE%\.bashrc or %USERPROFILE%\.profile
alias az='az.cmd'
However, if you want to be able to use az in bash scripts, you'll need something a little more drastic, run the following from cmd prompt:
echo "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" $1 $2 $3 $4 $5 $6 $7 $8 $9 ${10} ${11} ${12} ${13} ${14} ${15} > "%SYSTEMROOT%\az"
That will essentially create a passthrough az command in your windows folder that can be run from anywhere and passes parameters through to az.cmd.
In case using Git bash, navigate to the following directory:
C:\Program Files\Git\etc\profile.d
Edit aliases.sh, then add a new alias for az as below:
alias az='az.cmd'
You have to install the CLI to your computer.
There are multiple ways to do that.
I'm a friend of NodeJS so i use npm for the installation:
npm install -g azure-cli
More details here: https://www.npmjs.com/package/azure-cli
But you can do it also in other ways. A very nice way is to use docker. There are containers from Microsoft with a preinstalled version of Azure CLI.
docker run -it --name azure microsoft/azure-cli
On Windows 10 with the ubuntu bash you can use:
echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ wheezy main" | \
sudo tee /etc/apt/sources.list.d/azure-cli.list
sudo apt-key adv --keyserver packages.microsoft.com --recv-keys 417A0893
sudo apt-get install apt-transport-https
sudo apt-get update && sudo apt-get install azure-cli
Or as a python enthusiast run
pip install --user azure-cli
Most important is that the "az"/"az.bat" or "azure" bin is available via your path variable.
In other words Azure CLI for Windows is not compatible with Git Bash for Windows
https://github.com/Azure/azure-cli/issues/3445
In your ~/bin directory (in Windows it means c:\Users\<username>\bin) I've just created a file named az with:
"C:/Program Files (x86)/Microsoft SDKs/Azure/CLI2/python.exe" -IBm azure.cli "$#"
Then make the file executable
chmod a+x az
The content is "borrowed" from az.cmd.
When trying an alias approach mentioned before I had a problem with long commands, with a lot of parameters throwing an error
"'C:\Program' is not recognized as an internal or external command,operable program or batch file."
EDIT:
I've ended using Ubuntu through WLS. For all tools like az, terraform, kubectl, istioctl. The az tool runs good in interactive mode as well.
So I arrived here looking for a way to run the same az commands in a bash shell script on Azure DevOps Linux and Windows (git bash) build agents, so I could share the same code across both types of agent. This also works for a git bash shell on Windows 10. My longer answer is here: Azure DevOps Build Pipeline can't get secrets from Key Vault when secured with vnet and firewall
The gist of it is:
if [[ $(uname -s) == "Linux" ]]; then
azcmd="az"
else
# If we're in a bash shell on Windows, az commands don't work, but we can call the az.cmd batch file directly from git Bash if we can find it...
azcmd=$(where az.cmd)
fi
# Remember to specify CIDR /32 for removal
"$azcmd" keyvault network-rule remove -n <MyKeyVault> --ip-address 50.1.1.1/32
Basically just substitute "$azcmd" wherever you'd normally use az once the bootstrap code has executed.
My previous approach for this was just to add the Azure CLI Scripts folder to the $PATH inside my ~/.bashrc file. But, after updating Azure CLI to 2.2.0 via the MSI, this approach started to fail with this error:
C:\Program: can't open file 'Files': [Errno 2] No such file or directory
So, to fix this, I have my $PATH in ~/.bashrc include ~/bin and then I created a file with the following content as ~/bin/az (don't forget to chmod 0755 the new file):
#!/usr/bin/env bash
AZURE_CLI_PATH="/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2"
export PATH="${PATH}:${AZURE_CLI_PATH}:${AZURE_CLI_PATH}/Scripts"
export PYTHONPATH="${AZURE_CLI_PATH}/src"
export PYTHON_CMD="$(which python)"
winpty "${PYTHON_CMD}" -m 'azure.cli' "${#}"
After closing out my GIT Bash window and re-opening it, I can now run az again:
$ az version
This command is in preview. It may be changed/removed in a future release.
{
"azure-cli": "2.2.0",
"azure-cli-command-modules-nspkg": "2.0.3",
"azure-cli-core": "2.2.0",
"azure-cli-nspkg": "3.0.4",
"azure-cli-telemetry": "1.0.4",
"extensions": {}
}
Don't use the MSI installer at all. Since the Azure CLI is implemented in Python, use the Python installation method as #blndev wrote. This way instead of az.cmd you get az.bat and az shell script, and the installation path will not contain spaces.
pip install --user azure-cli
More detailed info on this method can be found at https://blogs.msdn.microsoft.com/brijrajsingh/2017/03/02/installing-azure-cli-2-0-on-windows/
The symlink worked for me most of the times, but some commands are still failing, e.g.
az dls fs access set-entry ...
'C:\Program' is not recognized as an internal or external command, operable program or batch file
I tried #mscrivo's solution. When using the az command in a shell script however you still have problems due to the spaces in the path.
Therefor I created a azproxy.cmd in %SYSTEMROOT% containing
#echo off
"C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" %*
And then create the mklink to that file
mklink "%SYSTEMROOT%\az" "SYSTEMROOT%\azproxy.cmd"
PS the expanded value of %SYSTEMROOT% should not contain spaces of course

Resources