Resuming curl download that goes through some bash script - bash

I am trying to download a huge file via curl. As far as I can see it there is some bash script hooked in between to deliver the correct file (in that case a virtual machine that runs IE10):
curl -s https://raw.githubusercontent.com/xdissent/ievms/master/ievms.sh | IEVMS_VERSIONS=10 bash
Due to a wobbly internet connection the download fails constantly so I need a way to resume the download at its current position. I've tried resuming the download like so:
curl -s -C - https://raw.githubusercontent.com/xdissent/ievms/master/ievms.sh | IEVMS_VERSIONS=10 bash
However, all I get is some MD5 check failed error...am I missing something?

The curl command you're running there doesn't download the VM images. It downloads a bash script called ievms.sh and then pipes the script to bash, which executes it.
Looking at the script, it looks like the file it downloads for IE10 is here:
http://virtualization.modern.ie/vhd/IEKitV1_Final/VirtualBox/OSX/IE10_Win8.zip
I think if you download that file (you could use your browser or curl) and put it in ~/.ievms, and then run the command again, it should see that the file has already been downloaded and finish the installation.
If the partially-downloaded file is already there, then you could resume that download with this command:
curl -L "http://virtualization.modern.ie/vhd/IEKitV1_Final/VirtualBox/OSX/IE10_Win8.zip" \
-C - -o ~/.ievms/IE10_Win8.zip
(Then run the original IEVMs curl command to finish installation.)

Related

How to launch WSL as if I've logged in?

I have a WSL Ubuntu distro that I've set up so that when I login 4 services start working, including a web API that I can test via Swagger to verify it is up and working.
I'm at the point where what I want to do now is start WSL via a script - that is, launch my distro, have all of the services start, and do it from Python. The problem is I cannot even figure out the correct syntax to get WSL to start from PowerShell in a manner where my services start.
Side note: "services" != systemctl (or similar) calls, but just executing bash CLI commands from either my .bashrc or .profile at login.
I've put the commands to execute in .profile & .bashrc. I've configured it both for root execution and non-root user execution. I've taken the commands out of those 2 files and put it into a script in the Windows file system that I pass in on the start of wsl. And I've put that shell script in the WSL file system as well. Nothing seems to work, and sometimes the distro starts and then stops after about 30 seconds.
Some of the PS CLI commands I've tried:
Start-Job -ScriptBlock{ wsl -d distro -u root }
Start-Job -ScriptBlock{ wsl -d distro -u root 'bash -i -l -c /root/bin/start.sh'
Start-Job -ScriptBlock{ wsl -d distro -u root 'bash -i -l -c .\start.sh'
wsl -d distro -u root -- bash -i -l -c /root/bin/start.sh
wsl -d distro -u root -- bash -i -l -c .\start.sh
wsl -d distro -u root -- /root/bin/start.sh
Permutations of the above that I've tried: replace root with my default login, and turning all of the Start-Job bash options into a comma-separated list of single-quoted strings (Ex: 'bash', '-i', '-l', ... ). Nothing I launch from the CLI will allow me access to the web API that is supposed to be hosted on my distro.
Any advice on what to try next?
Not necessarily an answer here as much as troubleshooting tips which will hopefully lead to an answer:
First, most of the forms that you are using seem to be correct. The only ones that absolutely shouldn't work are those that attempt to run the script from the Windows filesystem.
Make sure that you have a shebang line starting your script. I'm assuming you do, but other readers may come across this as well. For the moment, try this form:
#!/usr/bin/env -S bash -li
That's going to have the same effect as the bash -li you tried -- It will source both both interactive startup files such as ~/.bashrc as well as login profiles such as ~/.bash_profile (and /etc/profile.d/*, etc.).
Note that preferably, you won't need the -li. Best practice would be to move anything necessary for the services over from the startup scripts to your start.sh script, and avoid parsing the profile and rc. I need to go update some of my answers, since I just realized I've been guilty of giving some potentially bad advice ...
Specifically, though, I'm wondering if your interactive Bash config has something truly, well, "interactive" in it that might be preventing the automatic running of the script itself. Again, best practice would be for ~/.bashrc to only hold configuration that is needed for interactive shell sessions.
Make sure the script is set as executable (chmod +x start.sh). Again, I'm assuming this is the case for you.
With a shebang line and an executable script, use something like:
wsl -d distro -u root -e /root/bin/start.sh
The -e tells WSL to launch the script directly. Since it has a shebang line, it will be parsed by Bash. Most of the other forms you use above actually run Bash twice - Once when launching WSL and another when it finds the shebang line in the script.
Try some basic troubleshooting for your script like:
Add set -x to the top (right under the shebang line) to turn on script debugging.
Add a ps -efH at the end to show the processes that are running when the script completes
If needed, resort to quick-and-dirty echo statements to show where things have progressed in the script.
I'm hopeful that the above will at least show you the problem, but if not, add the debugging info that you gain from this to your question, and we can troubleshoot further.

curl -s https://laravel.build/example-app | bash is asking for a Uri

When if follow the Laravel getting started docs it says to use
curl -s https://laravel.build/example-app | bash
But it then asks me for a URI which the Laravel documentation does not tell me about.
If I try and put in something like:
d:laravel-test
The console comes back saying
Cannot recognize "D:laravel-test" as a System.Uri due to a format error.
I'm quite new to web development so please go easy on me.
You must use the build command from a linux (Bash) command line, not the powershell.
In the Windows command line (powershell), navigate to the folder where your Laravel projects are to be stored.
C:\Users\username>cd PATH TO FOLDER
Open the windows subsystem for linux by typing 'wsl' at the prompt
C:\Users\username\path to folder>wsl (This part isn't clear in the docs)
This will open the Linux/BASH command line where you should paste the Laravel curl build command
username#computer:/filepath$ curl -s https://laravel.build/example-app | bash
This will launch the installation. Note that it will take a long time the first time but will be much faster the next time. The Docker environment will continue to run in this terminal.
Not an expert but this worked for me.
curl -s https://laravel.build/example-app | bash

Installing/running JQ on git bash

I'm trying to implement a git hook that edits some JSON every time I push.
I have JQ installed on my Mac using homebrew "brew install jq", but when the git hook runs my .sh I get the error
jq: command not found
My latest attempts have been to use curl to download the jq library, point to it, and run jq that way:
jq=/usr/local/Cellar/jqz
curl -L -o $jq https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
Unfortunately, this is also returning the same 'command not found' error.
Sidenote: jq=/usr/bin/jq gives me a permission error when I try to write to it
jq=/usr/local/Cellar/jqz
curl -L -o $jq https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
It looks like you are storing the binary with the name jqz. No surprise that it cannot be executed as jq; you would have to invoke it as jqz.
I don't know if /usr/local/Cellar is part of your PATH?
The canonical way would be:
jq='/usr/local/bin/jq'
curl -L -o "$jq" https://github.com/stedolan/jq/releases/latest/download/jq-osx-amd64
you could also store it in the bin directory of your home folder: `jq="$HOME/bin" which should be added automatically to your PATH on most installations (might require a logout & login).

Teamcity with Subversion post commit script on windows

We would like Teamcity to build our solutions on every commit into subversion.
Following the documentation, we are to create a .sh script :-
SERVER=https://buildserver-url
USER=buildserver-user
PASS="<password>"
LOCATOR=$1
# The following is one-line:
(sleep 10; curl --user $USER:$PASS -X POST "$SERVER/app/rest/vcs-root-instances/commitHookNotification?locator=$LOCATOR" -o /dev/null) >/dev/null 2>&1 <&1 &
exit 0
Subversion is running on a windows environment, and so the .sh file will fail.
We are trying to convert this into a .bat file of which we have :-
set SERVER=https://buildserver-url
set USER=buildserver
set PASS=password
LOCATOR=%1%
(timeout 10; curl --user %USER%:%PASS% -X POST "%SERVER%/app/rest/vcs-root-instances/commitHookNotification?locator=%LOCATOR%" -o /dev/null) >/dev/null 2>%1% <%1% &
exit 0
However, this is still failing when trying to execute with
"The system cannot find the path specified"
It seems that perhaps we havnt converted this correctly?
Are the programs you're referencing (such as curl and timeout.exe) in locations that are present in the $PATH/%PATH% variable? How about any other files you're referencing - are you specifying full paths
Side note: Did you install curl and timeout.exe on the Windows server?
Also, /dev/null does not exist on Windows; you need to redirect to NUL. You can't just change the file extension and some of your syntax and expect a bash script to work on Windows.
Were I in your shoes, I'd skip batch altogether and write the script in something modern and sane like Powershell.

Cannot clone an aegir site

I am getting this error while cloning the site on aegir
Unknown option: --profile. See drush help provision-backup for available >options. To suppress this error, add the option --strict=0. [2.39 sec, 21.19 MB]
I am using aegir3 on ubuntu 14.04.4. Can anyone please help me solve this. I have searched on web but there is no solution regarding this issue. This issue also occurs while migrating.
you might find more Aegir users at the new community support forum
http://ask.aegir.support/
I had a similar issue and found a solution there
I found that debugging Aegir issues was a lot easier if you executed the Drush commands directly.
You can find the command by looking through the Log Messages keeping an eye out for entries like
Backend invoke: /usr/bin/php -d magic_quotes_gpc=Off -d magic_quotes_runti... (Expand)
Below entries like the above you will find the command that was executed, expand that by clicking the (Expand) and copy the command (I found sometimes the expand did not work, in cases like this I just use chrome tools to look at the source and copy it)
/usr/bin/php -d magic_quotes_gpc=Off -d magic_quotes_runtime=Off -d magic_quotes_sybase=Off /var/aegir/.composer/vendor/drush/drush/drush.php --php=/usr/bin/php --php-options=' -d magic_quotes_gpc=Off -d magic_quotes_runtime=Off -d magic_quotes_sybase=Off' --backend=2 --yes #centosintranet provision-install-backend --client_email='this.email#is.invalid' 2>&1
All you need to do now is run this as the Aegir user via bash. If you use the apt-get package to install Aegir you should be able to open a terminal and enter
sudo su - aegir -s /bin/bash
You will be prompted for your password, after entering it you will have a bash shell as the Aegir user. Paste in the above drush command and press enter.
Watch it execute, I found it easier to work out what was going on by removing the --backend=2 option. If you still can't see what the issue is try adding -vvv or --debug to the drush command.
Using this method I have solves all issues I have run into thus far when Aegir falls over.

Resources