The filename of my curl download target is unpredictable and globbing with an asterisk isn't possible. I can download the file using the following command, but only after I've determined its' name in advance:
curl -O -vvv -k -u user:password https://myURL/ws/myfile.zip
How can I tailor my curl command to succeed with an unpredictable target name?
There's no easy way to get a directory listing using HTTP. You can use curl to just print the HTML generated by the site. If there's an index with links to the files on that server, simply running
curl -s -u user:password https://myURL/ws/ | grep .zip
will print HTML-formatted links to the zip files available for download on that page.
Intro:
Like the OP, I had a similar issue scripting the download of a binary- for docker-compose- from Github because the version number keeps iterating making the file name unpredictable.
This is how I solved it. Might not be the tidiest solution, but if you have a more elegant way, ping me a comment and I'll update the answer.
Solution:
I merely used an auto-populating variable that takes the output of curl, prints the 1st line- which will be the most recent release- and thengrep for the release number prefaced by a "v". The result is saved to the the path /home/ubuntu as the arbitrary file name "docker-compose-latest"
curl -L "https://github.com/docker/compose/releases/download/$(curl https://github.com/docker/compose/releases | grep -m1 '<a href="/docker/compose/releases/download/' | grep -o 'v[0-9:].[0-9].[0-9]')/docker-compose-$(uname -s)-$(uname -m)" -o /home/ubuntu/docker-compose-latest
And we validate that we received the correct binary (I'm downloading to a Raspberry Pi which has an ARM processor on 64 bit Ubuntu 20.04 LTS:
file /home/ubuntu/docker-compose-latest
Produces the following feedback on the file:
/home/ubuntu/docker-compose-latest: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=QqyJMzYMWOofWehXt3pb/T7U4zg-t8Xqz_11RybNZ/ukJOlZCpzQuZzBcwSK3b/d6ecQ2m2VfqKb_EQRUZA, stripped
To validate this solution works, just execute the above commands remembering to change the path of the file command if not using Ubuntu.
Conclusion:
Again, might not be the most elegant solution, but it's a solution for how one can download a target with curl that has an unpredictable filename.
Related
I'm trying to verify an installer by typing this command.
grep node-vx.y.z.tar.gz SHASUMS256.txt | sha256sum -c -
But I get a response like this from Terminal
sha256sum: 'standard input': no properly formatted checksum lines found
Obviously, I already have SHASUMS256.txt file and on the right path on Terminal(MacOS).
Also prior to this, I've already installed CoreUtils from brew by running this command.
brew install coreutils
So what does 'standard input': no properly formatted checksum lines found mean in this context on MacOS Terminal?
It would help if you posted the contents of the file:
SHASUMS256.txt
If it is properly formatted it should contain a line that looks something like:
09ac98ea433e8cb19d09ead1a0bd439cafb1c3e4cb699af04b4e4da1f6ca3f02 node-vx.y.z.tar.gz
You could get your stderr output for 2 possible reasons.
The line in SHASUMS256.txt is not formatted correctly. It should be formatted as <ChecksumHash> <Filename> without the <>s.
The output from your grep search is blank (no output because grep returned no matches). This is more likely and I would suggest running the grep command by itself first to see what the output is prior to piping it into sha256sum.
In my macOS Monterey Version 12.61 I have to add an asterisk prior to the filename, in this way:
36a2c2c01723151b36987e377ebfc152337cc61699eaa6bf45d9460a4c3e216b *nifi-toolkit-1.17.0-bin.zip
And then run the command:
shasum -a 256 -c input.txt
Finally, I got this:
nifi-toolkit-1.17.0-bin.zip: OK
I have a lot of files with names in one language and I need to rename them all to another one.
Is there any script for this? (preferably on Mac)
Figured it out. We can do it with translate-shell CLI utility.
Install it with brew install translate-shell
Then run next script in your folder:
for i in *.txt
do
sleep 5
mv -i "$i" "$(echo ${i%.txt} | trans -b nl:en).txt";
done
translate-shell makes a call to Google Translate server to do translation
sleep 5 is needed to avoid being blocked by Google's server for too many requests in a second
trans is actual translate command
-b stands for "brief", as we don't need verbose output
nl:en are the source and destination languages
I need to download and run a firefox through a bash script, so I tried running the commands below:
curl -o ~/firefox.tar.bz2 https://download.mozilla.org/?product=firefox-latest-ssl&os=linux64
tar xjf ~/firefox.tar.bz2
~/firefox/firefox
Yet already the first command fails to download the tar file.
Note: The OS is Ubuntu 16, and I don't want to use apt-get.
Quote the address, otherwise the shell interprets the ampersand as a shell order and it ends up trying to download something different to what you expect. Also, add the -L parameter to tell cURL to follow the links:
curl -L -o ~/firefox.tar.bz2 "https://download.mozilla.org/?product=firefox-latest-ssl&os=linux64"
I want to download a program. For example "package.zip"
I want to download it in, for example "~/programs/downloaded"
I want to name it, for example "new.zip"
So I tried:
wget -P ~/programs/downloaded \
-O new.zip https://somewebsite.com/package.zip
But it only downloaded the package in the terminal's current directory and renamed it. The -P command does not work. Any idea how to make it work?
This is a feature of wget that has bitten many people. Unfortunately, it was a design decision taken many years ago and cannot be changed now for fear of breaking existing scripts. The crucial thing to understand here is that -O acts like shell redirection and hence is unaffected by the -P option.
The way to do what you want would be to directly provide the filename:
wget -O ~/programs/downloaded/new.zip <url>
Actually I'm trying to generate .h & .c files by given sysSwYear as MIBNODE and I'm using following command:
mib2c -c mib2c.scalar.conf sysSwYear
but it produces the following error:
You didn't give mib2c a valid OID to start with. IE, I could not find
any information about the mib node "sysSwYear.0". This could be caused
because you supplied an incorrectly node, or by the MIB that you're
trying to generate code from isn't loaded. To make sure your mib is
loaded, run mib2c using this as an example:
env MIBS="+MY-PERSONAL-MIB" mib2c -c mib2c.scalar.conf sysSwYear.0
You might wish to start by reading the MIB loading tutorial at:
http://www.net-snmp.org/tutorial-5/commands/mib-options.html
And making sure you can get snmptranslate to display information about
your MIB node. Once snmptranslate works, then come back and try mib2c
again.
I have already done everything needed like setting environment variable for MIB and defining a private MIB file in /usr/share/snmp/mibs ...but still no success. What to do?
You need to do exactly what it says: get the MIB to load first into the parser. Start with using snmptranslate to make sure you can load and parse the MIB:
# export MIBS="+NAME-OF-YOUR-MIB"
# snmptranslate -IR sysSwYear
If that doesn't work, then your MIB isn't being found or loaded because it has errors (or both). If so, run snmptranslate with the -Dparse option and it'll give you way too much information about what it's doing, but it'll let you know where the problems are if you read it all.
Once snmptranslate works as above, then mib2c should work fine (assuming you leave the MIBS environment variable set).
[note: I used export assuming you're using a sh-based shell; use setenv instead and no = sign if you are using a csh-based shell]
If you are on Linux, the default permissions for /var/lib/net-snmp/mib_indexes are 0700 resulting in interesting errors for normal users.
The fix for this is
linux ~ $ mkdir -p ~/.snmp/persist
linux ~ $ echo "persistentDir ~/.snmp/persist" >> ~/.snmp/snmp.conf
Try this if the commands work for superuser but not normal users. Please note that the accepted answer is right but does not address using those commands as a normal user as denoted by '#'.
I found it more intuitive to issue the command as follows:
mib2c -c mib2c.scalar.conf MY-PERSONAL-MIB::sysSwYear.0
The mib must be in one of the directories listed in the output of the following command:
linux ~ $ net-snmp-config --default-mibdirs