I'm starting to learn how to script bash and I've run into a problem with the echo command and a variable.
#!/bin/bash
LOGINOUTPUT = "`wget --no-check-certificate --post-data 'login=redacted&password=redacted' https://nessusserver:8834/login -O -`"
echo $LOGINOUTPUT
Running this script returns the following:
--2013-08-15 15:07:32-- https://nessusserver:8834/login
Resolving nessussserver (nessusserver)... 172.23.80.88
Connecting to nessusserver (nessusserver)|172.23.80.88|:8834... connected.
WARNING: cannot verify nessusserver's certificate, issued by ‘/C=FR/ST=none/L=Paris/O=Nessus Users United/OU=Certification Authority for nessusserver.healthds.com/CN=nessusserver.healthds.com/emailAddress=ca#besecmisc1.healthds.com’:
Unable to locally verify the issuer's authority.
WARNING: certificate common name ‘nessusserver.healthds.com’ doesn't match requested host name ‘nessusserver’.
HTTP request sent, awaiting response... 200 OK
Length: 461 [text/xml]
Saving to: ‘STDOUT’
100%[=============================================================================================================================================================>] 461 --.-K/s in 0s
2013-08-15 15:07:33 (90.4 MB/s) - written to stdout [461/461]
./nessus-output.sh: line 2: LOGINOUTPUT: command not found
Why does it think that LOGINOUTPUT is a command? Thanks in advance for any help!
EDIT: Updated script
#!/bin/bash
LOGINOUTPUT=$(wget --no-check-certificate --post-data 'login=redacted&password=redacted' https://nessusserver:8834/login -O -)
echo $LOGINOUTPUT
Still yields the same error, same if I leave $(...) as backticks.
This happens because you have spaces before and after the = in the variable assignment. The correct assignment is:
LOGINOUTPUT="....
with no spaces.
If you add spaces, then the shell interprets LOGINOUTPUT as a command, and tries to pass it two arguments: the "=" and the the quoted string. This of course fails, with the error LOGINOUTPUT: command not found
As a sidenote, it is better to use this syntax $(command) than backticks when doing process substitution.
Related
I'm new to shell scripting and I have a Bash script pulling in data from the Strava API and manipulating/reading it using jq.
When I copy and paste in the first line of code (the one calling in data) into the command line, it works. When I run bash strava.sh the entire program works. But when I execute the program through crontab, I'm getting the following error:
usage: http [--json] [--form] [--pretty {all,colors,format,none}]
[--style STYLE] [--print WHAT] [--headers] [--body] [--verbose]
[--all] [--history-print WHAT] [--stream] [--output FILE]
[--download] [--continue]
[--session SESSION_NAME_OR_PATH | --session-read-only SESSION_NAME_OR_PATH]
[--auth USER[:PASS]] [--auth-type {basic,digest}]
[--proxy PROTOCOL:PROXY_URL] [--follow]
[--max-redirects MAX_REDIRECTS] [--timeout SECONDS]
[--check-status] [--verify VERIFY]
[--ssl {ssl2.3,tls1,tls1.1,tls1.2}] [--cert CERT]
[--cert-key CERT_KEY] [--ignore-stdin] [--help] [--version]
[--traceback] [--default-scheme DEFAULT_SCHEME] [--debug]
[METHOD] URL [REQUEST_ITEM [REQUEST_ITEM ...]]
http: error: unrecognized arguments: https://www.strava.com/oauth/token client_id=xxx client_secret=xxx refresh_token=xxx grant_type=refresh_token
Here's what the line looks like in my script:
access_token=$(http POST "https://www.strava.com/oauth/token" client_id="xxx" client_secret="xxx" refresh_token="xxx" grant_type="refresh_token" | jq -r '.access_token')
When running through crontab, the above error is printed on the first line (i.e. line given above), so I'm fairly certain the problem lies in that line. What am I doing wrong?
The httpie manual (https://httpie.io/docs/cli/best-practices) advises to use of:
--ignore-stdin
For "non-interactive invocations".
Possibly a path issue - are there multiple copies of http installed?
Is there a "%" anywhere in your parameters? Crontab interprets % as a newline, so if you'll have to escape it - "%%".
As an aside - please put your subshell inside "s, lest one day strava returns something like "AC0f4;rm * 0cd-4b203"
access_token="$( http POST ...
The problem
SNMPD is correctly delegating SNMP polling requests to another program but the response from that program is not valid. A manual run of the program with the same arguments is responding correctly.
The detail
I've installed the correct LSI raid drivers on a server and want to configure SNMP. As per the instructions, I've added the following to /etc/snmp/snmpd.conf to redirect SNMP polling requests with a given OID prefix to a program:
pass .1.3.6.1.4.1.3582 /usr/sbin/lsi_mrdsnmpmain
It doesn't work correctly for SNMP polling requests:
snmpget -v1 -c public localhost .1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.1
I get the following response:
Error in packet
Reason: (noSuchName) There is no such variable name in this MIB.
Failed object: SNMPv2-SMI::enterprises.3582.5.1.4.2.1.2.1.32.1
What I've tried
SNMPD passes two arguments, -g and <oid> and expects a three line response <oid>, <data-type> and <data-value>.
If I manually run the following:
/usr/sbin/lsi_mrdsnmpmain -g .1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.0
I correctly get a correct three line response:
.1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.0
integer
30
This means that the pass command is working correctly and the /usr/sbin/lsi_mrdsnmpmain program is working correctly in this example
I tried replacing /usr/sbin/lsi_mrdsnmpmain with a bash script. The bash script delegates the call and logs the supplied arguments and output from the delegated call:
#!/bin/bash
echo "In: '$#" > /var/log/snmp-pass-test
RETURN=$(/usr/sbin/lsi_mrdsnmpmain $#)
echo "$RETURN"
echo "Out: '$RETURN'" >> /var/log/snmp-pass-test
And modified the pass command to redirect to the bash script. If I run the bash script manually /usr/sbin/snmp-pass-test -g .1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.0 I get the correct three line response as I did when I ran /usr/sbin/lsi_mrdsnmpmain manually and I get the following logged:
In: '-g .1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.0
Out: '.1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.0
integer
30'
When I rerun the snmpget test, I get the same Error in packet... error and the bash script's logging shows that the captured delegated call output is empty:
In: '-g .1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.0
Out: ''
If I modify the bash script to only echo an empty line I also get the same Error in packet... message.
I've also tried ensuring that the environment variables that are present when I manually call /usr/sbin/lsi_mrdsnmpmain are the same for the bash script but I get the same empty output.
Finally, my questions
Why would the bash script behave differently in these two scenarios?
Is it likely that the problem that exists with the bash scripts is the same as originally noticed (manually running program has different output to SNMPD run program)?
Updates
eewanco's suggestions
What user is running the program in each scenario?
I added echo "$(whoami)" > /var/log/snmp-pass-test to the bash script and root was added to the logs
Maybe try executing it in cron
Adding the following to root's crontab and the correct three line response was logged:
* * * * * /usr/sbin/lsi_mrdsnmpmain -g .1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.1 >> /var/log/snmp-test-cron 2>&1
Grisha Levit's suggestion
Try logging the stderr
There aren't any errors logged
Checking /var/log/messages
When I run it via SNMPD, I get MegaRAID SNMP AGENT: Error in getting Shared Memory(lsi_mrdsnmpmain) logged. When I run it directly, I don't. I've done a bit of googling and I may need lm_sensors installed; I'll try this.
I installed lm_sensors & compat-libstdc++-33.i686 (the latter because it said it was a pre-requisite from the instructions and I was missing it), uninstalled and reinstalled the LSI drivers and am experiencing the same issue.
SELinux
I accidently stumbled upon a page about extending snmpd with scripts and it says to check the script has the right SELinux context. I ran grep AVC /var/log/audit/audit.log | grep snmp before and after running a snmpget and the following entry is added as a direct result from running snmpget:
type=AVC msg=audit(1485967641.075:271): avc: denied { unix_read unix_write } for pid=5552 comm="lsi_mrdsnmpmain" key=558265 scontext=system_u:system_r:snmpd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=shm
I'm now assuming that SELinux is causing the call to fail; I'll dig further...see answer for solution.
strace (eewanco's suggestion)
Try using strace with and without snmp and see if you can catch a system call failure or some additional hints
For completeness, I wanted to see if strace would have hinted that SELinux was denying. I had to remove the policy packages using semodule -r <policy-package-name> to reintroduce the problem then ran the following:
strace snmpget -v1 -c public localhost .1.3.6.1.4.1.3582.5.1.4.2.1.2.1.32.1 >> strace.log 2>&1
The end of strace.log is as follows and unless I'm missing something, it doesn't seem to provide any hints:
...
sendmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(161), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)= [{"0;\2\1\0\4\20public\240$\2\4I\264-m\2"..., 61}], msg_controllen=32, {cmsg_len=28, cmsg_level=SOL_IP, cmsg_type=, ...}, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 61
select(4, [3], NULL, NULL, {0, 999997}) = 1 (in [3], left {0, 998475})
brk(0xab9000) = 0xab9000
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(161), sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)= [{"0;\2\1\0\4\20public\242$\2\4I\264-m\2"..., 65536}], msg_controllen=0, msg_flags=0}, MSG_DONTWAIT) = 61
write(2, "Error in packet\nReason: (noSuchN"..., 81Error in packet
Reason: (noSuchName) There is no such variable name in this MIB.
) = 81
write(2, "Failed object: ", 15Failed object: ) = 15
write(2, "SNMPv2-SMI::enterprises.3582.5.1"..., 48SNMPv2- SMI::enterprises.3582.5.1.4.2.1.2.1.32.1
) = 48
write(2, "\n", 1
) = 1
brk(0xaa9000) = 0xaa9000
close(3) = 0
exit_group(2) = ?
+++ exited with 2 +++
It was SELinux that was denying snmpd a delegated call to /usr/sbin/lsi_mrdsnmpmain (and probably beyond).
To identify it, I ran grep AVC /var/log/audit/audit.log and for each entry, I ran the following:
echo "<grepped-output>" | audit2allow -a -M <filename>
This creates a SELinux policy package that should allow the delegated call through. The package is then loaded using the following:
semodule -i <filename>.pp
I had to do this 5 times as there were different causes of denial (unix_read unix_write, associate, read write). I'll look to combine the modules into one.
Now when I run snmpget I get the correct delegated output:
SNMPv2-SMI::enterprises.3582.5.1.4.2.1.2.1.32.1 = INTEGER: 34
I have the following command
sudo wget --output-document=/dev/null http://speedtest.pixelwolf.ch which outputs
--2016-03-27 17:15:47-- http://speedtest.pixelwolf.ch/
Resolving speedtest.pixelwolf.ch (speedtest.pixelwolf.ch)... 178.63.18.88, 2a02:418:3102::6
Connecting to speedtest.pixelwolf.ch (speedtest.pixelwolf.ch) | 178.63.18.88|:80... connected.
HTTP Request sent, awaiting response... 200 OK
Length: 85 [text/html]
Saving to: `/dev/null`
100%[======================>]85 --.-K/s in 0s
2016-03-27 17:15:47 (8.79 MB/s) - `dev/null` saved [85/85]
I'd like to be able to parse the (8.79 MB/s) from the last line and store this in a file (or any other way I can get this into a local PHP file easily), I tried to store the full output by changing my command to --output-document=/dev/speedtest however this just saved "Could not reach website" in the file and not the terminal output of the command.
Not quite sure where to start with this, so any help would be awesome.
Not sure if it helps, but my intention is for this stored value (8.79) in this instance to be read by a PHP file and handled there, every 30 seconds which I'll achieve by: while true; do (run speed test and save speed variable to a file cmd); php handleSpeedTest.php; sleep 5; done where handleSpeedTest.php will be able to read that stored value and handle it accordingly.
I changed the URL to one that works. Redirected stderr onto stdout. Used grep --only-matching (-o) and a regex.
sudo wget -O /dev/null http://www.google.com 2>&1 | grep -o '\([0-9.]\+ [KM]B/s\)'
I'm trying to execute a long list of repetitive commands on Terminal.
The commands look like this:
wget 'http://api.tiles.mapbox.com/v3/localstarlight.hl2o31b8/-180,52,9/1280x1280.png' -O '/Volumes/Alaya/XXXXXXXXX/Downloads/MapTiles/Tile (52.-180) 0.png' \
wget 'http://api.tiles.mapbox.com/v3/localstarlight.hl2o31b8/-177,52,9/1280x1280.png' -O '/Volumes/Alaya/XXXXXXXXX/Downloads/MapTiles/Tile (52.-177) 1.png' \
If I copy the entire list into Terminal, it executes them all but seems to do it in such a rush that some only get partially downloadeed, and some missed out entirely. It doesn't seem to take them one by one and wait until each is finished before attempting the next.
I tried putting them entire list into a shell script and running it, but then for some reason it seems to download everything, but only produces one file, and looking at the output, it seems to be trying to save each file under the same filename:
2014-03-29 09:56:31 (4.15 MB/s) - `/Volumes/Alaya/XXXXXXXX/Downloads/MapTiles/Tile (52.180) 120.png' saved [28319/28319]
--2014-03-29 09:56:31-- http://%20%0Dwget/
Resolving \rwget... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ` \rwget'
--2014-03-29 09:56:31-- http://api.tiles.mapbox.com/v3/localstarlight.hl2o31b8/171,52,9/1280x1280.png
Reusing existing connection to api.tiles.mapbox.com:80.
HTTP request sent, awaiting response... 200 OK
Length: 33530 (33K) [image/jpeg]
Saving to: `/Volumes/Alaya/XXXXXXXX/Downloads/MapTiles/Tile (52.180) 120.png'
100%[======================================>] 33,530 --.-K/s in 0.008s
2014-03-29 09:56:31 (3.90 MB/s) - `/Volumes/Alaya/XXXXXXXX/Downloads/MapTiles/Tile (52.180) 120.png' saved [33530/33530]
--2014-03-29 09:56:31-- http://%20%0Dwget/
Resolving \rwget... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ` \rwget'
--2014-03-29 09:56:31-- http://api.tiles.mapbox.com/v3/localstarlight.hl2o31b8/174,52,9/1280x1280.png
Reusing existing connection to api.tiles.mapbox.com:80.
HTTP request sent, awaiting response... 200 OK
Length: 48906 (48K) [image/jpeg]
Saving to: `/Volumes/Alaya/XXXXXXXX/Downloads/MapTiles/Tile (52.180) 120.png'
100%[======================================>] 48,906 --.-K/s in 0.01s
2014-03-29 09:56:31 (4.88 MB/s) - `/Volumes/Alaya/XXXXXXXX/Downloads/MapTiles/Tile (52.180) 120.png' saved [48906/48906]
--2014-03-29 09:56:31-- http://%20%0Dwget/
Resolving \rwget... failed: nodename nor servname provided, or not known.
wget: unable to resolve host address ` \rwget'
--2014-03-29 09:56:31-- http://api.tiles.mapbox.com/v3/localstarlight.hl2o31b8/177,52,9/1280x1280.png
Reusing existing connection to api.tiles.mapbox.com:80.
HTTP request sent, awaiting response... 200 OK
Length: 45644 (45K) [image/jpeg]
Saving to: `/Volumes/Alaya/XXXXXXXX/Downloads/MapTiles/Tile (52.180) 120.png'
100%[======================================>] 45,644 --.-K/s in 0.01s
2014-03-29 09:56:31 (4.36 MB/s) - `/Volumes/Alaya/XXXXXXXX/Downloads/MapTiles/Tile (52.180) 120.png' saved [45644/45644]
So it's saving every file to this name: Tile (52.180) 120.png
Note that it doesn't do this if I put in each command separately...so I don't understand why it's doing that.
Can someone tell me how to execute this list of commands so that it does each one properly?
Thanks!
Your file should look like this:
#!/bin/bash
wget -q 'http://api.tiles.mapbox.com/v3/localstarlight.hl2o31b8/-180,52,9/1280x1280.png' -O 'a.png'
wget -q 'http://api.tiles.mapbox.com/v3/localstarlight.hl2o31b8/-177,52,9/1280x1280.png' -O 'b.png'
BUT... you have a backslash at the end of each wget line, which is a continuation character for long lines and which you don't need. Remove it.
Essentially you are asking wget to get a file and then another file called wget and then another file and then another file. Your script only does a single wget - the first one. All the other wget commands are seen as parameters to the first wget because of the continuation character.
You are doing this:
wget URL file wget URL file wget URL file
Quoting from the log you've posted:
http://%20%0Dwget/
This suggests that your script contains CR+LF line endings. Remove those before executing the script:
sed $'s/\r//' scriptname
or
tr -d '\r' < scriptname
I'm implementing an automated, command line file uploader using curl to a servlet.
The problem is, I've got tens of thousands of files with semicolons (;) in the filenames. I'm well aware of the annoyance of this but it is a legacy app that continues to produce new files each day. Renaming is not really an option for compatibility reasons downstream.
I've tried quoting, escaping, converting to "%3b", fully qualifying the path... the obvious stuff... but nothing seems to work, and it fails to send from the client side. I'm on my mac (bundled curl version 7.21.3) but that shouldn't make a difference?
Any ideas?
macbookpro:~$ curl -F upload=#"my file.txt" http://localhost:8080/data/upload
ok
macbookpro:~$ curl -F upload=#"my;file.txt" http://localhost:8080/data/upload
curl: (26) failed creating formpost data
macbookpro:~$ curl -F upload=#"my\;file.txt" http://localhost:8080/data/upload
curl: (26) failed creating formpost data
macbookpro:~$ curl -F upload=#"my\\;file.txt" http://localhost:8080/data/upload
curl: (26) failed creating formpost data
macbookpro:~$
curl uses ; to separate type (or other directives) from the actual name, so I'd simply use stdin instead:
cat 'my;file.txt' | curl -F upload=#- http://localhost:8080/data/upload
You may possibly add filename= directive as well if desired (but without the semicolon in the name!).
According to CURL man pages, when you have ; or , in your form data(file or raw) you should always enclose it in double quotes.
CURL MAN
So just enclose the filename in double quotes and it should work