Variable in command substitution is expanded into multiple arguments - bash

I have tried to create a method in a bash script which should be able to perform a curl operation with a variable number of headers, however I seem to get stuck in that the curl command consider the header arguments to be multiple arguments as they contain spaces.
When I run the following line in bash I get a 201 response:
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
If I run the following:
#!/bin/bash
submit_request () {
full_path=/home/mat/globalmetadata.json
header_option=""
header_options=""
for header in "${#:1}"; do # looping over the elements of the $# array ($1, $2...)
header_option=$(printf " -H %s" "$header")
header_options=$(printf '%s%s' "$header_options" "$header_option")
done
echo Headers: $header_options
executable=curl
#response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" $header_option )
echo $response
}
submit_request "\"Content-Type: application/json\""
I get this output:
Headers: -H "Content-Type: application/json"
======= 3
* Trying 127.0.0.1:9200...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9200 (#0)
> POST /index-template/globalmetadata HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 3232
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
} [3232 bytes data]
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 406 Not Acceptable
< X-elastic-product: Elasticsearch
< content-type: application/json; charset=UTF-8
< content-length: 97
<
{ [97 bytes data]
* Connection #0 to host localhost left intact
* Could not resolve host: application
* Closing connection 1
406000
What I have noticed is that even though the headers are -H "Content-Type: application/json, curl says Could not resolve host: application. I suspect that it splits the arguments into two due to space between Content-Type: and application/json.
I tried to mix and match quotes and double quotes in all kinds of combination but nothing really worked.

#GordonDavisson is right, you'll have to put your header_options in an array. Just be aware that using the array "${header_options[#]}" when it is empty will result in an empty argument in your curl command, but You can get rid of this problem by storing the whole command in an other array.
submit_request () {
local -a header_options
for arg; do header_options+=(-H "$arg"); done
local -a curl_command=( \
curl \
-X POST \
localhost:9200/index-template/globalmetadata \
--write-out '%{http_code}' \
--silent \
--output /dev/null \
--verbose \
--data '#/home/mat/globalmetadata.json' \
"${header_options[#]}" \
)
local response=$( "${curl_command[#]}" )
printf '%s\n' "$response"
}

Related

Using bombardier for server benchmarking - but can't add body parameters

So, I'm using bombardier on MacOS Sierra
for server benchmarking from command line.
In this example i'm using 1 connection and 1 request, and two headers:
"Authorization" and "Content-Type" and body: "{isTemplate:1}"
But server does not receiving body.
./bombardier -c 1 -n 1 -m PATCH -H "Authorization: Bearer MYBEARERGOESHERE" -H "Content-Type: application/x-www-form-urlencoded" -b "{isTemplate:1}" http://localhost:8082/presentation/6525/update
I tried:
-b "{isTemplate:1}"
-b "isTemplate:1"
Any thoughts?
Just found the solution, - you need to write body variables like this:
-b "isTemplate=1"
So final request:
./bombardier -c 1 -n 1 -m PATCH -H "Authorization: Bearer MYBEARERGOESHERE" -H "Content-Type: application/x-www-form-urlencoded" -b "isTemplate=1" http://localhost:8082/presentation/6525/update

Curl call from Terminal and Shellscript is diffrent?

I am calling the same curl command once from terminal and once from Shellscript - but the outcome is different:
#!/usr/bin/env bash
url="dev.test.ch"
http="http://"
test="192.168.178.107"
curl -H "'Host: $url'" "$http$test"
echo "'Host: $url'" "$http$test"
and
curl -H 'Host: dev.test.ch' http://192.168.178.107
Once I get the NGINX Startpage (from Shellscript) and once the right response (HTML Page from my App)
Any ideas?
Stupid me -
curl -H 'Host: '$newline "$http$test"
the braces were the problem.
You can use curl with -v option to see what exactly HTTP request was sent to server. Problem that in your script you curl adds 'Host header.
With this line curl -H "'Host: $url'" "$http$test" it sends
> GET http://192.168.178.107/ HTTP/1.1
> Host: 192.168.178.107
> User-Agent: curl/7.49.1
> Accept: */*
> 'Host: dev.test.ch'
If you remove single qoutes curl -v -H "Host: $url" "$http$test" it will send proper request:
> GET http://192.168.178.107/ HTTP/1.1
> Host: dev.test.ch
> User-Agent: curl/7.49.1
> Accept: */*

Curl POST call for server API which uploads a file

I want to translate a series of POSTMAN calls into bash in order to create a script. Super easy till now where I want to POST an xlsx file with roles with form-data.I use this script:
curl -i -X POST \
-H 'externalTenantId: 326c1027-bf20-4cd6-ac83-33581c50249b' \
-H "uid: user" \
-H "Content-Type: multipart/form-data" \
-F 'payload={
"importMode": "OVERWRITE",
"tenantId": "326c1027-bf20-4cd6-ac83-33581c50249b",
"file": "roles.xlsx"
}' \
-F 'file=#roles.xlsx' \
"http://server:8080/iamsvc/batchImport/v2/direct/roles"
This is the postman call which works:
POST http://server:8080/iamsvc/batchImport/v2/direct/roles
Headers:
uid: user#domain.com
externalTenantId: 4cd6-ac83-33581c50249b-327522
Payload:
{
"file": [Excel file to be uploaded],
"importMode": "OVERWRITE",
"tenantId": "4cd6-ac83-33581c50249b-327522"
}
This is the error that I get:
HTTP/1.1 100 Continue
HTTP/1.1 400 Bad Request
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=0BA814182C258E1DFE62ACF98409F9CD; Path=/iamsvc/;
Secure; HttpOnly
Content-Length: 0
Date: Mon, 26 Sep 2016 12:59:50 GMT
Connection: close
The answer was in curl --manual and it's working like this:
curl -i -X POST -H "uid: user" -H "externalTenantId: 326c1027-bf20-4cd6-ac83-33581c50249b" -F "file=#/home/user/zscripts/iamapi/roles.xlsx" -F "importMode=OVERWRITE" -F "tenantId=326c1027-bf20-4cd6-ac83-33581c50249b" http://server:8080/iamsvc/batchImport/v2/direct/roles

Can't insert variable with two headers in curl

I have following sh script
#!/usr/bin/env bash
headers='-H "custom1: ololo1" -H "custom2: ololo2"'
value_for_header="value"
curl -X "PUT" -H "Content-Type: application/json" -H "custom_ololo: $value_for_header" $headers http://localhost:8000/ -d '{"a": true}' -vv
Log when execute it:
* Rebuilt URL to: ololo1"/
* Hostname was NOT found in DNS cache
* Could not resolve host: ololo1"
* Closing connection 0
curl: (6) Could not resolve host: ololo1"
* Rebuilt URL to: ololo2"/
* Hostname was NOT found in DNS cache
* Could not resolve host: ololo2"
* Closing connection 1
curl: (6) Could not resolve host: ololo2"
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8000 (#2)
> PUT / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8000
> Accept: */*
> Content-Type: application/json
> custom_ololo: value
> Content-Length: 40
>
* upload completely sent off: 40 out of 40 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 400
< Date: Thu, 21 Jul 2016 12:32:13 GMT
< Server: WSGIServer/0.1 Python/2.7.6
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
<
* Closing connection 2
As we can see -H "custom_ololo: $value_for_header" works well > custom_ololo: value
But string $headers is not inserted correctly. I've tried put "$headers" and ${headers} but no result
So, my question is: How is correctly insert strings with several headers into sh script with curl.
You need to use an array, at which point you can put all the headers in the array and simplify your call.
#!/usr/bin/env bash
value_for_header="value"
headers=(
-H "custom1: ololo1"
-H "custom2: ololo2"
-H "Content-Type: application/json"
-H "custom_ololo: $value_for_header"
)
curl -X "PUT" "${headers[#]}" http://localhost:8000/ -d '{"a": true}' -vv
You need to put $headers into ""
curl -X "PUT" -H "Content-Type: application/json" -H "custom_ololo: $value_for_header" "$headers" http://localhost:8000/ -d '{"a": true}' -vv

How to programmatically set username and password on Geoserver at intall time

I am writing a bash script to install Geoserver by following the steps outlined here Install Instructions . I am trying to use CURL to post a custom data store config file. But I am having an authentication error.
* upload completely sent off: 42out of 42 bytes
< HTTP/1.1 401 Unauthorized
< Server: Apache-Coyote/1.1
< Set-Cookie: SPRING_SECURITY_REMEMBER_ME_COOKIE=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/geoserver
* Authentication problem. Ignoring this.
< WWW-Authenticate: Basic realm="GeoServer Realm"
< Content-Type: text/html;charset=utf-8
< Content-Length: 1299
< Date: Thu, 31 Jan 2013 05:16:17 GMT
I belive it is becuase I haven't set the username and password for Geoserver. The only way I can seem to find is to do it via the web admin interface. I would like to set it via my bash script.
Is there a way to achieve this?
Bash script section here
echo 'Downdloading geoserver'
wget http://downloads.sourceforge.net/project/geoserver/GeoServer/2.2.4/geoserver-2.2.4-war.zip
unzip geoserver-2.2.4-war.zip
sudo cp geoserver.war /var/lib/tomcat7/webapps/
sleep 120
echo 'setting up geoserver'
curl -u $U_NAME:$PASSWORD -v -XPOST -H 'Content-type: text/xml' \
-d '<workspace><name>catami</name></workspace>' \
http://localhost:8080/geoserver/rest/workspaces ;
curl -u $U_NAME:$PASSWORD -XPOST -T datastore-config.xml -H 'Content-type: text/xml' \
http://localhost:8080/geoserver/rest/workspaces/catami/datastores ;
curl -u $U_NAME:$PASSWORD -XPOST -H 'Content-type: text/xml' \
-d '<featureType><name>Force_image</name></featureType>' \
http://localhost:8080/geoserver/rest/workspaces/catami/datastores/CatamiImagePoints/featuretypes ;
curl -u $U_NAME:$PASSWORD -XPOST -H 'Content-type: application/vnd.ogc.sld+xml' \
-d #catami-colour-by-depth.sld http://localhost:8080/geoserver/rest/styles ;
curl -u $U_NAME:$PASSWORD -XPUT -H 'Content-type: text/xml' \
-d '<layer><defaultStyle><name>catami-colour-by-depth</name></defaultStyle><enabled>true</enabled></layer>' \
http://localhost:8080/geoserver/rest/layers/catami:Force_image
Apparently you can edit the username and password inside the GeoServer Data Directory. You can read more about it here.
And after you found what file to change what it, then you just need to sed that data to your liking.

Resources