Curl POST call for server API which uploads a file - bash

I want to translate a series of POSTMAN calls into bash in order to create a script. Super easy till now where I want to POST an xlsx file with roles with form-data.I use this script:
curl -i -X POST \
-H 'externalTenantId: 326c1027-bf20-4cd6-ac83-33581c50249b' \
-H "uid: user" \
-H "Content-Type: multipart/form-data" \
-F 'payload={
"importMode": "OVERWRITE",
"tenantId": "326c1027-bf20-4cd6-ac83-33581c50249b",
"file": "roles.xlsx"
}' \
-F 'file=#roles.xlsx' \
"http://server:8080/iamsvc/batchImport/v2/direct/roles"
This is the postman call which works:
POST http://server:8080/iamsvc/batchImport/v2/direct/roles
Headers:
uid: user#domain.com
externalTenantId: 4cd6-ac83-33581c50249b-327522
Payload:
{
"file": [Excel file to be uploaded],
"importMode": "OVERWRITE",
"tenantId": "4cd6-ac83-33581c50249b-327522"
}
This is the error that I get:
HTTP/1.1 100 Continue
HTTP/1.1 400 Bad Request
Server: Apache-Coyote/1.1
Set-Cookie: JSESSIONID=0BA814182C258E1DFE62ACF98409F9CD; Path=/iamsvc/;
Secure; HttpOnly
Content-Length: 0
Date: Mon, 26 Sep 2016 12:59:50 GMT
Connection: close

The answer was in curl --manual and it's working like this:
curl -i -X POST -H "uid: user" -H "externalTenantId: 326c1027-bf20-4cd6-ac83-33581c50249b" -F "file=#/home/user/zscripts/iamapi/roles.xlsx" -F "importMode=OVERWRITE" -F "tenantId=326c1027-bf20-4cd6-ac83-33581c50249b" http://server:8080/iamsvc/batchImport/v2/direct/roles

Related

can't use curl to query neo4j

I am trying to use curl to query neo4j
curl -X POST -H Accept:application/json -H Content-Type:application/json -u neo4j:password -v http://localhost:7474/db/neo4j/tx/commit -d '{"statements":[{"statement":"MATCH (n) RETURN n"}]}'
gives me this response
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:7474...
* Connected to localhost (127.0.0.1) port 7474 (#0)
* Server auth using Basic with user 'neo4j'
> POST /db/neo4j/tx/commit HTTP/1.1
> Host: localhost:7474
> Authorization: Basic bmVvNGo6cGFzc3dvcmQ=
> User-Agent: curl/7.79.1
> Accept:application/json
> Content-Type:application/json
> Content-Length: 47
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Wed, 27 Jul 2022 09:13:35 GMT
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Content-Length: 120
<
{"results":[],"errors":[{"code":"Neo.ClientError.Request.InvalidFormat","message":"Could not parse the incoming JSON"}]}* Connection #0 to host localhost left intact
If anyone could help please
Should have mentioned I'm on windows. Apparently you have to escape those double quotes in the json
This works for me now:
curl -X POST -H Accept:application/json -H Content-Type:application/json -u neo4j:password -v http://localhost:7474/db/neo4j/tx/commit -d "{\"statements\":[{\"statement\":\"MATCH (n) RETURN n\"}]}"

Variable in command substitution is expanded into multiple arguments

I have tried to create a method in a bash script which should be able to perform a curl operation with a variable number of headers, however I seem to get stuck in that the curl command consider the header arguments to be multiple arguments as they contain spaces.
When I run the following line in bash I get a 201 response:
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
If I run the following:
#!/bin/bash
submit_request () {
full_path=/home/mat/globalmetadata.json
header_option=""
header_options=""
for header in "${#:1}"; do # looping over the elements of the $# array ($1, $2...)
header_option=$(printf " -H %s" "$header")
header_options=$(printf '%s%s' "$header_options" "$header_option")
done
echo Headers: $header_options
executable=curl
#response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" -H "Content-Type: application/json" )
response=$($executable -X POST localhost:9200/index-template/globalmetadata --write-out '%{http_code}' --silent --output /dev/null --verbose --data "#${full_path}" $header_option )
echo $response
}
submit_request "\"Content-Type: application/json\""
I get this output:
Headers: -H "Content-Type: application/json"
======= 3
* Trying 127.0.0.1:9200...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9200 (#0)
> POST /index-template/globalmetadata HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Length: 3232
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
} [3232 bytes data]
* We are completely uploaded and fine
* Mark bundle as not supporting multiuse
< HTTP/1.1 406 Not Acceptable
< X-elastic-product: Elasticsearch
< content-type: application/json; charset=UTF-8
< content-length: 97
<
{ [97 bytes data]
* Connection #0 to host localhost left intact
* Could not resolve host: application
* Closing connection 1
406000
What I have noticed is that even though the headers are -H "Content-Type: application/json, curl says Could not resolve host: application. I suspect that it splits the arguments into two due to space between Content-Type: and application/json.
I tried to mix and match quotes and double quotes in all kinds of combination but nothing really worked.
#GordonDavisson is right, you'll have to put your header_options in an array. Just be aware that using the array "${header_options[#]}" when it is empty will result in an empty argument in your curl command, but You can get rid of this problem by storing the whole command in an other array.
submit_request () {
local -a header_options
for arg; do header_options+=(-H "$arg"); done
local -a curl_command=( \
curl \
-X POST \
localhost:9200/index-template/globalmetadata \
--write-out '%{http_code}' \
--silent \
--output /dev/null \
--verbose \
--data '#/home/mat/globalmetadata.json' \
"${header_options[#]}" \
)
local response=$( "${curl_command[#]}" )
printf '%s\n' "$response"
}

How to download ovpn file from openvpn-as via cURL

I am trying to authenticate to my openvpn-as server via curl command. My end goal is to be able to download the ovpn file as I would via the browser but via the cli. I was able to get the cURL command from the browser but I get a 403.
My curl command looks like this:
curl -iL 'https://192.168.35.229:943/?src=connect' \
-H 'Connection: keep-alive' \
-H 'Accept: text/plain, */*; q=0.01' \
-H 'X-OpenVPN: 1' \
-H 'X-Requested-With: XMLHttpRequest' \
-H 'X-CWS-Proto-Ver: 2' \
-H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' \
-H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36 OPR/70.0.3728.178' \
-H 'Origin: https://192.168.35.229:943' \
-H 'Sec-Fetch-Site: same-origin' \
-H 'Sec-Fetch-Mode: cors' \
-H 'Sec-Fetch-Dest: empty' \
-H 'Accept-Language: en-US,en;q=0.9' \
--data-raw 'username=dummy&password=dummy' \
--compressed \
--insecure
The response looks like this:
HTTP/1.1 302 Found
Transfer-Encoding: chunked
Set-Cookie: openvpn_sess_c6bec1b42f39a6d1034a54580d5422dc=36bce77626a6e403eb6ae9f02576d460; Expires=Wed, 16 Sep 2020 05:15:12 GMT; Path=/; Secure; HttpOnly
Server: OpenVPN-AS
Location: https://192.168.35.229:943/__session_start__/
Date: Wed, 16 Sep 2020 04:45:12 GMT
X-Frame-Options: SAMEORIGIN
Content-Type: text/html; charset=utf-8
HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Date: Wed, 16 Sep 2020 04:45:12 GMT
X-Frame-Options: SAMEORIGIN
Content-Type: text/html; charset=UTF-8
Server: OpenVPN-AS
<html><head><title>Forbidden</title></head><body><h1>Forbidden</h1>Request was forbidden.</body></html>#
I wasn't able to find any examples or instructions about this in the documentation. Is there a way to achieve this via CLI - via bash script ?
I was able to find a solution based on this article . The curl command would look like:
curl -u dummy:dummy -k https://192.168.35.229:943/rest/GetUserlogin

Can't insert variable with two headers in curl

I have following sh script
#!/usr/bin/env bash
headers='-H "custom1: ololo1" -H "custom2: ololo2"'
value_for_header="value"
curl -X "PUT" -H "Content-Type: application/json" -H "custom_ololo: $value_for_header" $headers http://localhost:8000/ -d '{"a": true}' -vv
Log when execute it:
* Rebuilt URL to: ololo1"/
* Hostname was NOT found in DNS cache
* Could not resolve host: ololo1"
* Closing connection 0
curl: (6) Could not resolve host: ololo1"
* Rebuilt URL to: ololo2"/
* Hostname was NOT found in DNS cache
* Could not resolve host: ololo2"
* Closing connection 1
curl: (6) Could not resolve host: ololo2"
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8000 (#2)
> PUT / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8000
> Accept: */*
> Content-Type: application/json
> custom_ololo: value
> Content-Length: 40
>
* upload completely sent off: 40 out of 40 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 400
< Date: Thu, 21 Jul 2016 12:32:13 GMT
< Server: WSGIServer/0.1 Python/2.7.6
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
<
* Closing connection 2
As we can see -H "custom_ololo: $value_for_header" works well > custom_ololo: value
But string $headers is not inserted correctly. I've tried put "$headers" and ${headers} but no result
So, my question is: How is correctly insert strings with several headers into sh script with curl.
You need to use an array, at which point you can put all the headers in the array and simplify your call.
#!/usr/bin/env bash
value_for_header="value"
headers=(
-H "custom1: ololo1"
-H "custom2: ololo2"
-H "Content-Type: application/json"
-H "custom_ololo: $value_for_header"
)
curl -X "PUT" "${headers[#]}" http://localhost:8000/ -d '{"a": true}' -vv
You need to put $headers into ""
curl -X "PUT" -H "Content-Type: application/json" -H "custom_ololo: $value_for_header" "$headers" http://localhost:8000/ -d '{"a": true}' -vv

How to programmatically set username and password on Geoserver at intall time

I am writing a bash script to install Geoserver by following the steps outlined here Install Instructions . I am trying to use CURL to post a custom data store config file. But I am having an authentication error.
* upload completely sent off: 42out of 42 bytes
< HTTP/1.1 401 Unauthorized
< Server: Apache-Coyote/1.1
< Set-Cookie: SPRING_SECURITY_REMEMBER_ME_COOKIE=""; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Path=/geoserver
* Authentication problem. Ignoring this.
< WWW-Authenticate: Basic realm="GeoServer Realm"
< Content-Type: text/html;charset=utf-8
< Content-Length: 1299
< Date: Thu, 31 Jan 2013 05:16:17 GMT
I belive it is becuase I haven't set the username and password for Geoserver. The only way I can seem to find is to do it via the web admin interface. I would like to set it via my bash script.
Is there a way to achieve this?
Bash script section here
echo 'Downdloading geoserver'
wget http://downloads.sourceforge.net/project/geoserver/GeoServer/2.2.4/geoserver-2.2.4-war.zip
unzip geoserver-2.2.4-war.zip
sudo cp geoserver.war /var/lib/tomcat7/webapps/
sleep 120
echo 'setting up geoserver'
curl -u $U_NAME:$PASSWORD -v -XPOST -H 'Content-type: text/xml' \
-d '<workspace><name>catami</name></workspace>' \
http://localhost:8080/geoserver/rest/workspaces ;
curl -u $U_NAME:$PASSWORD -XPOST -T datastore-config.xml -H 'Content-type: text/xml' \
http://localhost:8080/geoserver/rest/workspaces/catami/datastores ;
curl -u $U_NAME:$PASSWORD -XPOST -H 'Content-type: text/xml' \
-d '<featureType><name>Force_image</name></featureType>' \
http://localhost:8080/geoserver/rest/workspaces/catami/datastores/CatamiImagePoints/featuretypes ;
curl -u $U_NAME:$PASSWORD -XPOST -H 'Content-type: application/vnd.ogc.sld+xml' \
-d #catami-colour-by-depth.sld http://localhost:8080/geoserver/rest/styles ;
curl -u $U_NAME:$PASSWORD -XPUT -H 'Content-type: text/xml' \
-d '<layer><defaultStyle><name>catami-colour-by-depth</name></defaultStyle><enabled>true</enabled></layer>' \
http://localhost:8080/geoserver/rest/layers/catami:Force_image
Apparently you can edit the username and password inside the GeoServer Data Directory. You can read more about it here.
And after you found what file to change what it, then you just need to sed that data to your liking.

Resources