Curl using Environment variables in GitLab-ci.yml - continuous-integration

I'm trying to send a slack notification using the .gitlab-ci.yml and I need to pass the Commit's message in the message like this:
"The version ${CI_COMMIT_TAG} Version is available!"
But i'm still not able to get the environment variable desired when receiving the notification on my channel and passing it like this in the file:
script:
- "curl -X POST -H 'Content-type: application/json' --data '{\"text\":\"The version ${CI_COMMIT_TAG} version is available!\"} ' https://hooks.slack.com/services/....../......"
Do you have any clues ? I'm not used to Curl and Yaml
Thanks and Have a good day!

--data '...'
Variable expansion in bash does not work within single quotes. Use double quotes instead.
Alternatively, use a data file to avoid formatting JSON inline:
curl -d "#data.json" ...

Related

How do I format a variable inside the broken double quotes of a curl command?

I am trying to test the Sumo Logic API by updating the information of my collector. The second curl command is the one that is causing the issue 'curl: (55) Failed sending PUT request'. It works in my terminal but not in the bash script.
#!/bin/bash
readonly etag=$(curl -u '<accessId>:<accessKey>' -I -X GET https://api.sumologic.com/api/v1/collectors/<id> | grep -Fi etag | awk '{print $2}' | tr -d \''"\')
echo ${etag}
curl -vvv -u '<accessId>:<accessKey>' -X PUT -H "Content-Type: application/json" -H "If-Match: \"${etag}\"" -T updated_collector.json https://api.sumologic.com/api/v1/collectors/<id>
set -x
The first curl command is assigned to the variable called 'etag' which stores the necessary etag. The etag is used in the second curl command to make a request to update the information stored in the 'updated_collector.json'. The updated_collector.json file is not the issue as I have successfully updated the information via the terminal with it. I suspect the content-type is not being sent in the header because someone ran the script on their end and it was not showing that information with the -vvv tag.
Here you can find the Sumo Logic Collector API Methods and Examples from which I got the curl commands to test the API: https://help.sumologic.com/APIs/Collector-Management-API/Collector-API-Methods-and-Examples
Update: I retieved the etag and then ran the second command in a bash script. I manually inserted the etag into the ${etag} portion of the second curl command. I then ran the script and it worked. Therefore, the etag variable isn't correctly formatted inside the second curl command. I do not know how to fix this.
The issue was partially the syntax but after fixing that, I was still getting an error. "If-Match: \"${etag}\" in my command should be "If-Match: ${etag}" instead. I had to add the --http1.1 flag for it to work. I'm sure this is a sumo logic issue. I am able to execute GET requests no problem using http2.0.

How to override .travis.yml environment variables from ruby script using travis client

I'm using the travis client to write a ruby script to interact with my TravisCI builds. I have a working .travis.yml file with a series of encrypted env vars. I'm trying to trigger the build from my script with a new env_var that overwrites one of the existing encrypted env vars in the travis.yml but I am unable to override the .yml configuration.
When making API call using CURL, I can successfully override the env var.
body='{
"request": {
"branch":"master",
"config": {
"merge_mode": "deep_merge",
"env": {
"SOME_ENV_VAR_DEFINED_IN_YML": '"$some_new_value_for_the_old_key"'
}
}
}
}'
curl -s -X POST \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-H "Travis-API-Version: 3" \
-H "Authorization: token MYSECRETTOKEN" \
-d "$body" \
https://api.travis-ci.com/repo/MYORG%2FMYREPO/requests
This is what I'm trying to do with the client in the script but not able to.
I've tried getting the repo and setting some env vars thusly:
my_repo = Travis::Pro::Repository.find("MYREPO")
my_repo.env_vars.upsert("SOME_ENV_VAR_DEFINED_IN_YML", "some_new_value", public: true)
Nada. It sets the env var on the repo that I can see in the travis ui but doesn't override the .yml config.
I've also tried setting the config object on the build that I want.
build_that_i_am_targeting.config["global_env"][index_of_env_var]="SOME_ENV_VAR_DEFINED_IN_YML=some_new_value"
Also nada - when setting off the build it resets the global env vars to the previous version.
Link to what I thought was relevant documentation: https://github.com/travis-ci/travis.rb#build-environment-variables
I was considering using the client session (just below the above in the docummentation) but I feel like there is something simpler that I'm missing. Any suggestions/ideas much appreciated! Thanks!
I know this is very late, but for future reference. I'm not totally sure what you're trying to do, but I had a similar problem and here are two possible solutions.
If your curl call is working the way you want it to, but you just need to convert it to ruby, you can use this online "curl to ruby converter" tool: https://jhawthorn.github.io/curl-to-ruby/
If you want to add an environment variable to your .travis.yml file locally using travis ci (and subsequently push it out when you're ready), you can do something like:
#your_ruby_script.rb
env=some_value
travis_cmd = "travis encrypt MY_NEW_ENV=#{env} --add env.global"
system(travis_cmd)
This answer has more info on calling shell commands from ruby: How to call shell commands from Ruby

Get Jenkins job status by using curl

I know, that there is a way to enable or disable jenkins job by using commands
curl -u user:password -X POST http://server/job/jobname/enable
curl -u user:password -X POST http://server/job/jobname/disable
But what I need - is to get the status of job "enabled/disables" and write it to the bash $status variable.
Is there a way to do it?
You can check if job is enabled or disabled using the API
http://server:port/job/jobname/api/xml?xpath=*/buildable
So, with crumb, you can use something like this:
CRUMB=$(curl -s 'http://USER:PASSWORD#SERVER:PORT/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
myStatus=$(curl -X POST -H "$CRUMB" "http://USER:PASSWORD#SERVER:PORT/job/jobname/api/xml?xpath=*/buildable")
And in variable myStatus you get
<buildable>true</buildable>
or
<buildable>false</buildable>

Escaping curl command in Windows

I'm trying to run a curl command from the command line in Windows, but for the life of me I can't figure out how I'm supposed to escape it.
I'm executing this:
C:\WINDOWS\system32>curl --anyauth --user user:password -X POST -d "{\"rest-api\":{\"name\":\"BizSimDebug3\"}}" -H "Content-type: application/xml" http://localhost:8002/v1/rest-apis
And I'm getting this:
<rapi:error xmlns:rapi="http://marklogic.com/rest-api">
<rapi:status-code>400</rapi:status-code>
<rapi:status>Bad Request</rapi:status>
<rapi:message-code>RESTAPI-INVALIDCONTENT</rapi:message-code>
<rapi:message>Your bootstrap payload caused the server to throw an error. Underlying error message: XDMP-DOCROOTTEXT: xdmp:get-request-body() -- Invalid root text "{&quot;rest-api&quot;:{&quot;name&quot;:&quot;BizSimDebug3&quot;}}" at line 1</rapi:message>
</rapi:error>
Is there something else I need to do to escape the inner quotes in the -d flag? Or am I overlooking the real issue entirely?
This works in Windows:
curl -i -X POST -H "Content-Type: application/json" -d "{\"Field1\": 123, \"Field2\": 456 }" "http://localhost:8080"
The XDMP-DOCROOTTEXT error indicates the server is trying to parse the payload as XML and failing.
The Content-Type header is telling the server that you're sending XML, but the payload is JSON.
Try changing the Content-Type header to application/json
Quoting is hell. By "Windows Command Line and your prompt I presume you mean cmd.com ?. That doest quote the same as linux shells.
For this simplistic experiment I recommend going for 2 kinds of quotes to avoid escaping But even then its unlikely to work
curl --anyauth --user user:password -X POST -d "{'rest-api':{'name':'BizSimDebug3'}}" -H "Content-type: application/xml" http://localhost:8002/v1/rest-apis
Better luck might be had by going with a unix-like shell such as running cygwin (http://www.cygwin.com/) or maybe xmlsh (www.xmlsh.org) which escape like linux does.
You really are going to have a nightmare running anything complex through the windows command line natively.
-David

How to post PUT request under the Windows using curl?

I need to post XML data via curl.exe under windows using PUT request.
In the curl help I found:
-d/--data <data> HTTP POST data (H)
What should I supply for <data>?
curl sample calls
# with inlining plain data
curl -X PUT -d "payload" http://localhost
# referrring file
curl -X PUT -d #myXmlFile.xml http://localhost
If your windows curl-port does not support it go for cygwin. It is a linux-like environment for windows and also offers "a proper" curl.
In windows, if a double-quoted argument itself contains a double quote character, the double quote must be doubled.
For example, enter 'This is "quoted" payload' as "This is ""quoted"" payload" which is very different than in Unix.
Example:
curl -X PUT -d "This is ""quoted"" payload" http://localhost
in windows you'll need to put the # inside the quotes for the file you're sending:
curl -XPUT --data-binary "#uploadme.txt"
otherwise you'll get weird errors as it tries to use the content of the file as the url:
curl: (6) Couldn't resolve host 'upload'
curl: (6) Couldn't resolve host 'me!'
(uploadme.txt contains "upload me!")
on Windows CMD, curl refers to C:\Windows\System32\curl.exe so you can use
curl -X PUT -d "payload" http://localhost
instead on Windows PowerShell curl refers to Invoke-WebRequest so it is not working with curl syntax. you can use curl.exe on PowerShell to call C:\Windows\System32\curl.exe so it will solve the issue.
curl.exe -X PUT -d "payload" http://localhost

Resources