I'm trying to write some Ruby code to update GitLab CI/CD variables using the REST endpoint update variable. When I perform a curl with the same path, the same private token, and the same --form data it updates the variable as expected. When I use the Ruby code that I put together based on reading stackoverflow and the net::http docs, it fails with a 404 URL not found.
I can use a similar piece of code to create a new CI/CD variable successfully. I can also delete an existing variable, and re-create it, but it I would like to know the mistake I am making in the update call.
Can someone point out what I did wrong?
#!/usr/bin/env ruby
require 'net/http'
require 'uri'
token = File.read(__dir__ + '/.gitlab-token').chomp
host = 'https://gitlab.com/'
variables_path = 'api/v4/projects/123456/variables'
env_var = 'MY_VAR'
update_uri = URI(host + variables_path + '/' + env_var)
# I've written the above this way because my actual code
# has a delete and create in order to "update" the variable
response = Net::HTTP.start(update_uri.host, update_uri.port, use_ssl: true) do |http|
update_request = Net::HTTP::Post.new(update_uri)
update_request['PRIVATE-TOKEN'] = token
form_data = [
['value', 'a new value']
]
update_request.set_form(form_data, 'multipart/form-data')
response = http.request(update_request)
response.body
end
Related
I have been trying for days to pull down activity data from the Withings API using the OAuth Ruby gem. Regardless of what method I try I consistently get back a 503 error response (not enough params) even though I copied the example URI from the documentation, having of course swapped out the userid. Has anybody had any luck with this in the past. I hope it is just something stupid I am doing.
class Withings
API_KEY = 'REMOVED'
API_SECRET = 'REMOVED'
CONFIGURATION = { site: 'https://oauth.withings.com', request_token_path: '/account/request_token',
access_token_path: '/account/access_token', authorize_path: '/account/authorize' }
before do
#consumer = OAuth::Consumer.new API_KEY, API_SECRET, CONFIGURATION
#base_url ||= "#{request.env['rack.url_scheme']}://#{request.env['HTTP_HOST']}#{request.env['SCRIPT_NAME']}"
end
get '/' do
#request_token = #consumer.get_request_token oauth_callback: "#{#base_url}/access_token"
session[:token] = #request_token.token
session[:secret] = #request_token.secret
redirect #request_token.authorize_url
end
get '/access_token' do
#request_token = OAuth::RequestToken.new #consumer, session[:token], session[:secret]
#access_token = #request_token.get_access_token oauth_verifier: params[:oauth_verifier]
session[:token] = #access_token.token
session[:secret] = #access_token.secret
session[:userid] = params[:userid]
redirect "#{#base_url}/activity"
end
get '/activity' do
#access_token = OAuth::AccessToken.new #consumer, session[:token], session[:secret]
response = #access_token.get("http://wbsapi.withings.net/v2/measure?action=getactivity&userid=#{session[:userid]}&startdateymd=2014-01-01&enddateymd=2014-05-09")
JSON.parse(response.body)
end
end
For other API endpoints I get an error response of 247 - The userid provided is absent, or incorrect. This is really frustrating. Thanks
So I figured out the answer after copious amount of Googleing and grasping a better understanding of both the Withings API and the OAuth library I was using. Basically Withings uses query strings to pass in API parameters. I though I was going about passing these parameters correctly when I was making API calls, but apparently I needed to explicitly set the OAuth library to use the query string scheme, like so
http_method: :get, scheme: :query_string
This is appended to my OAuth consumer configuration and all worked fine immediately.
I'm green when it comes to Ruby. Right now I'm mucking about with a script which connects to the Terremark eCloud API Explorer. I'm trying to use the httpclient gem, but I'm a bit confused as to how I'm supposed to construct my client.
#!/usr/bin/ruby
require "httpclient"
require 'base64'
require 'hmac-sha1'
require 'openssl'
# Method definitions
def get_date
# Get the time and date in the necessary format
result = Time.now.strftime('%a, %d %b %Y %H:%M:%S GMT')
end
def get_signature(action,date,headers,resource,user,pass)
string_to_sign = "#{action}
#{date}
#{headers}
#{resource}\n"
return Base64.encode64(OpenSSL::HMAC.digest('sha1', "#{user}:#{pass}", "#{string_to_sign}"))
end
# Initial variables
date = get_date
domain = "https://services.enterprisecloud.terremark.com"
password = 'password'
query = {}
tmrk_headers = Hash.new
tmrk_headers['x-tmrk-date: '] = date
tmrk_headers['x-tmrk-version: '] = '2013-06-01'
uri = '/cloudapi/spec/networks/environments/1'
url = "#{domain}#{uri}"
username = 'user#terremark.com'
verb = 'GET'
signature = get_signature(verb,date,tmrk_headers,uri,username,password)
tmrk_headers['Authorization: '] = "Basic \"#{signature}\""
puts signature
client = HTTPClient.new
client.get_content(url,query,tmrk_headers)
EDIT: This is no longer valid as I've moved beyond this error with some help:
Right now I'm not concerned about seeing what is returned from the connection. I'm just looking to create an error-free run. For instance, if I run the script without the client.get_content line it will return to a prompt without issue (giving me the impression that everything ran cleanly, if not uselessly).
How am I supposed to construct this? The httpclient documentation uses the example with external headers:
extheader = [['Accept', 'image/jpeg'], ['Accept', 'image/png']]
clnt.get_content(uri, query, extheader)
I'm making the assumption that the query is the URI that I've defined.
In all reality, it isn't set up right in the first place. I need to be able to include the string in the auth_header variable in the string to be signed but the signature is actually part of the variable. I've obviously created a hole in that regard.
Any assistance with this will be more than appreciated.
EDIT2: Removed strace pastebin. Adding Ruby backtrace:
/home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1023:in `create_request': undefined method `each' for #<String:0x0000000207d1e8> (NoMethodError)
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:884:in `do_request'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:959:in `follow_redirect'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:594:in `get_content'
from ./test.rb:42:in `<main>'
EDIT3: Updated script; adding further backtrace after making necessary script modifications:
/
home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:975:in `success_content': unexpected response: #<HTTP::Message::Headers:0x00000001dddc58 #http_version="1.1", #body_size=0, #chunked=false, #request_method="GET", #request_uri=#<URI::HTTPS:0x00000001ddecc0 URL:https://services.enterprisecloud.terremark.com/cloudapi/spec/networks/environments/1>, #request_query={}, #request_absolute_uri=nil, #status_code=400, #reason_phrase="Bad Request", #body_type=nil, #body_charset=nil, #body_date=nil, #body_encoding=#<Encoding:US-ASCII>, #is_request=false, #header_item=[["Content-Type", "text/html; charset=us-ascii"], ["Server", "Microsoft-HTTPAPI/2.0"], ["Date", "Thu, 27 Mar 2014 23:12:53 GMT"], ["Connection", "close"], ["Content-Length", "339"]], #dumped=false> (HTTPClient::BadResponseError)
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:594:in `get_content'
from ./test.rb:52:in `<main>'
The issue that you're having as stated by your backtrace
/home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1023:in `create_request': undefined method `each' for #<String:0x0000000207d1e8> (NoMethodError)
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:884:in `do_request'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:959:in `follow_redirect'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:594:in `get_content'
from ./test.rb:42:in `<main>'
is that it seems like you're passing a String object to one of the arguments in get_content where it expects an object that responds to the method each.
From looking at the documentation of httpclient#get_content http://www.ruby-doc.org/gems/docs/h/httpclient-xaop-2.1.6/HTTPClient.html#method-i-get_content
It expects the second parameter to be a Hash or Array of arguments
From your code sample and showing only the relevant parts
uri = '/cloudapi/spec/networks/environments/1'
url = "https://services.enterprisecloud.terremark.com"
tmrk_headers = "x-tmrk-date:\"#{date}\"\nx-tmrk-version:2014-01-01"
auth_header = "Authorization: CloudApi AccessKey=\"#{access_key}\" SignatureType=\"HmacSHA1\" Signature=\"#{signature}\""
full_header = "#{tmrk_headers}\n#{auth_header}"
client = HTTPClient.new
client.get_content(url,uri,full_header)
There are two things that I see wrong with your code.
You're passing in a String value for the query. Specifically, you're passing in uri which has a value of what I'm assuming is the path that you want to hit.
For the extra headers parameter, you're passing in a String value which is in the full_header
What you need to do in order to fix this is pass in the full url for the first parameter.
This means it should look something like this:
url = "https://services.enterprisecloud.terremark.com/cloudapi/spec/networks/environments/1"
query = {} # if you have any parameters to pass in they should be here.
headers = {
"x-tmrk-date" => date, "x-tmrk-version" => "2014-01-01",
"Authorization" => "CloudApi AccessKey=#{access_key} SignatureType=HmacSHA1 Signature=#{signature}"
}
client = HTTPClient.new
client.get_content(url, query, headers)
I'm doing a Post request to github at this url:
https://api.github.com/gists/2710948/comments
Theoretically, this should create a comment with the text being formed from what's in the request body. However, when I try to make that post, I get a 404 error. That leads me to believe that the gist is not being found, however, if you do a Get request at the same address it comes up just fine.
Is there an authentication thing I need to be doing? I've tried adding a username and password to my headers collection but I've got no idea if I'm using the right format. I've tried making this work via Ruby, HTTP Client, and curl, and I get the same error either way.
The curl command I'm using is this:
curl -X POST -d "This is my sample comment" https://api.github.com/gists/2710948/comments
I think that if I can get the curl command working, I'll be able to figure out the HTTP Client and then the Ruby. This will be my first attempt at consuming an API, so there's nothing too basic for me to double-check; all suggestions will be helpful.
curl -d '{ "body": "Test comment" }' -u "Username:Pass" -X POST https://api.github.com/gists/2710948/comments
Ruby code:
require 'net/http'
uri = URI("https://api.github.com/gists/2710948/comments")
req = Net::HTTP::Post.new(uri.to_s)
req.basic_auth("Username", "Pass")
req.body = '{"body": "Test message"}' # `to_json` can be used
req["content-type"] = "application/json"
Net::HTTP.start(uri.host, uri.port, :use_ssl => true) do |http|
p response = http.request(req)
end
See also http://developer.github.com/v3/gists/comments/
Ruby's Net:HTTP needs to be given a full URL in order for it to connect to the server and get the file properly. By "full URL" I mean a URL including the http:// part and the trailing slash if it needs it. For instance, Net:HTTP won't connect to a URL looking like this: example.com, but will connect just fine to http://example.com/. Is there any way to make sure a URL is a full URL, and add the required parts if it isn't?
EDIT: Here is the code I am using:
parsed_url = URI.parse(url)
req = Net::HTTP::Get.new(parsed_url.path)
res = Net::HTTP.start(parsed_url.host, parsed_url.port) {|http|
http.request(req)
}
If this is only doing what the sample code shows, Open-URI would be an easier approach.
require 'open-uri'
res = open(url).read
This would do a simple check for http/https:
if !(url =~ /^https?:/i)
url = "http://" + url
end
This could be a more general one to handle multiple protocols (ftp, etc.)
if !(url =~ /^\w:/i)
url = "http://" + url
end
In order to make sure parsed_url.path gives you a proper value (it should be / when no specific path was provided), you could do something like this:
req = Net::HTTP::Get.new(parsed_url.path.empty? ? '/' : parsed_url.path)
I am doing a video crawler in ruby. In there I have to log in to a page by enabling cookies and download pages. For that I am using the CURL library in ruby. I can successfully log in, but I can't download the pages inside that with curl. How can I fix this or download the pages otherwise?
My code is
curl = Curl::Easy.new(1st url)
curl.follow_location = true
curl.enable_cookies = true
curl.cookiefile = "cookie.txt"
curl.cookiejar = "cookie.txt"
curl.http_post(1st url,field)
curl.perform
curl = Curl::Easy.perform(2nd url)
curl.follow_location = true
curl.enable_cookies = true
curl.cookiefile = "cookie.txt"
curl.cookiejar = "cookie.txt"
curl.http_get
code = curl.body_str
What I've seen in writing my own similar "post-then-get" script is that ruby/Curb (I'm using version 0.7.15 with ruby 1.8) seems to ignore the cookiejar/cookiefile fields of a Curl::Easy object. If I set either of those fields and the http_post completes successfully, no cookiejar or cookiefile file is created. Also, curl.cookies will still be nil after your curl.http_post, however, the cookies ARE set within the curl object. I promise :)
I think where you're going wrong is here:
curl = Curl::Easy.perform(2nd url)
The curb documentation states that this creates a new object. That new object doesn't have any of your existing cookies set. If you change your code to look like the following, I believe it should work. I've also removed the curl.perform for the first url since curl.http_post already implicitly does the "perform". You were basically http_post'ing twice before trying your http_get.
curl = Curl::Easy.new(1st url)
curl.follow_location = true
curl.enable_cookies = true
curl.http_post(1st url,field)
curl.url = 2nd url
curl.http_get
code = curl.body_str
If this still doesn't seem to be working for you, you can verify if the cookie is getting set by adding
curl.verbose = true
Before
curl.http_post
Your Curl::Easy object will dump all the headers that it gets in the response from the server to $stdout, and somewhere in there you should see a line stating that it added/set a cookie. I don't have any example output right now but I'll try to post a follow-up soon.
HTTPClient automatically enables cookies, as does Mechanize.
From the HTTPClient docs:
clnt = HTTPClient.new
clnt.get_content(url1) # receives Cookies.
clnt.get_content(url2) # sends Cookies if needed.
Posting a form is easy too:
body = { 'keyword' => 'ruby', 'lang' => 'en' }
res = clnt.post(uri, body)
Mechanize makes this sort of thing really simple (It will handle storing the cookies, among other things).