I was trying to send API request to my rails server from AWS Lambda function.
I was using httparty gem to send request.
I have tried with below code
require "httparty"
class PostManager
include HTTParty
def initialize
end
def create_post(job_id)
puts "----------------- Inside post manager ------------------"
puts "----------------- #{ENV["BASE_URI"]} ------------------"
puts "#{ENV['BASE_URI']}/call_response?job_id=#{job_id}"
response = HTTParty.get("#{ENV['BASE_URI']}")
puts "******************HTTP Response -: #{response}******************"
response
end
end
I am triggering this code from aws lambda main handler like below.
post_manager = PostManager.new
response = post_manager.create_post(job_id)
But lambda function gets timeout. Request not reaching to rails server at all.
Please guide me if i am missing something. Other alternatice to send post request to external server from aws lambda function is also invited.
Since http party is a http client, I recommend read the documentation, start experimenting with pry and a site like http://httpbin.org
So went will have all the thinks. reading your code I'm not sure of what you want to achieve, but I think that you want to connect to some en point that is on:
The domain inside this shell variable => #{ENV['BASE_URI']}
the path of the http method => call_response
and some path parameters like => job_id=#{job_id}
You allways say that this is a post but you are doing a get => HTTParty.get
So let's start with some object like the one showed in the documentation in order to attack to this method with curl will be something like this:
❯ curl -X GET "http://httpbin.org/get?job_id=4" -H "accept: application/json" ~/learn/ruby/stackoverflow
{
"args": {
"job_id": "4"
},
"headers": {
"Accept": "application/json",
"Host": "httpbin.org",
"User-Agent": "curl/7.64.1",
"X-Amzn-Trace-Id": "Root=1-613b73ef-2d00166a2ae40e704b448352"
},
"origin": "83.53.251.55",
"url": "http://httpbin.org/get?job_id=4"
}
Then for an http client object
add this to afile called httpbin_client.rb :
require 'httparty'
class HTTPbinClient
include HTTParty
base_uri ENV['BASE_URI']
def initialize
end
def ask_for_job_id(job_id)
self.class.get('/get', {query: {job_id: job_id}})
end
end
http_bin = HTTPbinClient.new
puts http_bin.ask_for_job_id(28)
call like this:
❯ BASE_URI=httpbin.org ruby httpbin_client.rb ~/learn/ruby/stackoverflow
{
"args": {
"job_id": "28"
},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip;q=1.0,deflate;q=0.6,identity;q=0.3",
"Host": "httpbin.org",
"User-Agent": "Ruby",
"X-Amzn-Trace-Id": "Root=1-613b776d-47036b9b29bb1ae34b4a0e50"
},
"origin": "83.53.251.55",
"url": "http://httpbin.org/get?job_id=28"
}
Related
Description
I want to query the data in our Elastic Cloud instance using the REST API with authorization via API Key.
Steps taken
I tried using the SQL API as well as the Search API. NB that I would much preferably use the SQL API.
Based on the following curl commands provided in the documentation:
curl -X POST "localhost:9200/_sql?format=txt&pretty" -H 'Content-Type: application/json' -d'
{
"query": "SELECT * FROM library WHERE release_date < \u00272000-01-01\u0027"
}
'
[source]
curl -u elastic:password https://CLUSTER_ID.REGION.PLATFORM.found.io:9243/my_index/my_type -XPOST -d '{
"title": "One", "tags": ["ruby"]
}'
{"_index":"my_index","_type":"my_type","_id":"AV3ZeXsOMOVbmlCACuwj","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}
[source]
and the documentation about the REST API, I attempted the following:
import base64
import json
import requests
if __name__ == '__main__':
response1 = requests.post(
'https://{redacted-id}.northamerica-northeast1.gcp.elastic-cloud.com:9243/_sql?format=txt',
headers={
"Authorization": base64.standard_b64encode(bytes('{API_KEY_ID}:{API_KEY_KEY}', 'utf-8')),
"Content-Type": 'application/json'
},
data={
"query": """
...
"""
}
)
print('response1')
print(response1)
response2 = requests.get(
'https://{redacted-id}.northamerica-northeast1.gcp.elastic-cloud.com:9243/logs-pubsub/_search',
headers={
"Authorization": base64.standard_b64encode(bytes('{API_KEY_ID}:{API_KEY_KEY}', 'utf-8')),
"Content-Type": 'application/json'
},
data={
"query": {
# ...
}
}
)
print('response2')
print(response2)
But both queries answer with 404 - Not Found.
What did I miss? Am I missing a part of the path like /api/..., /rest/...? Or is this a misdirection from a 403 to a 404 and the issue is the API Key?
Thanks!
The kind folks at ElasticStack provided me with the following answer. There were two issues:
The URL above was pointing to the Kibana instance, not to the ElasticCloud instance
In order to get the appropriate URL:
Navigate to https://cloud.elastic.co/deployments/
Click on your deployment:
Click on Copy Endpoint for the Elasticsearch endpoint:
The Authorization header was malformed
The Authorization header for an ApiKey powered request is the following:
ApiKey {B64_ENCODE('API_KEY_ID:API_KEY_KEY')}
Which can be written in python as:
"ApiKey " + str(base64.standard_b64encode(bytes('API_KEY_ID:API_KEY_KEY', 'utf-8')), 'utf-8')
On a final note, the team also strongly suggested I look at their Free on-demand Elasticsearch training.
Hopefully this will be helpful to whoever passes by here!
I've been working on creating a CLI gem for a job board. I've been setting up my API class, but I have been struggling to get it to work correctly in terms of successful calls; I'm using HTTParty to parse. When I have been testing this, it keeps giving me a method error for "[]". I've gone over everything, made sure the syntax is correct but have hit a wall in figuring out what seems to break this. Here is the method I created to list all of the jobs on the specific board:
def all_jobs_call
url = "https://boards-api.greenhouse.io/v1/boards/flatironschoolcareers/jobs"
response = HTTParty.get(url)
response["absolute_url"]["location"]["metadata"]["id"]["title"].each do |job|
absolute_url = job["absolute_url"]
location = job["location"]
metadata = job["metadata"]
id = job["id"]
title = job["title"]
end
end
I would greatly appreciate any insight as to what I could be doing wrong or if I'm missing something glaring. Thanks!
The JSON response you get from https://boards-api.greenhouse.io/v1/boards/flatironschoolcareers/jobs looks like this:
{
"jobs": [
{
"absolute_url": "https://boards.greenhouse.io/flatironschoolcareers/jobs/4460392002",
"internal_job_id": 4375855002,
"location": {
"name": "New York, NY"
},
"metadata": [
{
"id": 4019377002,
"name": "Employment Type",
"value": "Full-time",
"value_type": "single_select"
},
...
HTTParty converts that response to Ruby objects. So just like in that JSON response, response has a top level "jobs" key which contains an array of jobs.
In order to get the 1st job you'd use:
response["jobs"][0]
#=> {"absolute_url"=>"https://boa...", "internal_job_id"=>4375855002, ...}
and to get it's absolute_url:
response["jobs"][0]["absolute_url"]
#=> "https://boards.greenhouse.io/flatironschoolcareers/jobs/4460392002"
And to traverse all jobs you call each on the array, i.e.:
response["jobs"].each do |job|
puts job["absolute_url"]
end
Output:
https://boards.greenhouse.io/flatironschoolcareers/jobs/4460392002
https://boards.greenhouse.io/flatironschoolcareers/jobs/4460383002
https://boards.greenhouse.io/flatironschoolcareers/jobs/4472889002
...
I have a standalone instance of Wiremock server. The mappings are stored as json files under the mappings folder. I have a POST request that needs to return a dynamic ID(integer) in the response. Is there a way to configure this in the json file?
To make the above examples work, I had to run the standalone jar with the --global-response-templating. Then I saw, for example, {{now}} working which is what I wanted. Not sure if the documentation specifies this -- I tried the always-useful --help.
In WireMock there are a number of response template helper functions for generating random strings. In the below example I'm using the one for generating a UUID, but several other options exist.
Mapping file: dynamic_id.json
{
"request": {
"method": "POST",
"url": "/dynamic_id"
},
"response": {
"headers": {
"Content-Type": "application/json"
},
"status": 200,
"body": "{{randomValue type='UUID'}}",
"transformers": ["response-template"]
}
}
Using an empty POST http://wiremock/dynamic_id will return an id similar to: c2e6bf32-c9a3-45c0-b988-94fad04cc7a2.
Start WireMock:
java -jar wiremock-standalone-2.18.0.jar --port 8181 --verbose --local-response-templating
This seems like a perfect use-case for OpenTable's Wiremock Body Transformer.
It can be easily integrated with the Standalone Server like this:
java -cp "wiremock-body-transformer-1.1.6.jar:wiremock-2.3.1-standalone.jar" com.github.tomakehurst.wiremock.standalone.WireMockServerRunner --verbose --extensions com.opentable.extension.BodyTransformer
And allows you to easily specify a dynamic variable that you would want to match in the response.
Here is an example to get a random integer without having to specify anything in the request, however if you need to match a specific variable in the request to the response, then that is also very doable with this extension and numerous examples can be found in the readme.
{
"request": {
"method": "POST",
"urlPath": "/transform",
},
"response": {
"status": 200,
"body": "{\"randomInteger\": \"$(!RandomInteger)\"}",
"headers": {
"Content-Type": "application/json"
},
"transformers": ["body-transformer"]
}
}
As #Jeff mentions, If you are running it as a stand-alone process, you need to add this flag --global-response-templating. This will apply templating to each and every reponse. However, few of your responses may be jsut plain json with no templating required.
In that case use --local-response-templating. and add this field inside reponse json:
response:{
"transformers": ["response-template"]
}
I have a json object that I'm sending to Google's QXP Express API. The idea is that I send the object with the relevant travel information. In terminal, through curl, it's very easy to send it. I just use the following curl command. Doc.json is the file name of the json.
curl -d #doc.json --header "Content-Type: application/json" https://www.googleapis.com/qpxExpress/v1/trips/search?key=AIzaSyAaLHEBBLCI4aHLNu2jHiiAQGDbCunBQX0
This is my code to do it in Ruby.
uri = URI('https://www.googleapis.com/qpxExpress/v1/trips/search?key=MYAPIKEY')
req = Net::HTTP::Post.new uri.path
req.body = {
"request" => {
"passengers" => {
"adultCount" => 1
},
"slice" => [
{
"origin" => "BOS",
"destination" => "LAX",
"date" => "2014-10-14"
},
{
"origin" => "LAX",
"destination" => "BOS",
"date" => "2014-11-14"
}
]
}
}.to_json
res = Net::HTTP.start(uri.host, uri.port, :use_ssl => true) do |http|
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
http.ssl_version = :SSLv3
http.request req
end
puts res.body
However I'm getting back the following error.
{
"error": {
"errors": [
{
"domain": "global",
"reason": "parseError",
"message": "This API does not support parsing form-encoded input."
}
],
"code": 400,
"message": "This API does not support parsing form-encoded input."
}
}
I just need to send it with the json file, but nothing I can find online covers sending json's to APIs. Please help, I'm very stuck.
It is always matter of taste what tools you prefer, but as for me i am currently using the rest-client gem for accessing REST APIs. With this library your example could be written like this:
require 'json'
require 'rest-client'
response = RestClient.post 'https://www.googleapis.com/qpxExpress/v1/trips/search?key=AIzaSyAaLHEBBLCI4aHLNu2jHiiAQGDbCunBQX0',
{
request: {
passengers: {
adultCount: 1
},
slice: [
{
origin: "BOS",
destination: "LAX",
date: "2014-10-14"
},
{
origin: "LAX",
destination: "BOS",
date: "2014-11-14"
}
]
}
}.to_json,
:content_type => :json
puts response.body
But if you want a Net::HTTP only solution, this might not be a suitable answer for you.
I have an integration test which uses eventmachine to receive http requests. This is my eventmachine handler:
class NotificationRecipient < EM::Connection
def receive_data(data)
EM.stop
end
end
I need to test various properies of the received request, e.g. I want to extract a json payload from an HTTP POST request string I've received like this. Is there a nicely packaged way to do it?
Googling finds lots of ways to make a request and parse the response, e.g. rest-client will parse response automatically. However, since I'm receiving the request, not making it, none of these ways work for me.
I would make use of WEBrick. WEBrick::HTTPRequest has a serviceable parser, and all you need to do is pass an IO object to its parse method and you have yourself an object you can manipulate.
This example declares a POST request with a JSON body in a string and uses StringIO to make it accessible as an IO object.
require 'webrick'
require 'stringio'
Request = <<-HTTP
POST /url/path HTTP/1.1
Host: my.hostname.com
Content-Type: application/json
Content-Length: 62
{
"firstName": "John",
"lastName": "Smith",
"age": 25
}
HTTP
req = WEBrick::HTTPRequest.new(WEBrick::Config::HTTP)
req.parse(StringIO.new(Request))
puts req.path
req.each { |head| puts "#{head}: #{req[head]}" }
puts req.body
output
/url/path
host: my.hostname.com
content-type: application/json
content-length: 62
{
"firstName": "John",
"lastName": "Smith",
"age": 25
}