if I have a log file such as the one below, how can I take advantage of logstash to extract pieces of information I need and push that into ES?
test_web_events.py: START: Mon Apr 27 13:35:25 2015
# TESTCASE TestWebPost ==================================================
# START TEST METHOD #################################: test_10_post_valid_json
[2015-04-27T13:35:25.657887] HTTP DELETE http://pppdc3mu.net:8080/rastplatz/v1/sink/db?k0=bradford4
{}
HTTP response: 200
0
POSTING event_id b29b6c7c-48cd-4cd9-b3c4-aa0a7edc1f35 to ctg-business
Content-Type: text/plain
POSTING event_id 13678af1-3e3a-4a6e-a61c-404eb94b9768 to ctg-business
Content-Type: text/plain
POSTING event_id 47b70306-2e7c-4cb2-9e75-5755d8d101d4 to ctg-business
Content-Type: text/plain
POSTING event_id 6599cdb2-0630-470d-879d-1130cf70c605 to ctg-business
Content-Type: text/plain
POSTING event_id d088ce29-fa0d-4f45-b628-045dba1fd045 to ctg-business
Content-Type: text/plain
POSTING event_id 07d14813-b561-442c-9b86-dc40d1fcc721 to ctg-business
Content-Type: text/plain
POSTING event_id b6aea24a-5424-4a0f-aac6-8cbaecc410db to ctg-business
Content-Type: text/plain
POSTING event_id 016386bd-eac5-4f1c-8afc-a66326d37ddb to ctg-business
Content-Type: text/plain
POSTING event_id 6610485d-71af-4dfa-9268-54be5408a793 to ctg-business
Content-Type: text/plain
POSTING event_id 92786434-02f7-4248-a77b-bdd9d33b57be to ctg-business
Content-Type: text/plain
Posted 10 events
# END TEST METHOD ###################################: test_10_post_valid_json
test_web_events.py: FINISH: Mon Apr 27 13:35:36 2015
Use the multiline filter to join everything up to one event, and roll it into Elasticsearch.
You can grok{} and otherwise filter it on the way through, too.
Related
I'm accessing a REST service for data query and download. This is the very first call that performs the authentication. The response is a json structure that contains an authentication token.
When I do this call with curl ...
$ curl -v -X POST ${AUTH_URL} \
-H 'Content-Type: application/x-www-form-urlencoded' \
-d 'apikey='${API_KEY}'&grant_type=api_key&client_id=IDP'
... I get the following response.
First the headers:
< server: IIS
< date: Thu, 25 Feb 2021 17:59:34 GMT
< content-type: application/json
< content-length: 1500
< x-content-type-options: nosniff
< x-xss-protection: 1; mode=block
< strict-transport-security: max-age=31536000; includeSubdomains;
< cache-control: no-store
< set-cookie: KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT; Max-Age=0; Path=/auth/realms/IDP/; HttpOnly
< pragma: no-cache
< x-frame-options: SAMEORIGIN
< referrer-policy: no-referrer
< vary: Origin
< via: 1.1 google
< alt-svc: clear
And the content:
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIxMThSRENzZTlqUWR4UVRnSkt2ZXlvSHBaaWE4R0pIVEU5RjJPSmE1M3N3In0.eyJleHAiOjE2MTQyOTAzNzQsImlhdCI6MTYxNDI3NTk3NCwianRpIjoiNzBkMmMwZGMtZWY3Yy00NDM5LWJiNmUtNmQ4MDEzZGU2YTU0IiwiaXNzIjoiaHR0cHM6Ly9hdXRoZW50aWNhdGUuZm91bmRhdGlvbi5hcGkub25lYXRsYXMuYWlyYnVzLmNvbS9hdXRoL3JlYWxtcy9JRFAiLCJhdWQiOiJJRFAiLCJzdWIiOiJmNmI0ZjVkNC0zMzZiLTRlMTctODc3Ni1jNjA1ZTczNTRjYmIiLCJ0eXAiOiJCZWFyZXIiLCJhenAiOiJJRFAiLCJzZXNzaW9uX3N0YXRlIjoiODJkYWM4MjMtMTViOS00MmVlLWE2YzEtZjg2ZDQ2MzY1ZjAzIiwiYWNyIjoiMSIsInNjb3BlIjoiIiwibmJmIjowLCJyb2xlIjoie1wiZ2VvLmlkcC5ub3RpZnlcIjpbXCJ1c2VyXCJdLFwiZ2VvLmFwcC5vYWRcIjpbXCJ1c2VyXCJdLFwiZ2VvLmlkcC5kYXRhc3RvcmVcIjpbXCJ1c2VyXCJdLFwiZ2VvLmFwcC53b3JrYmVuY2hcIjpbXCJ1c2VyXCJdfSIsInJvbGVzIjp7Imdlby5pZHAubm90aWZ5IjpbInVzZXIiXSwiZ2VvLmFwcC5vYWQiOlsidXNlciJdLCJnZW8uaWRwLmRhdGFzdG9yZSI6WyJ1c2VyIl0sImdlby5hcHAud29ya2JlbmNoIjpbInVzZXIiXX0sInN1aWQiOiIxMTg2NTEzNTQ2IiwidXVpZCI6Ijc5Yjg0MTZlLTY0NDYtNGMwYy1hODg1LWY1MzE2ZGMzOWMyZSIsImxvYSI6MTAwfQ.5W_E4fkhirbJZNAJ_TwMbLhcKdmnHBXOjvLUr4vW-DBRvSFfQrpdlDHLMIVI4B7bZ-OU_FVnH__i_diKYJFRH4l3Zqy8maa1pyj_WhZJksqBB69ehv8xx_3qtuJCZ0z0hln0FzmyG_Ep_uaru3gK_h33SuFxjdKr4F5XocyrYpGE-ewm-mBLj4DOBnZSJ4HgV0BG02LJIPIU8BybTmvgV-4mW3LXOVKDUJMmP4TF_ZEUzNz4a1vhoW4VIOvaNkk_8v8m_R4zjNOGmd_4jWEywORBZ1ofqvn72usY7TWEVpGBxR-rKYgzWXrdeBE4_l61MT420rBID9dbI2zRgEyVIQ","expires_in":14400,"refresh_expires_in":0,"token_type":"bearer","not-before-policy":0,"session_state":"82dac823-15b9-42ee-a6c1-f86d46365f03","scope":""}
Notice that the length of the content is 1500. That is also what the content-length: 1500 header says. When I do the same test using Python, I get the same result: a 1500 characters result.
But when I do the same test using Oracle UTL_HTTP, the result is only 1453 characters. Here is debugging from my PL/SQL code:
% resp.status_code=200
% resp.reason_phrase=OK
% resp.http_version=HTTP/1.0
% resp.get_headers
% .. Server: IIS
% .. Date: Thu, 25 Feb 2021 17:50:51 GMT
% .. Content-Type: application/json
% .. Content-Length: 1453
% .. X-Content-Type-Options: nosniff
% .. X-XSS-Protection: 1; mode=block
% .. Strict-Transport-Security: max-age=31536000; includeSubdomains;
% .. Cache-Control: no-store
% .. Set-Cookie: KC_RESTART=; Version=1; Expires=Thu, 01-Jan-1970 00:00:10 GMT;
Max-Age=0; Path=/auth/realms/IDP/; HttpOnly
% .. Pragma: no-cache
% .. X-Frame-Options: SAMEORIGIN
% .. Referrer-Policy: no-referrer
% .. Vary: Origin
% .. Via: 1.1 google
% .. Alt-Svc: clear
% Response:
{"access_token":"eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICIxMThSRENzZT
lqUWR4UVRnSkt2ZXlvSHBaaWE4R0pIVEU5RjJPSmE1M3N3In0.eyJleHAiOjE2MTQyODk4NTEsImlhd
CI6MTYxNDI3NTQ1MSwianRpIjoiMDQ4ZjhlMjctZTgwZi00MjIyLWFmNDAtMjZlNDdmYTFhMDg0Iiwi
aXNzIjoiaHR0cHM6Ly8zNS4xOTAuNTkuNzkvYXV0aC9yZWFsbXMvSURQIiwiYXVkIjoiSURQIiwic3V
iIjoiZjZiNGY1ZDQtMzM2Yi00ZTE3LTg3NzYtYzYwNWU3MzU0Y2JiIiwidHlwIjoiQmVhcmVyIiwiYX
pwIjoiSURQIiwic2Vzc2lvbl9zdGF0ZSI6ImYzYmZlOGJiLTFiZGYtNDQ5OS04NDQwLWIxODk5OGYwY
jg5NiIsImFjciI6IjEiLCJzY29wZSI6IiIsIm5iZiI6MCwicm9sZSI6IntcImdlby5pZHAubm90aWZ5
XCI6W1widXNlclwiXSxcImdlby5hcHAub2FkXCI6W1widXNlclwiXSxcImdlby5pZHAuZGF0YXN0b3J
lXCI6W1widXNlclwiXSxcImdlby5hcHAud29ya2JlbmNoXCI6W1widXNlclwiXX0iLCJyb2xlcyI6ey
JnZW8uaWRwLm5vdGlmeSI6WyJ1c2VyIl0sImdlby5hcHAub2FkIjpbInVzZXIiXSwiZ2VvLmlkcC5kY
XRhc3RvcmUiOlsidXNlciJdLCJnZW8uYXBwLndvcmtiZW5jaCI6WyJ1c2VyIl19LCJzdWlkIjoiMTE4
NjUxMzU0NiIsInV1aWQiOiI3OWI4NDE2ZS02NDQ2LTRjMGMtYTg4NS1mNTMxNmRjMzljMmUiLCJsb2E
iOjEwMH0.XHwxx3TzNNwgzVMv18Jav4bqXW9Q4n2bP_HV1iy0K4VPH-w84tXsHjXfH_f05Ynn2CXqv-
rdHds6KtuZaI1aypNnIvNvmbUiNHd6M1geLY4w8Yy9rg9-WFjYiFXbLTP7vvUAMSHueJmeT6WvzAsUT
Z7IQdp0w5aLDQ6ElV8pX1khBMCC7uXedRRDK-UC1MlJBrWtbhIMu5MaqpdpPeBcBMCvmqUBFTFfW6dQ
Ko01jeDjxePz_gZ2wdyU8fkV8UNTzkS3i6PYUkcxi3pmEC5r93JSNGVRUsZ53y5IjcaJK4aRXvvZQzV
iOitsbu8Pfciii2E_NDlk3qYgSqlxVrmzNA","expires_in":14400,"refresh_expires_in":0,
"token_type":"bearer","not-before-policy":0,"session_state":"f3bfe8bb-1bdf-4499
-8440-b18998f0b896","scope":""}
Notice the content length is now 1453 characters. The difference is with the token information inside the JSON response. It should be 1330 characters, but is only 1283 characters. The rest of the JSON document is the same. And the returned token is not valid for further use.
I can't find any explanation as to why the response is shorter when requested from UTL_HTTP. I first thought it had something to do with character set encoding. I have everything set to UTF-8. The token is returned in a base64 encoding.
Here is the code I use (I did not include the debugging code):
-- Setup the http request type and add the content
http_req := utl_http.begin_request(url, 'POST', 'HTTP/1.0');
utl_http.set_header(http_req, 'Content-Type', content_type);
utl_http.set_body_charset(http_req,'UTF-8');
utl_http.set_header(http_req, 'Content-Length', length(post_content));
utl_http.write_text(http_req, post_content);
-- Call the REST endpoint and fetch the http response
http_resp := utl_http.get_response(http_req);
utl_http.set_body_charset(http_resp,'UTF-8');
-- Read the response content
begin
json_response := '';
i := 0;
loop
utl_http.read_line(http_resp, response_line, false);
json_response := json_response || response_line;
i := i + 1;
end loop;
exception
when utl_http.end_of_body then
utl_http.end_response(http_resp);
end;
I have been staring at this problem for hours. I tried various things, like setting or not setting explicit character set encoding, all to not effect. I can't see what I'm doing wrong and why Oracle would do anything with the response. I could imagine it truncating it for some reason - but why would characters be removed from inside the response ?
I'm slightly ashamed. The answer is simple: in the begin_request() call I was explicitly specifying the HTTP 1.0 protocol:
http_req := utl_http.begin_request(url, 'POST', 'HTTP/1.0');
Once I got rid of that (which means Oracle uses the HTTP 1.1 protocol):
http_req := utl_http.begin_request(url, 'POST');
Then everything started working correctly: I now get the full and entire response from the server.
There is still an oddity, in that when I do curl --http1.0 to also force use of HTTP 1.0, I still get the right answer. So I assume that there is something inside the Oracle implementation that causes trouble when using HTTP 1.0 on some workloads.
Is there any way to get artifact details such as timestamp, size and name from rest api, or any groovy script that we can place in task which will list timestamp, size and name and save to as json.
I want to display artifact size, timestamp & name component, search and asset doesn't give any specific details, any groovy or shell script that can capture all the details. Or we can add our own api or groovy to nexus3
Please help me here.
regards,
SAMURAI
assets list gives you the downloadUrl where the file is located
you can make http HEAD request to get all headers including size and date
for example this command
curl -I -v https://repo1.maven.org/maven2/commons-io/commons-io/2.8.0/commons-io-2.8.0-javadoc.jar
prints:
....
< HTTP/1.1 200 OK
< Connection: keep-alive
< Content-Length: 1138955
< ETag: "f9962ba4adc0193d474f4006872713b7"
< Content-Type: application/java-archive
< Last-Modified: Sun, 06 Sep 2020 13:55:02 GMT
< X-Checksum-MD5: f9962ba4adc0193d474f4006872713b7
< X-Checksum-SHA1: 1754c67b623e5df978230fab077602b51a7e5b0e
< Via: 1.1 varnish, 1.1 varnish
< Accept-Ranges: bytes
< Date: Mon, 09 Nov 2020 19:00:20 GMT
< Age: 10256
< X-Served-By: cache-bwi5140-BWI, cache-bos4634-BOS
< X-Cache: HIT, HIT
< X-Cache-Hits: 1, 1
< X-Timer: S1604948421.582954,VS0,VE2
where you have headers: Content-Length - size and Last-Modified - date.
When I send a message with attachment using the Gmail API, the recipient receives the message without the attachment.
What is strange though is that:
1: in the sent folder of the sender, I do see the attachment properly
2: if I send to myself, both message are fine (in sent folder and in inbox folder)
3: if I use with GMail SMTP with the same raw message, it works fine
4: if I use a 3rd party SMTP with the same raw message, it works fine.
Point number 1+2 super puzzling.
Here is the source of the original message in the sent folder:
Received: from 13936824666 named unknown by gmailapi.google.com with HTTPREST; Wed, 25 Jan 2017 18:44:30 -0500
Date: Wed, 25 Jan 2017 18:44:30 -0500
From: Jeremy Chatelaine <source#gmail.com>
To: Jeremy <target#domain.com>
Message-Id: <CABX8Avad0vTtu8=jotRD5HM1r0My-ZKeV7RFRo0TmTxf5PNd0g#mail.gmail.com>
Subject: Export
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="--==_mimepart_5889385be5066_a133fd0f785e20837629"; charset=UTF-8
Content-Transfer-Encoding: 7bit
----==_mimepart_5889385be5066_a133fd0f785e20837629
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Please find attached the message
----==_mimepart_5889385be5066_a133fd0f785e20837629
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 7bit
Please find attached the message
----==_mimepart_5889385be5066_a133fd0f785e20837629
Content-Type: text/csv; charset=UTF-8; filename=export.csv
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=export.csv
Content-ID: <5889385be6fd7_a133fd0f785e20837735#jeremy.mail>
Some text...
----==_mimepart_5889385be5066_a133fd0f785e20837629--
Here is the source of the original message in the recipient inbox folder:
Delivered-To: target#domain.com
Received: by 10.55.110.193 with SMTP id j184csp1999707qkc;
Wed, 25 Jan 2017 16:30:42 -0800 (PST)
X-Received: by 10.107.34.213 with SMTP id i204mr540101ioi.203.1485390642385;
Wed, 25 Jan 2017 16:30:42 -0800 (PST)
Return-Path: <source#domain.com>
Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com. [2607:f8b0:4001:c0b::22b])
by mx.google.com with ESMTPS id 88si336719ioq.54.2017.01.25.16.30.42
for <target#domain.com>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Wed, 25 Jan 2017 16:30:42 -0800 (PST)
Received-SPF: pass (google.com: domain of source#domain.com designates 2607:f8b0:4001:c0b::22b as permitted sender) client-ip=2607:f8b0:4001:c0b::22b;
Authentication-Results: mx.google.com;
dkim=pass header.i=#domain.com;
spf=pass (google.com: domain of source#domain.com designates 2607:f8b0:4001:c0b::22b as permitted sender) smtp.mailfrom=source#domain.com
Received: by mail-it0-x22b.google.com with SMTP id 203so119411500ith.0
for <target#domain.com>; Wed, 25 Jan 2017 16:30:42 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=domain.com; s=domain;
h=from:mime-version:date:message-id:subject:to;
bh=wx26/V0bJk9VItDp3TAvKl28UAn7IRQq4NITJZDM+Co=;
b=L3KTzPTCoIJUfAacuJy+PE8jHnY9iwGuXUWSpZzneRs5bvMysigSMyPGn1YicyIvQ6
d/LvbEJPlsu+S0zElhIVPITjAmXKDKNIKwLQDHpkcKnnI3btBUrENN923fMtS1fDdHyV
3At0QenKrb34uQqYYoHtX2WU4nyYrISbYKL62=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:mime-version:date:message-id:subject:to;
bh=wx26/V0bJk9VItDp3TAvKl28UAn7IRQq4NITJZDM+Co=;
b=tGPSHjI3iRzjrI0jGHVX76uTw7OXYC4B2bP0qQKgQotBWru+Pn5Ci9A1Qop9oGN1Ys
fnxCgLOLG8ZJU165ODBNX1DGjPa8ud9SWg18FTsxIjNw9qTr1yJqbWr0LToJi7HdQUr8
7Aaiqil7PbPUf5SdxLCqwBNf660Rn9Sd/ADZeT1Bc2+iYQcizjiK/rOPPX+X1ZndvqxP
Ok3Ac2yyIWxi+m4xaPEztcF4JXFZDlJWFdclUDv4s5Jdc0eb1HmB5d2r1qroGLo5MTjd
d7jO1nRsKTO5I9I69p9AgC+LpDiWBxgzZMBVsU6vVpeZ03/pCroyk9DHDUAjn3ijtlFh
O2vw==
X-Gm-Message-State: AIkVDXJo2tPJlkHkthgEjnxYp7vz5Rfpn91pWCS7zEurkSiDhJtyzLpUoSDORq37K/7ATCRSyLAypKIYrdYwEiL2
X-Received: by 10.36.84.148 with SMTP id t142mr20263701ita.90.1485390641934; Wed, 25 Jan 2017 16:30:41 -0800 (PST)
Received: from 13936824667 named unknown by gmailapi.google.com with HTTPREST; Wed, 25 Jan 2017 19:30:41 -0500
From: Jeremy Chatelaine <source#domain.com>
Mime-Version: 1.0
Date: Wed, 25 Jan 2017 19:30:41 -0500
Message-ID: <CABK5xHonVstaJHWHfvBJFF1BK5Y0B8NJsAJTPFokPNAUcawhGA#mail.gmail.com>
Subject: Export
To: Jeremy <target#domain.com>
Content-Type: multipart/alternative; boundary=001a1143a6cc90c9040546f4753f
--001a1143a6cc90c9040546f4753f
Content-Type: text/plain; charset=UTF-8
Please find attached your requested export
--001a1143a6cc90c9040546f4753f
Content-Type: text/html; charset=UTF-8
Please find attached your requested export
--001a1143a6cc90c9040546f4753f--
As you can see, the mime part where I had my text attachment vanished.
Here is how the message is produced (cut for clarity)
msg = Mail.new
html_part = Mail::Part.new do
content_type 'text/html; charset=UTF-8'
body html_body
end
msg.html_part = html_part
new_text = plan_text
text_part = Mail::Part.new do
body new_text
end
msg.text_part = text_part
file_paths.each do |file_path|
msg.add_file(file_path)
# I also tried like that, same result
#open(file_path) do |file|
# msg.attachments[file_path] = file.read
#end
end
raw_message = msg.to_s
Here is how I send with the gmail api
client = Google::APIClient.new(:application_name => "app", :application_version => "1")
client.authorization.client_id = "someverylongnumbers.apps.googleusercontent.com"
client.authorization.client_secret = "morerandomletters"
client.authorization.access_token = token
client.authorization.scope = [
"https://www.googleapis.com/auth/gmail.modify"
]
gmail_api = client.discovered_api('gmail', 'v1') # https://www.googleapis.com/auth/gmail.modify
result = client.execute(
:api_method => gmail_api.users.messages.to_h['gmail.users.messages.send'],
:parameters => {
'userId' => "me"
},
:body_object => {
'raw' => Base64.urlsafe_encode64(raw_message)
}
)
What is wrong with this?
Like the title says, nothing shows up in real time events nor in any other event
The request being sent is done with ruby's faraday gem
require 'faraday'
require 'json'
GOOGLE_ANALYTICS_SETTINGS = {}
class GoogleAnalyticsApi
def event(client_id = '555', category, action, label, value, user_agent)
return unless !GOOGLE_ANALYTICS_SETTINGS["tracking_code"].empty?
params = {
v: GOOGLE_ANALYTICS_SETTINGS["version"],
tid: GOOGLE_ANALYTICS_SETTINGS["tracking_code"],
cid: client_id,
t: "event",
ec: category,
ea: action,
el: label,
ev: value
}
begin
puts params
c = Faraday.new() do |f|
f.request :url_encoded
f.response :logger
f.adapter Faraday.default_adapter
end
response = c.post GOOGLE_ANALYTICS_SETTINGS["endpoint"], params, { "User-Agent" => user_agent || "Subscribing Worker theSkimm" }
puts response.headers
puts response.body
return true
rescue Exception => rex
return false
end
end
end
From the console:
{:v=>1, :tid=>"UA-XXXXXXXX-1", :cid=>"1999999999.1389999972", :t=>"event", :ec=>"test-event", :ea=>"test-action", :el=>nil, :ev=>nil}
I, [2014-01-15T18:04:04.555555 #7666] INFO -- : post http://www.google-analytics.com/collect
D, [2014-01-15T18:04:04.555667 #7666] DEBUG -- request: User-Agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
Content-Type: "application/x-www-form-urlencoded"
I, [2014-01-15T18:04:04.670273 #7666] INFO -- Status: 200
D, [2014-01-15T18:04:04.670512 #7666] DEBUG -- response: pragma: "no-cache"
expires: "Mon, 07 Aug 1995 23:30:00 GMT"
cache-control: "private, no-cache, no-cache=Set-Cookie, proxy-revalidate"
access-control-allow-origin: "*"
last-modified: "Sun, 17 May 1998 03:00:00 GMT"
x-content-type-options: "nosniff"
content-type: "image/gif"
date: "Wed, 15 Jan 2014 23:04:04 GMT"
server: "Golfe2"
content-length: "35"
alternate-protocol: "80:quic"
connection: "close"
{"pragma"=>"no-cache", "expires"=>"Mon, 07 Aug 1995 23:30:00 GMT", "cache-control"=>"private, no-cache, no-cache=Set-Cookie, proxy-revalidate", "access-control-allow-origin"=>"*", "last-modified"=>"Sun, 17 May 1998 03:00:00 GMT", "x-content-type-options"=>"nosniff", "content-type"=>"image/gif", "date"=>"Wed, 15 Jan 2014 23:04:04 GMT", "server"=>"Golfe2", "content-length"=>"35", "alternate-protocol"=>"80:quic", "connection"=>"close"}
GIF89a�����,D;
For whomever encounters this the problem was setting nil value for el and ev. When the params hash is moved through Faraday the nil valued items get converted to the key name without any value and google didn't like that.
IE a query string with nil values going through Faraday would be:
...&el&ev
Not good for google's Measurement Protocol.
body = "<CreateBucketConfiguration><LocationConstraint>EU</LocationConstraint></CreateBucketConfiguration>"
content_length = body.bytesize
content_type = "text/plain"
url = URI.parse("http://#{#name}.s3.amazonaws.com/")
req = Net::HTTP::Put.new(url.path)
req.body = body
req.add_field 'Date' , #time
req.add_field 'Host', "#{#name}.s3.amazonaws.com"
req.add_field 'Content-Type', "#{content_type}"
req.add_field 'Authorization', "#{signature}"
req.add_field 'Content-Length', "#{content_length}"
response = Net::HTTP.new(url.host, url.port).start do |http|
http.request(req)
end
puts response.read_body
returns 200 and creates bucket but in U.S Standard and not in EU. What am I missing here? Thanks for the help.
Here is the entire conversation
PUT / HTTP/1.1
Accept: */*
User-Agent: Ruby
Date: Wed, 19 Jan 2011 22:14:31 -0800
Host: mytest.s3.amazonaws.com
Content-Type: text/plain
Authorization: AWS AC8RVKAXAU8Q:41uTqvfncc2mE561YabgpGUouio=
Content-Length: 146
<CreateBucketConfiguration xmlns='http://s3.amazonaws.com/doc/2006-03-01/'>
<LocationConstraint>EU</LocationConstraint>
</CreateBucketConfiguration>
HTTP/1.1 200 OK
x-amz-id-2: lrlPt8Y19ZxFXPbZf9Gf6dYxTGLYkkMzo0tSNXCNk29o9xghcob502mcttQ/oo4W
x-amz-request-id: 3504CCA0E7AFFE95
Date: Thu, 20 Jan 2011 06:14:32 GMT
Location: /mytest
Content-Length: 0
Server: AmazonS3
HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Date: Thu, 20 Jan 2011 06:14:32 GMT
Connection: close
Server: AmazonS3
0
The only thing that I can see is that you haven't included the xmlns in the request body - not sure if that will make any difference though.
xmlns="http://s3.amazonaws.com/doc/2006-03-01/"
body = "<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LocationConstraint>EU</LocationConstraint></CreateBucketConfiguration>"