I am using logstash 2.1.3,I want to show the #timestamp field in my timezone, like "#timestamp" => "2015-03-09T04:24:29.718+08:00", I try to using ruby conversion in filter.
filter
{
ruby {
code => "event['#timestamp'] =
event['#timestamp'].localtime('+08:00')"
}
}
but i got the error msg as below,
Ruby exception occurred: undefined method `localtime' for "2016-04-29T00:40:51.926Z":LogStash::Timestamp {:level=>:error}
Any help is welcome to resolve this.
It looks like you're trying to call localtime on a String (the result of event['#timestamp'], and it should instead be called on a Time object. You'll need to parse that String into a Time object.
Check out this irb session:
> require 'date'
true
> fmt = '%Y-%m-%dT%H:%M:%S.%L%z'
"%Y-%m-%dT%H:%M:%S.%L%z"
> s = "2016-04-29T00:40:51.926Z"
"2016-04-29T00:40:51.926Z"
> localtime_as_string = DateTime.strptime(s,fmt).to_time.localtime('+08:00').strftime(fmt)
"2016-04-29T08:40:51.926+0800"
In your case, you could use:
require 'date'
fmt = '%Y-%m-%dT%H:%M:%S.%L%z'
s = event['#timestamp'].to_s
tz = '+08:00'
event['#timestamp'] = DateTime.strptime(s, fmt).to_time.localtime(tz).strftime(fmt)
On further examination, it looks from the output like event['#timestamp'] contains an instance of Logstash::Timestamp. I looked at that API (https://github.com/elastic/logstash/blob/master/logstash-core-event/lib/logstash/timestamp.rb) and it looks like there is a publicly accessible time method that I presume returns an instance of Time. If that is the case, then the above will not work but this may:
require 'date'
# For fmt you may also be able to use Logstash::Timestamp::ISO8601_STRFTIME
fmt = '%Y-%m-%dT%H:%M:%S.%L%z'
time = event['#timestamp'].time
tz = '+08:00'
event['#timestamp'] = time.localtime(tz).strftime(fmt)
However, are you sure you want to replace event['#timestamp']? Another approach would be to use it for display but not modify the original variable. This might be safer because if the variable is used elsewhere it will not be known which of the 2 it is.
Related
Printing the contents of a variable gives me a bunch of data.
I want to access part of that data, but get an error.
I'm using Viewpoint::EWS and am successfully accessing the data I need.
calendaritems = folder.find_items({:folder_id => folder.folder_id, :calendar_view => {:start_date => sd.rfc3339(), :end_date => ed.rfc3339()}})
calendaritems.each do |event|
...
end
Printing the variable "event", I can see the data I need: "date_time_stamp" (or "appointment_reply_time").
#<Viewpoint::EWS::Types::CalendarItem:0x00005652b332dfa0
#ews_item=
:date_time_stamp=>{:text=>"2019-03-18T12:01:49Z"},
:appointment_reply_time=>{:text=>"2019-03-18T13:01:55+01:00"},
However, trying to access using "event.date_time_stamp" (or "event.appointment_reply_time") leads to the error
undefined method `date_time_stamp' for <Viewpoint::EWS::Types::CalendarItem:0x00005622f83c3d38> (NoMethodError)
Here's the code:
calendaritems = folder.find_items({:folder_id => folder.folder_id, :calendar_view => {:start_date => sd.rfc3339(), :end_date => ed.rfc3339()}})
calendaritems.each do |event|
if event.recurring?
puts "#{event.date_time_stamp} | #{(event.start-event.date_time_stamp).to_i} | #{event.organizer.email_address}"
if (event.start-event.date_time_stamp).to_i == reminderDays
executeSomething()
end
end
end
I'm looking through recurring appointments for a resource within a week. Since those will be silently dropped after a year, the plan is to set up a system to remind people that this will happen, so they can rebook the resource.
At first I tried using the creation date of the appointment (event.date_time_created), which works as expected, but then noticed, that people can update their appointments, thus resetting the 1 year timer.
That's why I also need the date of the last update.
The debug output you supplied says that event variable has an attribute "ews_item" and then it has a hash with an attribute "date_time_stamp", so try event.ews_item[:date_time_stamp]
I want to create a ruby script which will take barcodes from a text file, search a webservice for that barcode and download the result.
First I tried to test the webservice download. In a file when I hardcode the query things work fine:
result_download = open('http://webservice.org/api/?query=barcode:78686112327', 'User-Agent' => 'UserAgent email#gmail.com').read
It all works fine.
When I try to take the barcode from a textfile and run the query I run into problems.
IO.foreach(filename) {|barcode| barcode
website = "'http://webservice.org/api/?query=barcode:"+barcode.to_str.chomp + "', 'User-Agent' => 'UserAgent email#gmail.com'"
website = website.to_s
mb_metadata = open(website).read
}
The result of this is:
/home/user/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:37:in `initialize': No such file or directory # rb_sysopen - http://webservice.org/api/?query=barcode:78686112327', 'User-Agent' => 'UserAgent email#gmail.com' (Errno::ENOENT)
I can't figure out if this problem occurs because the string I generate somehow isn't a valid url and ruby is trying to open a non-existent file, or is the issue that I am doing all this in a for loop and the file/url doesn't exist there. I have tried using open(website).write instead of open(website).read but that produces the same error.
Any help would be much appreciated.
The error message you get explicitly states, that there is no such file:
http://webservice.org/api/?query=barcode:78686112327', 'User-Agent' => 'UserAgent email#gmail.com'.
You try to pass all the parameters to open method using 1 big string (website), which is wrong. You should do it like that.
IO.foreach(filename) do |barcode|
website = "http://webservice.org/api/?query=barcode:#{barcode.to_str.chomp}"
mb_metadata = open(website, 'User-Agent' => 'UserAgent email#gmail.com').read
end
Been messing around with Kickbox's api for email verification. I'm trying to have the program only display the result object in the returned JSON.
Here's the code:
require "kickbox"
require 'httparty'
require 'json'
client = Kickbox::Client.new('ac748asdfwef2fbf0e8177786233a6906cd3dcaa')
kickbox = client.kickbox()
response = kickbox.verify("test#easdfwf.com")
file = File.read(response)
json = JSON.parse(file)
json['result']
I'm getting an error verify.rb:10:in read': no implicit conversion of Kickbox::HttpClient::Response into String (TypeError)
from verify.rb:10:in'
Here's a sample response:
{
"result":"undeliverable",
"reason":"rejected_email",
"role":false,
"free":false,
"disposable":false,
"accept_all":false,
"did_you_mean":"bill.lumbergh#gmail.com",
"sendex":0,
"email":"bill.lumbergh#gamil.com",
"user":"bill.lumbergh",
"domain":"gamil.com",
"success":true,
"message":null
}
You are getting this error:
read': no implicit conversion of Kickbox::HttpClient::Response into String (TypeError)
Because, in this line:
file = File.read(response)
Your response is a Kickbox::HttpClient::Response type object, but the File.read is expecting a String object instead (possibly a file name with path).
I'm not sure what you are trying to do, but this: file = File.read(response) is wrong. You can't do this and that's why you are getting the mentioned error.
If you really want to use file, then you can write the response to a file and then read the response back from the file and use that:
f = File.new('response.txt', 'w+') # creating a file in read/write mode
f.write(response) # writing the response into that file
file_content = File.read('response.txt') # reading the response back from the file
So, the issue is not about Accessing a 3rd party API JSON object in ruby, but you are trying to use File.read in a wrong way.
You can get the response from the API by doing this:
client = Kickbox::Client.new('YOUR_API_KEY')
kickbox = client.kickbox()
response = kickbox.verify("test#easdfwf.com")
Then, you can play with the response e.g. can do a puts response.inspect or puts response.body.inspect and see what's inside that object.
And, from there you can extract your required outputs only.
How do you get the full path of the URL given the following
uri = URI("http://foo.com/posts?id=30&limit=5#time=1305298413")
I just want posts?id=30&limit=5#time=1305298413
I tried uri.path and that returns /posts and ui.query returns 'id=30&limit=5'
The method you are looking for is request_uri
uri.request_uri
=> "/posts?id=30&limit=5"
You can use any method you'd like to remove the leading / if needed.
Edit: To get the part after the # sign, use fragment:
[uri.request_uri, uri.fragment].join("#")
=> "/posts?id=30&limit=5#time=1305298413"
You can ask the URI object for its path, query, and fragment like this:
"#{uri.path}?#{uri.query}##{uri.fragment}"
# => "/posts?id=30&limit=5#time=1305298413"
or (a little more consice, but less explicit):
"#{uri.request_uri}##{uri.fragment}"
# => "/posts?id=30&limit=5#time=1305298413"
A bit of an artsy solution, for completeness – others may be more reasonable :)
uri.dup.tap { _1.scheme = _1.user = _1.password = _1.host = _1.port = nil }.to_s
Though a benefit of this solution is that it also handles URI::Generic like you'd get from parsing a standalone path:
uri = URI.parse("/foo?bar#baz") # => #<URI::Generic /foo?bar#baz>
Solutions that rely on URI::HTTP#request_uri won't work with URI::Generic. So if you're writing code that accepts both full URLs and paths, that may be a factor.
File.basename("http://foo.com/posts?id=30&limit=5#time=1305298413")
# => "posts?id=30&limit=5#time=1305298413"
I'm green when it comes to Ruby. Right now I'm mucking about with a script which connects to the Terremark eCloud API Explorer. I'm trying to use the httpclient gem, but I'm a bit confused as to how I'm supposed to construct my client.
#!/usr/bin/ruby
require "httpclient"
require 'base64'
require 'hmac-sha1'
require 'openssl'
# Method definitions
def get_date
# Get the time and date in the necessary format
result = Time.now.strftime('%a, %d %b %Y %H:%M:%S GMT')
end
def get_signature(action,date,headers,resource,user,pass)
string_to_sign = "#{action}
#{date}
#{headers}
#{resource}\n"
return Base64.encode64(OpenSSL::HMAC.digest('sha1', "#{user}:#{pass}", "#{string_to_sign}"))
end
# Initial variables
date = get_date
domain = "https://services.enterprisecloud.terremark.com"
password = 'password'
query = {}
tmrk_headers = Hash.new
tmrk_headers['x-tmrk-date: '] = date
tmrk_headers['x-tmrk-version: '] = '2013-06-01'
uri = '/cloudapi/spec/networks/environments/1'
url = "#{domain}#{uri}"
username = 'user#terremark.com'
verb = 'GET'
signature = get_signature(verb,date,tmrk_headers,uri,username,password)
tmrk_headers['Authorization: '] = "Basic \"#{signature}\""
puts signature
client = HTTPClient.new
client.get_content(url,query,tmrk_headers)
EDIT: This is no longer valid as I've moved beyond this error with some help:
Right now I'm not concerned about seeing what is returned from the connection. I'm just looking to create an error-free run. For instance, if I run the script without the client.get_content line it will return to a prompt without issue (giving me the impression that everything ran cleanly, if not uselessly).
How am I supposed to construct this? The httpclient documentation uses the example with external headers:
extheader = [['Accept', 'image/jpeg'], ['Accept', 'image/png']]
clnt.get_content(uri, query, extheader)
I'm making the assumption that the query is the URI that I've defined.
In all reality, it isn't set up right in the first place. I need to be able to include the string in the auth_header variable in the string to be signed but the signature is actually part of the variable. I've obviously created a hole in that regard.
Any assistance with this will be more than appreciated.
EDIT2: Removed strace pastebin. Adding Ruby backtrace:
/home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1023:in `create_request': undefined method `each' for #<String:0x0000000207d1e8> (NoMethodError)
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:884:in `do_request'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:959:in `follow_redirect'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:594:in `get_content'
from ./test.rb:42:in `<main>'
EDIT3: Updated script; adding further backtrace after making necessary script modifications:
/
home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:975:in `success_content': unexpected response: #<HTTP::Message::Headers:0x00000001dddc58 #http_version="1.1", #body_size=0, #chunked=false, #request_method="GET", #request_uri=#<URI::HTTPS:0x00000001ddecc0 URL:https://services.enterprisecloud.terremark.com/cloudapi/spec/networks/environments/1>, #request_query={}, #request_absolute_uri=nil, #status_code=400, #reason_phrase="Bad Request", #body_type=nil, #body_charset=nil, #body_date=nil, #body_encoding=#<Encoding:US-ASCII>, #is_request=false, #header_item=[["Content-Type", "text/html; charset=us-ascii"], ["Server", "Microsoft-HTTPAPI/2.0"], ["Date", "Thu, 27 Mar 2014 23:12:53 GMT"], ["Connection", "close"], ["Content-Length", "339"]], #dumped=false> (HTTPClient::BadResponseError)
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:594:in `get_content'
from ./test.rb:52:in `<main>'
The issue that you're having as stated by your backtrace
/home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1023:in `create_request': undefined method `each' for #<String:0x0000000207d1e8> (NoMethodError)
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:884:in `do_request'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:959:in `follow_redirect'
from /home/msnyder/.rvm/gems/ruby-2.1.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:594:in `get_content'
from ./test.rb:42:in `<main>'
is that it seems like you're passing a String object to one of the arguments in get_content where it expects an object that responds to the method each.
From looking at the documentation of httpclient#get_content http://www.ruby-doc.org/gems/docs/h/httpclient-xaop-2.1.6/HTTPClient.html#method-i-get_content
It expects the second parameter to be a Hash or Array of arguments
From your code sample and showing only the relevant parts
uri = '/cloudapi/spec/networks/environments/1'
url = "https://services.enterprisecloud.terremark.com"
tmrk_headers = "x-tmrk-date:\"#{date}\"\nx-tmrk-version:2014-01-01"
auth_header = "Authorization: CloudApi AccessKey=\"#{access_key}\" SignatureType=\"HmacSHA1\" Signature=\"#{signature}\""
full_header = "#{tmrk_headers}\n#{auth_header}"
client = HTTPClient.new
client.get_content(url,uri,full_header)
There are two things that I see wrong with your code.
You're passing in a String value for the query. Specifically, you're passing in uri which has a value of what I'm assuming is the path that you want to hit.
For the extra headers parameter, you're passing in a String value which is in the full_header
What you need to do in order to fix this is pass in the full url for the first parameter.
This means it should look something like this:
url = "https://services.enterprisecloud.terremark.com/cloudapi/spec/networks/environments/1"
query = {} # if you have any parameters to pass in they should be here.
headers = {
"x-tmrk-date" => date, "x-tmrk-version" => "2014-01-01",
"Authorization" => "CloudApi AccessKey=#{access_key} SignatureType=HmacSHA1 Signature=#{signature}"
}
client = HTTPClient.new
client.get_content(url, query, headers)