Ruby code in logstash failing with _rubyexception - ruby

I would like to get help with the code in ruby filter.
logstash version is 5.0.0 alpha4.
I am testing the code in ruby filter as below but I am getting _rubyexception.
ruby {
code => "
event['newfield'] = 'test'
"
}
ruby filter is defined inside filter { }.
The logstash.log shows as below,
:timestamp=>"2016-08-03T15:26:47.291000+0900", :message=>"Ruby exception occurred: undefined method[]=' for 2016-08-03T06:26:46.829Z test %{message}:LogStash::Event", :level=>:error}
I cant find the reason why ruby filter is unable to use event object.
I appreciate if I could get some help to cope this issue.
Thanks,
Yu

There is a new event API in Logstash 5.0.0 and now you need to set fields on the event as follows:
ruby {
code => "
event.set('newfield', 'test')
"
}

For get the value from field name:
ruby {
code => "
value = (event.get('field_name'))
"
}

Related

Why do I get the error fromFile not a built-in function in botframework

I have upgraded to the latest version of botframework and started to get the error:
Oops, it looks like something went wrong. Error:[Error] D:\home\site\wwwroot\Dialogs\RootDialog\LG\en\RootDialog_en.lg line 15:2 - line 15:69: Error occurred when parsing expression ‘json(fromFile(’…/…/Cards/en/OnChooseIntentAdaptiveCard.json’))'. fromFile does not have an evaluator, it’s not a built-in function or a custom function.
LG code:
# AdaptiveCard
[Activity
Attachments = ${json(AdaptiveCard.Definition())}
]
# AdaptiveCard.Definition
- ${json(fromFile('../../Cards/en/OnChooseIntentAdaptiveCard.json'))}
# chooseIntentResponseWithCard_en()
[Activity
Attachments = ${ActivityAttachment(json(fromFile('../../Cards/en/OnChooseIntentAdaptiveCard.json')), 'adaptiveCard')}
]
# Greeting_en()
[Activity
Attachments = ${ActivityAttachment(json(fromFile('../../Cards/en/GreetingCard.json')), 'adaptiveCard')}
AttachmentLayout = list
]
What am I missing?
fromFile is now disabled by default. You can change set the value on the Microsoft.Bot.Builder.LanguageGeneration.Templates class:
Templates.EnableFromFile = true;

Can a logstash filter error be forwarded to elastic?

I'm having these json parsing errors from time to time:
2022-01-07T12:15:19,872][WARN ][logstash.filters.json ] Error parsing json
{:source=>"message", :raw=>" { the invalid json }", :exception=>#<LogStash::Json::ParserError: Unrecognized character escape 'x' (code 120)
Is there a way to get the :exception field in the logstash config file?
I opened the exact same thread on the elastic forum and got a working solution there. Thanks to #Badger on the forum, I ended up using the following raw ruby filter:
ruby {
code => '
#source = "message"
source = event.get(#source)
return unless source
begin
parsed = LogStash::Json.load(source)
rescue => e
event.set("jsonException", e.to_s)
return
end
#target = "jsonData"
if #target
event.set(#target, parsed)
end
'
}
which extracts the info I needed:
"jsonException" => "Unexpected character (',' (code 44)): was expecting a colon to separate field name and value\n at [Source: (byte[])\"{ \"baz\", \"oh!\" }\r\"; line: 1, column: 9]",
Or as the author of the solution suggested, get rid of the #target part and use the normal json filter for the rest of the data.

How to access JSON from external data source in Terraform?

I am receiving JSON from a http terraform data source
data "http" "example" {
url = "${var.cloudwatch_endpoint}/api/v0/components"
# Optional request headers
request_headers {
"Accept" = "application/json"
"X-Api-Key" = "${var.api_key}"
}
}
It outputs the following.
http = [{"componentID":"k8QEbeuHdDnU","name":"Jenkins","description":"","status":"Partial Outage","order":1553796836},{"componentID":"ui","name":"ui","description":"","status":"Operational","order":1554483781},{"componentID":"auth","name":"auth","description":"","status":"Operational","order":1554483781},{"componentID":"elig","name":"elig","description":"","status":"Operational","order":1554483781},{"componentID":"kong","name":"kong","description":"","status":"Operational","order":1554483781}]
which is a string in terraform. In order to convert this string into JSON I pass it to an external data source which is a simple ruby function. Here is the terraform to pass it.
data "external" "component_ids" {
program = ["ruby", "./fetchComponent.rb",]
query = {
data = "${data.http.example.body}"
}
}
Here is the ruby function
#!/usr/bin/env ruby
require 'json'
data = JSON.parse(STDIN.read)
results = data.to_json
STDOUT.write results
All of this works. The external data outputs the following (It appears the same as the http output) but according to terraform docs this should be a map
external1 = {
data = [{"componentID":"k8QEbeuHdDnU","name":"Jenkins","description":"","status":"Partial Outage","order":1553796836},{"componentID":"ui","name":"ui","description":"","status":"Operational","order":1554483781},{"componentID":"auth","name":"auth","description":"","status":"Operational","order":1554483781},{"componentID":"elig","name":"elig","description":"","status":"Operational","order":1554483781},{"componentID":"kong","name":"kong","description":"","status":"Operational","order":1554483781}]
}
I was expecting that I could now access data inside of the external data source. I am unable.
Ultimately what I want to do is create a list of the componentID variables which are located within the external data source.
Some things I have tried
* output.external: key "0" does not exist in map data.external.component_ids.result in:
${data.external.component_ids.result[0]}
* output.external: At column 3, line 1: element: argument 1 should be type list, got type string in:
${element(data.external.component_ids.result["componentID"],0)}
* output.external: key "componentID" does not exist in map data.external.component_ids.result in:
${data.external.component_ids.result["componentID"]}
ternal: lookup: lookup failed to find 'componentID' in:
${lookup(data.external.component_ids.*.result[0], "componentID")}
I appreciate the help.
can't test with the variable cloudwatch_endpoint, so I have to think about the solution.
Terraform can't decode json directly before 0.11.x. But there is a workaround to work on nested lists.
Your ruby need be adjusted to make output as variable http below, then you should be fine to get what you need.
$ cat main.tf
variable "http" {
type = "list"
default = [{componentID = "k8QEbeuHdDnU", name = "Jenkins"}]
}
output "http" {
value = "${lookup(var.http[0], "componentID")}"
}
$ terraform apply
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
http = k8QEbeuHdDnU

Transloadit: unable to generate the right signature with ruby

I tried the example at https://transloadit.com/docs/api-docs#auth-example.
p = JSON.generate({ auth: { expires: "2010/10/19 09:01:20+00:00", key: "2b0c45611f6440dfb64611e872ec3211"}, steps: { encode: { robot: "/video/encode" } } })
signature = OpenSSL::HMAC.hexdigest(OpenSSL::Digest.new('sha1'), "d805593620e689465d7da6b8caf2ac7384fdb7e9", p)
The result is 00320965b86d42b6d983d1fad3f126ee7385b962
But according to the documentation it should be 6032b7b62879829941b84df5cfaebfe4644ab4e3
The Ruby code is borrowed from https://transloadit.com/docs/api-docs#authentication-implementations
and seems to be the same at https://github.com/transloadit/ruby-sdk/blob/master/lib/transloadit/request.rb#L200
PS: Since I only need the signature logic I do not want to use the Transloadit Ruby or Rails SDK.
I use Ruby 2.1.1 and Rails 4.1
So there was an issue in the Transloadit documentation that is now fixed and I messed up with the parameters order.
For more information see https://github.com/transloadit/ruby-sdk/issues/21

Puppet syntax error at '|'

I seem to have run into a syntax error on a previously-working puppet manifest. This is running on a local vagrant box with Ubuntu 12.04, and Puppet version 3.4.2. The puppet stuff was all generated at puphpet.com.
The error I'm getting is:
Error: Could not parse for environment production: Syntax error at '|'
at /tmp/vagrant-puppet/manifests/default.pp:263:29 on node
vagrant.example.com
Line 263 of default.pp is the second line of this snippet:
if count($php_values['ini']) > 0 {
$php_values['ini'].each { |$key, $value|
puphpet::ini { $key:
entry => "CUSTOM/${key}",
value => $value,
php_version => $php_values['version'],
webserver => $php_webserver_service
}
}
}
It looks like you haven't set parser to future.
Run this command:
puppet config print parser
If it returns current, you don't have access to the .each function. To change this, edit /etc/puppet/puppet.conf, and put parser = future under the [main] block. The above command should then return future.
Reference: http://docs.puppetlabs.com/references/latest/function.html#each

Resources