I kept running into a wall trying to get a ruby file to work for a filter and I finally simplified my config file to simply this:
filter {
ruby {
code => "event.cancel"
}
}
But even with this simple example, running logstash.bat -t gives me the following:
The given configuration is invalid. Reason: uninitialized constant LogStashCompilerLSCLGrammar::LogStash::Compiler::LSCL::AST::Hash::ConfigurationError
Can anyone point me in the right direction to get a ruby filter to work at all? Same thing happening with fresh install of 6.2.2.
Well I should have gone further and simplified even more. I still received the error even after removing then entire ruby { } section which meant my issue was elsewhere. Sorry to waste viewing time. I found my issue with another filters add_field not working correctly. Lesson learned: Make one change at a time and test...
Related
I'm learning how to use logstash and I'm facing some problems in reading a file with logstash which is constantly updated. Here is my test:
logstash.conf
input {
file {
path => ["/usr/share/logs_data/first_log_test.log"]
start_position => "beginning"
}
}
filter {
grok {
match => ["message", "(?<execution_date>\d{4}-\d{2}-\d{2}) (?<execution_time>\d{2}:\d{2}:\d{2})%{GREEDYDATA}ParaBrutos/configs/pipelines/(?<crawler_category>([^/])+)/(?<crawler_subcategory>([^-])+)-(?<crawler_name>([^.])+).json"]
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
user => "elastic"
password => "changeme"
ecs_compatibility => disabled
index => "logs_second"
}
}
First, I use this repo's code, for installing ELK stack in a dockerized way.
I started with this test log file empty, and then added few lines that maches the pattern, one by one, in a text editor, simultaneously seeing the index patterns in kibana updating. However, each new row I add in this test log is not added alone, and I see hits of old entries in kibana index patterns.
What happens?
Could it be anything related to not selecting time filters in index pattern creation?
Or something related to since_db settings? But what? Because by default, isn't it supposed to save the last read position of the file?
Or something related to start_position? Even though, by the plugin documentation, it is supposed to make effect only when the first read of the file is done?
I'm a bit lost, tried lots of things and still am not understanding well what's happening. Could you help me?
If you are using a text editor then you are probably creating a new file each time you exit it.
That could be an inode reuse issue. There are links to various issues in the META issue 211. Especially see 251.
Tracking which files have been read when those files can get rotated is an extremely hard problem. Way harder than most folks would initially think. A good option to get it right is to checksum the file contents (although this is not foolproof). The file input does not do that, because it can get ridiculously expensive. Instead it implements a very cheap technique that almost always gets it right (but in a few cases it decides it has already read a file that it has not read).
There are other cases where it gets it wrong by duplicating data (which is what you are hitting). As I said, it is a really hard problem.
I've been doing a Ruby course on Skillcrush (still very much an amateur) and have come across a part of the course where my code just doesn't work.
The app uses Sinatra, and is supposed to show the views/people/index.erb when going to localhost:9292/people, but instead it goes to the error page which it should when the wrong extension is given after localhost:9292/ (normally a date format, but if anything else is entered it should bring an error).
I had to switch computers half way through the course, so have a feeling it may be to do with my setup. I've used the code that they've supplied and have checked for discrepancies using diff --brief -r dir1/ dir2/ and can only see some in my Gemfile.lock file. I'm using Ruby 2.4 due to issues with gems on pre-2.0 Ruby and wondered if this might be the case?
My code can be seen here.
Can anyone see any glaring issues?
I believe what is happening is that Sinatra is pattern matching your url localhost:9292/people to the first route of your index controller get '/:birthdate' instead of get '/people'. Sinatra takes the request and then checks each of the routes in order, the first one to match then handles the request.
To test this:
try changing get '/:birthdate' to get '/birthdate/:birthdate' (if it works you would then have to change any links to birthdate appropriately).
or
comment out the birthdate route
or
move all the routes into the same file and change the order they are arranged in to get a feel for how the pattern matching is occurring.
I am trying to print rss/atom feed updates using ruby. Feedjira seems to be the best bet for this. Unfortunately the update feature does not seem to work properly. I get duplicate entries.
Here is a simple example that produces the problem:
require 'feedjira'
require 'pp'
feed = Feedjira::Feed.fetch_and_parse "http://lorem-rss.herokuapp.com/feed?unit=second&interval=10"
loop do
feed = Feedjira::Feed.update(feed)
pp feed.new_entries
sleep 20
end
Any suggestions? Maybe other libraries? Or am I missing something important when using Feedjira?
There a several questions around this topic for Feedzirra the former name for Feedjira, but the update feature seems to be a new feature: http://feedjira.com/updating-feeds.html
The updated functionality was removed from feedjira due to serious problems. See
(commit) https://github.com/feedjira/feedjira/commit/6f56516934a9bdb8691f2bbe98be0f2b7c25b7ea
(discussion) https://github.com/feedjira/feedjira/issues/218
I'm trying to fix the Magento search issue where 'OR' is used for comparison of multiple search terms instead of 'AND'.
I've seen quite a number of suggestions on the web regarding how to fix this, and the general idea is as follows:
Copy
app/code/core/Mage/CatalogSearch/Model/resource/Fulltext.php to
app/code/local/Mage/CatalogSearch/Model/resource/Fulltext.php and in the copy, change the instances of 'OR' to 'AND', where the SQL queries are built.
However, my changes don't seem to work as expected and what is even more confusing is that the prepareResult() method (which is where the SQL changes above are made) doesn't even seem to run at all when searches are done. I've tested this by putting some debugging code in the function.
I've used the same debug code to verify that the file gets loaded. But the debug code doesn't run when inserted at the prepareResult() function. (The debug code basically writes to a file on disk).
What am I missing here?
Kindly note that this is not so much about the right way to accomplish the end goal of fixing Magento search. I'm aware it's best done via an extension. I'd just like to figure out why prepareResult() isn't being called as expected.
I would like to get a current date time with erlang.
I have tried using the code below;
{{Year,Month,Day},{Hour,Min,Sec}} = erlang:localtime().
But sometimes it got an error like '** exception error: no match of right hand side value {{2012,5,6},{23,40,58}}'
Looks like there is a problem with 1 digit. I try searching couple of webs but still cannot find a way to cope this.
I believe this is quite an easy one but as a erlang newbie, I cannot resolve this. I try my best.
Env:
Erlang {"OTP APN 181 01","R15B01"} installed with windows binary version
Windows XP
Thanks in advance,
No, there is no problem with 1 digit. It should match perfectly well (check with {{Year,Month,Day},{Hour,Min,Sec}} = {{2012,5,6},{23,40,58}}). Most likely you have already assigned one of the variables to something earlier in the function.