logstash 5 ruby filter - ruby

I've recently upgraded an elk-stack cluster from an old version to 5.1 and although everything looks great, I have an exception occurring frequently in the logstash log, which looks like this:
logstash.filters.ruby Ruby exception occurred: Direct event field references (i.e. event['field']) have been disabled in favor of using event get and set methods (e.g. event.get('field')). Please consult the Logstash 5.0 breaking changes documentation for more details.
The filter I have looks like this:
filter {
ruby {
init => "require 'time'"
code => "event.cancel if event['#timestamp'] < Time.now-(4*86400)"
}
}
Any suggestions ?

The exception contains the answer:
Direct event field references (i.e. event['field']) have been disabled in favor of using event get and set methods (e.g. event.get('field')).
From that, it seems like event.get('#timestamp') is now preferred over event['#timestamp'].

Related

How to Fix Document Not Found errors with find

I have a collection of Person, stored in a legacy mongodb server (2.4) and accessed with the mongoid gem via the ruby mongodb driver.
If I perform a
Person.where(email: 'some.existing.email#server.tld').first
I get a result (let's assume I store the id in a variable called "the_very_same_id_obtained_above")
If I perform a
Person.find(the_very_same_id_obtained_above)
I got a
Mongoid::Errors::DocumentNotFound
exception
If I use the javascript syntax to perform the query, the result is found
Person.where("this._id == #{the_very_same_id_obtained_above}").first # this works!
I'm currently trying to migrate the data to a newever version. Currently mongodbrestore-ing on amazon documentdb to make tests (mongodb 3.6 compatible) and the issue remains.
One thing I noticed is that those object ids are peculiar:
5ce24b1169902e72c9739ff6 this works anyway
59de48f53137ec054b000004 this requires the trick
The small number of zeroes toward the end of the id seems to be highly correlated with the problem (I have no idea of the reason).
That's the default:
# Raise an error when performing a #find and the document is not found.
# (default: true)
raise_not_found_error: true
Source: https://docs.mongodb.com/mongoid/current/tutorials/mongoid-configuration/#anatomy-of-a-mongoid-config
If this doesn't answer your question, it's very likely the find method is overridden somewhere in your code!

Application.Current.Properties - System.AggregateException

I'm trying to get some data from Application.Current.Properties storage. Unfortunately, any time I want to use this Dictionary, I see this error:
An exception of type 'System.AggregateException' occurred in mscorlib.ni.dll but was not handled in user code
Additional information: One or more errors occurred.
And in details I found this:
{"Error in line 1 position 206. Element 'http://schemas.microsoft.com/2003/10/Serialization/Arrays:Value' contains data of the 'http://schemas.microsoft.com/2003/10/Serialization/Arrays:ArrayOfstring' data contract. The deserializer has no knowledge of any type that maps to this contract. Add the type corresponding to 'ArrayOfstring' to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding it to the list of known types passed to DataContractSerializer."}
It seems like I tried to save some non-string data to Application.Current.Properties. Unfortunately I can't run .Clear() method to erease all data, bacause I receive this error any time I'm trying to access this property.
What should I do to make it work?
Well, as its name suggests AggregateException, is just a container for one or more exceptions which may be thrown when using PLINQ or TPL.
As such exceptions may be thrown on different threads and may also occur concurrently, the system automatically catches and rethrows them within an AggregateException wrapper to ensure that they all get reported in one place. The exceptions themselves are exposed via the InnerExceptions property.
You can catch an AggregateException and check which exceptions it actually contains with code such as the following:
try
{
// perform some parallel operation
}
catch (AggregateException aex)
{
string messages = "";
foreach(Exception ex in aex.InnerExceptions)
{
messages += ex.Message + "\r\n";
}
MessageBox.Show(messages);
}
So I suggest you do this to see what is causing the problem
Please, remove your app from your device, Settings - Applications- Uninstall, this works for me. The Auth Object was crash in debug mode.Clean and Rebuild can be Helpfull to.

Joomla >1.7 hide log messages from browser

I'm developing an extension for Joomla!; at the moment I'm trying to make it 3.0 compatible - as with 3.0 the logging changed a little (*). Building on the answer from this related question, my current code looks like this:
JLog::addLogger(array(
'text_file' => 'plg_system_myplg.log.php'
));
JLog::add('blah blah log msg');
The problem is that the log also goes to the messages which are shown to the user - this I want to prevent, I want the log msg only to go to the log file. I think it has to do with the "category" that JLog::add takes as a 3rd (optional) parameter, but I have no idea what to pass there?
Can anybody tell me how to hide the messages / or tell me if I'm on the right way with the categories and what value I should use?
Thanks!
(*) It actually changed already with 1.7 as far as I gathered so far, but the old method of calling addEntry on the return of JLog::getInstance(...) seems to have been removed from 2.5 to 3.0.
Edit: Think I found a way now; using:
JLog::addLogger(array(
'text_file' => 'plg_system_myplg.log.php',
JLog::ALL,
'myplg'
));
JLog::add('blah blah log msg', JLog::INFO, 'myplg');
all my log entries go only into my log file (and not to the messages shown to the user). However, I also get a few deprecation warnings - one about my code, but also some unrelated ones:
WARNING deprecated JAccess::getActions is deprecated. Use JAccess::getActionsFromFile or JAcces::getActionsFromData instead.
WARNING deprecated JSubMenuHelper::getEntries() is deprecated. Use JHtmlSidebar::getEntries() instead.
WARNING deprecated JSubMenuHelper::getFilters() is deprecated. Use JHtmlSidebar::getFilters() instead.
WARNING deprecated JSubMenuHelper::getAction() is deprecated. Use JHtmlSidebar::getAction() instead.
Not sure what to make of those - why do they appear in my log file, shouldn't they go to the default error.log file instead of my file ?
This is what I am using, works for Joomla 1.5 - 3.2:
if(version_compare(JVERSION,'1.7.0','ge')) {
jimport('joomla.log.log'); // Include the log library (J1.7+)
$priorities = JLog::ALL ^ JLog::WARNING; // exclude warning (because of deprecated)
// In J3.0 we need to ensure that log messages only go to our file, thus use the categories (already supported in J2.5)
if(version_compare(JVERSION,'2.5.0','ge')) {
$logCategory = 'com_mycomponent';
JLog::addLogger(array('text_file' => $logFileName), $priorities, $logCategory);
JLog::add($msg, JLog::INFO, $logCategory);
}else{
JLog::addLogger(array('text_file' => $logFileName), $priorities);
JLog::add($msg, JLog::INFO);
}
} else {
// Joomla! 1.6 and 1.5
jimport('joomla.error.log'); // Include the log library
$log = &JLog::getInstance($logFileName);
$log->addEntry(array('comment' => $msg, 'level' => 'INFO'));
}
This shows the trick for gettring of the deprecated messages.
And yes, you have to include a category for your messages to ensure they are not showing up as system messages.
Use
new JException('Something happened');
This will only add it to debug log but will not show anything.
It seems, that Joomla 3.0 has no default logger enabled. The same in Joomla 3.0.3. Nothing turns logging on by default - even Debug mode.
Finally I think I have solved my issue with unrelated log entries showing up.
A close look at the API documentation of the addLogger function revealed that the third parameter, $categories, is supposed to be an array of categories for which this log will be used.
This is in contradiction to the version of http://docs.joomla.org/Using_JLog that is current at the time of this writing, where a single category is given instead of an array.
Changing my call to addLogger to use an array, like this:
JLog::addLogger(array(
'text_file' => 'plg_system_myplg.log.php',
JLog::ALL,
array('myplg')
));
And keeping my fingers crossed that this will fix the issue!
Edit: unfortunately even this still doesn't solve my issue - still got unrelated entries :(.
I found the answer.. hope this script make you understand.. I already built as function . this code work on joomla 3. hope work in joomla 2
<?php
function logWrite($level, $values, $file='%s.php',$path='',$showOnTop=0,
$option='',$component=''){
/****
jlog Joomla 3.4
created by:gundambison (2015.04.26).
THX: hbit#stackoverflow
****/
jimport('joomla.log.log');
$level=strtoupper($level);
//You can change this com_name
$component= $component==''? 'com_gundambison': $component;
$date= date("Ymd");
$filename= sprintf($file, $date);
$format= $option=='' ?"{TIME}\t{CLIENTIP}\t{CATEGORY}\t{MESSAGE}": $option;
// create options and text
$txt = is_array($values)? json_encode($values): $values;
$options = array('text_file' => $filename,'text_entry_format'=>$format );
$options['text_file_path']=$path==''?'logs': $path;
JLog::addLogger ($options);
/*
if you want the error to show in your page. just see the different
*/
if($showOnTop==1){
JLog::add("$txt");
}
else{
JLog::add("$level\t$txt",$level,$component);
}
}

Joomla 3.0 generic database error handling

Going from Joomla 2.5 to 3.0 with my extension, I'm struggling with how to do the DB error handling (since GetErrorNum is deprecated, see also Joomla! JDatabase::getErrorNum() is deprecated, use exception handling instead).
The way that seems to be the one to go according to the question linked above, is to add the following code for each db->query() code:
if (!$db->query()) {
throw new Exception($db->getErrorMsg());
}
In my opinion, that makes DB error handling more awkward than it was before. So far, I simply called a checkDBError() function after a DB call, which queried the ErrorNum and handled any possible error accordingly.
That was independent from how the DB query was actually triggered - there are different ways to do that, and different results on an error: $db->loadResult() returns null on error, $db->query() returns false. So there will now be different checks for different DB access types.
Isn't there any generic way to handle this, e.g. a way to tell Joomla to throw some exception on DB problems? Or do I have to write my own wrapper around the DatabaseDriver to achieve that? Or am I maybe missing something obvious?
Or should I just ignore the deprecation warning for now and continue with using getErrorNum()? I'd like to make my extension future-proof, but I also don't want to clutter it too much with awkward error handling logic.
Just found this discussion: https://groups.google.com/forum/#!msg/joomla-dev-general/O-Hp0L6UGcM/XuWLqu2vhzcJ
As I interpret it, there is that deprecation warning, but there is no proper replacement yet anyway...
Unless somebody points out any other proper documentation of how to do it in 3.0, I will keep to the getErrorNum method of doing stuff...
Get getErrorNum() function will solve your problem....
$result = $db->loadResult();
// Check for a database error.
if ($db->getErrorNum())
{
JFactory::getApplication()->enqueueMessage($db->getErrorMsg());
return false;
}

Splunk-client (with Nokogiri) giving Undefined Namespace Prefix

I'm using splunk-client to extract results from splunk. Here's the code:
query = "sourcetype=collection #{order_id}"
search = #splunk_client.search(query)
search.wait
The search is happening fine, and it seems like I'm doing everything according to the example (https://github.com/cbrito/splunk-client), but I get this error on the 'search.wait' line:
Undefined namespace prefix: //s:key[#name='isDone']
Any ideas what could be going wrong? Running these commands in irb works fine. Is there some sort of blocking issue?
There is currently very little error checking which occurs within the gem itself. The reason for the error is that wait looks for the status of the isDone key to change to true.
Since your credentials were not properly setup in the first place, the gem creates a search object with an invalid session. The search does not initially fail, because enough response came back from Splunk that Nokogiri processes it into an object without a Splunk search sid.
In the future I should likely raise an exception if a proper sid is not returned to avoid confusion.
Source: I wrote the gem.
I found out the issue -- the splunk client wasn't authenticating properly, and so search was actually a broken SplunkJob object (with a nil username and authentication key). It's strange that there was no error raised until the wait command, but upon inspecting the search object, one of the fields stated that the object was malformed.

Resources