Lets assume I have a log file full of errors. but of just a handful of types.
com.company.exception.ExceptionOne: customer cart is empty for customerId: QWEDSA
com.company.exception.ExceptionOne: customer cart is empty for customerId: fdsadf
com.company.exception.ExceptionOne: customer cart is empty for customerId: ...
com.company.exception.ExceptionOne: customer cart is empty for customerId: ---
com.company.exception.ExceptionTWO: ITEM NOT FOUND on purchase: ---
com.company.exception.ExceptionTWO: ITEM NOT FOUND on purchase: ...
com.company.exception.ExceptionTWO: ITEM NOT FOUND on purchase: ---
com.company.exception.ExceptionTWO: unknown error.
com.company.exception.ExceptionTHREE: error found while requesting from service A {requestId: 1, ...}
com.company.exception.ExceptionTHREE: error found while requesting from service A {requestId: 2, ...}
Ideal output:
4 com.company.exception.ExceptionOne: customer cart is empty for customerId:
3 com.company.exception.ExceptionTWO: ITEM NOT FOUND on purchase:
2 com.company.exception.ExceptionTHREE: error found while requesting from service A {requestId:
1 com.company.exception.ExceptionTWO: unknown error.
It'd be easy if similar lines were identical with uniq -c | sort -bgr
Looking at your sample log entries, you are lucky, all the fields you want to ignore are located after the second :. You could therefore do:
cat log | cut -d':' -f1-2 | uniq -c | sort -bgr
Where log is the filename of your log file. Replace to match your reality.
Related
I have created Sumologic dashboard to show some errors in the application. What I want is show the error per date. It show the error but it doesn't aggregate the same error messages as the messages have some GUID.
This is the sample part of the query:
_sourceCategory="playGames/web-app-us"
and ERR
| timeslice 1d
| count _timeslice, message
enter image description here
I believe you need to format the message and remove the GUID. So all the messages with GUID will be aggregate to single message.
You can use regex to format the messages and remove the GUID. The sample query look like this and use as needed.
The sample error message is like this
Error occurred. Exception: System.Exception: my custom error message: 1121fd05-065b-499f-b174-2a13efdaf8b5
And the Sumologic query
_sourceCategory="dev/test-app"
and "[Error]"
and "Error occurred"
// | timeslice 1d
| formatDate(_receiptTime, "yyyy-MM-dd") as date
| replace(_raw,/my custom error message: ([0-9A-Fa-f\-]{36})/,"my custom error message") as finalMessage
| count date, finalMessage
| transpose row date column finalMessage
This video shows step by step guidance. https://youtu.be/Nxzp7G-rUh8
In GeoServer I successfully created WMS layer based on PostgreSQL table which has such structure:
| COLUMN NAME | DATA TYPE |
|-------------|-----------|
| id | numeric |
| geom | geometry |
| city | varchar |
| info | jsonb |
Each record in that table is unique and it's one polygon. In other words, this layer has a lot of polygons. With GetMap WMS request I put this layer to the map of the web application. When the user clicks to one of these polygons I want to know information about it. For example information from city and info columns. As far as I understand I have to make a GetFeatureInfo WMS request for this task, right? I tried such GET request to GeoServer, but it returns me the empty result. What I did wrong?
GET REQUEST:
http://{{domain_name}}/geoserver/{{namespace_name}}/wms?&
SERVICE=WMS&
VERSION=1.3.0&
REQUEST=GetFeatureInfo&
LAYERS={{layer_name}}&
SRS=EPSG%3A4326&
CRS=CRS%3A84&
FORMAT=image%2Fpng8&
BBOX=51.08443921044546%2C71.3090464064941%2C51.18218384993084%2C71.55709709619134&
WIDTH=1366&
HEIGHT=905&
QUERY_LAYERS={{layer_name}}&
INFO_FORMAT=application%2Fjson&
FEATURE_COUNT=50&
I=498&
J=391&
EXCEPTIONS=application%2Fvnd.ogc.se_xml&
STYLES=squaremesh_style
RESULT:
{
"type": "FeatureCollection",
"features": [],
"totalFeatures": "unknown",
"numberReturned": 0,
"timeStamp": "2019-12-24T17:59:23.429Z",
"crs": null
}
It's possible that there is nothing visible at pixel (669, 491) - did you try other points?
However, there are a number of issues with your request (though I think GeoServer takes a lenient view of some of them):
For version 1.3.0 of WMS you should use I and J for the query point.
Your SRS should be EPSG:4326.
You are missing a STYLES parameter
Finally, if all else fails don't be afraid to check the log file to see if an error is being logged.
I am trying to filter out a few records from the tail input to fluent-bit. But that does not seem to work. From the log files I need to exclude from all records with key value 'log'
1) Records that have 1 or more digits followed by a space
2) records with value 'Series' anywhere on the line
3) records with the value 'transacttime' anywhere on the line.
They could be the same or different records
[INPUT]
Name tail
Path /mnt/volume_nyc3_03/xenfix*.out
Tag genfix
DB /mnt/volume_nyc3_03/ggantel-gf.db
[FILTER]
Name grep
Match *
Exclude log ^[0-9]*\
Exclude log *Series*
Exclude log *transacttime*
[OUTPUT]
Name pulsar
Match *
Host somerandom-id.us-east-1.elb.amazonaws.com
Port 6650
Topic persistent://public/default/genfixlogs
[OUTPUT]
Name stdout
Match genfix
This does not exclude any records from the output as seen below
{"log":"0 1"}
----- got message -----
{"log":"2019-09-17 21:25:08.636465 Series([], Name: transacttime, dtype: datetime64[ns])"}
----- got message -----
{"log":"2019-09-17 21:25:08.633038 Series([], Name: transacttime, dtype: datetime64[ns])"}
----- got message -----
{"log":"2019-09-17 21:25:08.680237 Series([], Name: transacttime, dtype: datetime64[ns])"}
----- got message -----
{"log":"2019-09-17 21:25:08.890903 Series([], Name: transacttime, dtype: datetime64[ns])"}
You need to enclose in // when using regex.
Below is an example.
[FILTER]
Name grep
Match *
Exclude log /^[0-9]*/
Exclude log /.*Series.*/
Exclude log /.*transacttime.*/
This step in my Ruby feature file to find and access a customer record:
When I search with the following details: "<recordReference>", "<secondIdentifier>", "<postcode>":
| recordReference | secondIdentifier| postcode |
| REFABCDE12345678| IDcode1234 | PC11 1PC |
It has this step definition:
When(/^I search with the following details: "(.*?)", "(.*?)", "(.*?)":$/) do |recordReference, secondIdentifier, postcode|
find(:css, "#dln").native.send_keys recordReference
find(:css, "#nino").native.send_keys secondIdentifier
find(:css, "#postcode").native.send_keys postcode
check 'confirmation'
click_button 'submit'
end
When it's run, I get the following error:
Cucumber::ArityMismatchError: Your block takes 3 arguments, but the Regexp matched 4 arguments.
features/step_definitions/refactored_commands.rb:207:in `/^I search with the following details: "(.*?)", "(.*?)", "(.*?)":$/'
What have I done wrong and how can it be fixed?
For info - I get the same error message if the parenthases are take out of the step definition:
When /^I search with the following details: "(.*?)", "(.*?)", "(.*?)":$/ do |recordReference, secondIdentifier, postcode|
The fourth argument is the DataTable. Remove the first 3 parameters and put in just the DataTable option, you will get all the data from the DataTable. Suggest you use dryRun=true option to let Cucumber create the proper step definition matcher, this is from my Java knowledge dont know how this dryRun option is in ruby.
Plus you will have to change your step in the feature file to remove the mentioned 3 parameters
It looks like you're mixing up scenario outlines with passing data tables to steps
From the format of your table it looks like what you're going for should actually be
Scenario Outline: xxx
...
When I search with the following details: "<recordReference>", "<secondIdentifier>", "<postcode>"
...
Examples:
| recordReference | secondIdentifier| postcode |
| REFABCDE12345678| IDcode1234 | PC11 1PC |
and then each the outline will be called once for each row of Examples with the values filled in - so your step would be
When(/^I search with the following details: "(.*?)", "(.*?)", "(.*?)"$/) do |recordReference, secondIdentifier, postcode|
...
end
On a side note - is there any reason you're calling .native.send_keys - I believe every driver now supports the normal Capybara send_keys API so it would just be find(:css, '#dln').send_keys recordReference (or of course just fill_in('dln', with: recordReference) )
I'm trying to remove a duplicate document in my collection. Here's my code:
class Sites
include Mongoid::Document
store_in collection: "sites"
end
Sites.destroy_all(conditions: { _id: BSON::ObjectId("5685a45be4b06ab5911dcd12")})
Here's what is returned:
D, [2015-12-31T17:39:16.488657 #10126] DEBUG -- : MONGODB | localhost:27017 | rails.find | STARTED | {"find"=>"sites", "filter"=>{"conditions"=>{"_id"=>BSON::ObjectId('5685a45be4b06ab5911dcd12')}}}
D, [2015-12-31T17:39:16.488946 #10126] DEBUG -- : MONGODB | localhost:27017 | rails.find | SUCCEEDED | 0.000189083s
But when I search the document is still there :(
{"_id"=>BSON::ObjectId('5685a45be4b06ab5911dcd12'), "name"=>"StackOverflow", "title"=>"Stack Overflow", "type"=>"Forum", "url"=>"http://www.stackoverflow.com"}
How do I permanently delete a document by reference using Mongoid?
Mongoid with 5.x follows a lot more closely to the AR methods we are used to so your id field is behaves in a similar way.
Sites.where(id: '5685a45be4b06ab5911dcd12').delete
To have all the necessary callbacks be called as well just use .destroy. Just know this loads everything into memory and can be expensive.
Sites.where(id: '5685a45be4b06ab5911dcd12').destroy
After doing a lot of iterations this works for me:
Sites.destroy_all({ :_id => BSON::ObjectId('5685a45be4b06ab5911dcd12')})
I've found that Mongoid is not very helpful when troubleshooting. I have to do a lot more trial and error to discover how to do things :(