How to index the document with three letter month format? - elasticsearch

Following is the document I have.
{
"host": "172.31.183.9",
"log_sequence": "47280",
"log_date": "Nov 3 12:29:26.096"
}
I have tried the following date formats for mapping
a. dateOptionalTime
b. MMM d HH:mm:ss.SSSZ
but it is still showing mapper parsing exception. Can anybody suggest a way for the mapping format?

The problem is with the Z at the end of your date format, which expects a timezone offset/id, which is missing from your example.
So this pattern MMM d HH:mm:ss.SSS should work.

Elasticsearch official date format documentation says it support custom datetime format under joda.
Joda documentation for datetime mention
M month of year month July; Jul; 07
Month: 3 or over, use text, otherwise use number.
So you can refer this documentation to create a custom format for elasticsearch mappings and index the data accordingly.
I think you can figure out a way if you will play around with custom joda datetime formats.
Post the exact mappings if this helps

Related

SSRS Date Formatting and European numbers

I created a number of fields in order to sort out why I couldn't easily format my date field as a date (it is date in the database, but with FetchXML it seems to lose the formatting. I know this should be accomplished with nested formulas, but since I kept having errors, I wanted to understand where the problem was exactly.
Date is "CreatedOn" : dd/MM/yyyy hh:mm:ss
I created a field called "SignUpLeft" with this expression:
=Left(Fields!createdon.Value,10)
I created another field, and I called this "Cdate":
=Format(CDate(Fields!SignDateLeft.Value),"dd/MM/yyyy")
I have my report sorting by the "Cdate" field, but it is sorting by the Day, rather than by the month (see pic below.) I have changed the report attribute localization to "es-es".
What else can I do to get the date formatted correctly, in order to sort.

Can't remove hours and minutes from datatable

I'm trying to get release date field with y-m-d format.. Actually time format (for y-m-d) seems fine but also it gives h:m:s too, I changed time format at server side and removed h:m:s but datatable still shows them
What I get
2012-04-11 00:00:00
What I want
2012-04-11
Released_at field (Metronic theme - json datatable)
{
field: "released_at",
title: "Release Date",
type: "date",
format: "YYYY/MM/DD"
}
time format (I'm using laravel framework)
protected $dateFormat = "Y-m-d";
How do I fix this ?
Datatable accept column type same as mysql type.
You are trying to convert mysql dateTime column to Date in datatable, which is not possible from datatable.
In your code you have declared
format: "YYYY/MM/DD" // but actual type is dateTime as per mysql.So it will append time next to it.
If you are storing only date in mysql then you can change column type to Date, and your problem gets solved.
And if you want to convert string to date in laravel you can follow this post.
Your database column is of type dateTime which, by definition, includes both date and time information. If you want to remove the time at a database level then use the date type instead
$table->date("released_at")->nullable();
If instead of removing the time from the database, you just want to ignore it for certain parts of your application, you can leverage the fact than in Laravel all dates coming from your models are Carbon\Carbon instances, so you could do
$model->releasedAt->format('y-m-d'); // Returns '18-09-11'

How to update date display format in kibana?

I want to change the date display format in kibana. Kibana provides this feature via moment.js, but not enough documentation available, or at least it doesn't work properly.
Current date format: "MMMM Do YYYY, HH:mm:ss.SSS"
Required date format: "DD MMM YY" (like 29 July 17)
I tried changing the date format of the Date field in "Management" section, but that didn't work. (do I need to restart the server or something to make it work)
Or
Where am I going wrong
Or
What are some other ways to change date display format.
and What does popularity of the field do here ?
Some screenshots for the reference.
As a solution, I had to fix it in the scripts feeding data to elasticsearch itself.

How to get the week day name from a date in Apache pig?

Given "03/09/1982" how can we say it is which week day. In this case it will be "Tue".
Is it possible to get in a single query?
Thanks
You can convert this string into date object using ToDate(), then again into string with desired format using ToString(), and dont forget that Pig uses Java SimpleDateFormat class to deal with dates.
ToString( ToDate('03/09/1982','dd/MM/yyyy'), 'EEE' )

Filter by time and date in Hadoop

I have a table of data which have date and time as two separate field where date format is
dd/mm/yyyy and dd-mm-yyyy and time format is like hh:mm:ss(eg: 6:52:53)
i need to filter the record for a particular time period that both time and date wise filtering.
is there any predefined filter available with hive or pig?
Hive does recognize certain strings as unixtime dates.
You might try a where condition while concatenating the time & date together into unixtime format.
Some documentation on Hive date functions/formats are located here: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions
I suppose you have one column having two date format ie. dd/mm/yyyy and dd-mm-yyyy
What You can try
1) Replacing '/' to '-' so that complete column will be in dd-mm-yyyy format.
2) Try concatanating this field with time field
3) filter it by Casting concatinated field.
Hope this helps.
just possibility :- Have you tried casting that concatenated field to date datatype and then try date functions for desired output ?
eg. to_date()
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF

Resources