Flume Hive sink failed to serialize JSON with array - hadoop

Question

I am trying to load JSON data to Hive via Hive Sink.
But it fails with the following error:
WARN org.apache.hive.hcatalog.data.JsonSerDe: Error [java.io.IOException: Field name expected] parsing json text [{"id": "12345", "url": "https://mysite", "title": ["MyTytle"]}].
INFO org.apache.flume.sink.hive.HiveWriter: Parse failed : Unable to convert byte[] record into Object : {"id": "12345", "url": "https://mysite", "title": ["MyTytle"]}
Example of data:
{"id": "12345", "url": "https://mysite", "title": ["MyTytle"]}
Description of Hive table:
id string
url string
title array<string>
time string
# Partitions
time string
And the same way it works fine if JSON data doesn't contain arrays (and Hive table either).
Flume version: 1.7.0 (Cloudera CDH 5.10)
Does it possible to load JSON data with arrays via Flume Hive sink?

Answer 1

Is it possible to load JSON data with arrays via Flume Hive sink?
I assume it is possible, despite I never tried myself. From:
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_HDP_RelNotes/content/ch01s08s02.html
Following serializers are provided for Hive sink:
JSON: Handles UTF8 encoded Json (strict syntax) events and requires no
configuration. Object names in the JSON are mapped directly to columns
with the same name in the Hive table. Internally uses
org.apache.hive.hcatalog.data.JsonSerDe but is independent of the
Serde of the Hive table. This serializer requires HCatalog to be
installed.
So maybe you are implementing something wrong in the SerDe. This user solved the problem of serialising a JSON with arrays by performing a previous regexp:
Parse json arrays using HIVE
Another thing you may try is to change the SerDe. At least you have this two options (maybe there are some more):
'org.apache.hive.hcatalog.data.JsonSerDe'
'org.openx.data.jsonserde.JsonSerDe'
(https://github.com/sheetaldolas/Hive-JSON-Serde/tree/master)