I want to get specific embedded array from mongodb document and add new document in that embedded array using "mongodb.Driver" .net driver.
am inserting doc as:
{
"_id": "5c41b5c6b0ce0437dc576c53",
"ProjectId": "234",
"OwnerId": "62",
"ProjectName": "proj4h46m",
"FileDetails": [
{
"TotalWord": "-1",
"RepeatedWord": "-1",
"TMWordCount": "-1",
"TranslationRequired": "-1",
"ParentFileName": "test",
"ChildFileName": "test_AR-SA",
"Status": "Newly Uploaded"
}
]
}
I expect to get "FileDetails" array from it and add new doc and update to mongodb. as shown below:
{
"_id": "5c41b5c6b0ce0437dc576c53",
"ProjectId": "234",
"OwnerId": "62",
"ProjectName": "proj4h46m",
"FileDetails": [
{
"TotalWord": "-1",
"RepeatedWord": "-1",
"TMWordCount": "-1",
"TranslationRequired": "-1",
"ParentFileName": "test",
"ChildFileName": "test_AR-SA",
"Status": "Newly Uploaded"
},
{
"TotalWord": "10",
"RepeatedWord": "3",
"TMWordCount": "12",
"TranslationRequired": "1",
"ParentFileName": "test2",
"ChildFileName": "test_AR-KSA",
"Status": "Newly Uploaded"
}
]
}
I got this by using below method:-
var query2 = Query.EQ(""ProjectId", "234");
var document=#"{""TotalWord"": ""10"",""RepeatedWord"": "3",""TMWordCount"": ""12"",""TranslationRequired"": ""1"",""ParentFileName"": ""test2"",""ChildFileName"": ""test_AR-KSA"",""Status"": ""Newly Uploaded""}";
var update = Update.Push("FileDetails", document.ToBsonDocument());
collec.Update(query2, update);
Related
My NiFi application receives two kinda different types of JSON's.
First of them looks like:
[
{
"campaign": {
"resourceName": "customers/8952771329/campaigns/11381694617",
"status": "ENABLED",
"name": "Saint_Spring_Active Minerals_oct-nov_2020_trueview_skip_5766500views",
"id": "11381694617"
},
"metrics": {
"interactionEventTypes": [
"VIDEO_VIEW"
],
"clicks": "6",
"videoQuartileP100Rate": 0.44493171079034244,
"videoQuartileP25Rate": 0.9747718298919024,
"videoQuartileP50Rate": 0.7339309987701469,
"videoQuartileP75Rate": 0.5337562301767105,
"videoViewRate": 0.4471109114825628,
"videoViews": "27872",
"viewThroughConversions": "0",
"contentBudgetLostImpressionShare": 0.0000013066088274492382,
"contentImpressionShare": 0.0999,
"contentRankLostImpressionShare": 0.9001,
"conversionsValue": 0,
"conversions": 0,
"costMicros": "9338700950",
"ctr": 0.00009624947864865732,
"currentModelAttributedConversions": 0,
"currentModelAttributedConversionsValue": 0,
"engagementRate": 0,
"engagements": "0",
},
"segments": {
"device": "CONNECTED_TV",
"date": "2020-12-20"
}
}
]
And second:
[
{
"adGroup": {
"resourceName": "customers/5404177717/adGroups/110501283582",
"campaign": "customers/5404177717/campaigns/11628802542"
},
"metrics": {
"interactionEventTypes": [
"CLICK"
],
"clicks": "1",
"averageCpm": 95497428.02172929,
"gmailForwards": "0",
"gmailSaves": "0",
"gmailSecondaryClicks": "0",
"impressions": "4418",
"interactionRate": 0.00022634676324128565,
"interactions": "1"
},
"adGroupAd": {
"resourceName": "customers/5404177717/adGroupAds/110501283582~480227690139",
"status": "ENABLED",
"ad": {
"resourceName": "customers/5404177717/ads/480227690139",
"id": "480227690139",
"name": "20 sec perek"
},
"adGroup": "customers/5404177717/adGroups/110501283582"
},
"segments": {
"device": "DESKTOP",
"date": "2020-11-21"
}
}
]
I already have 2 tables in my database to save this data. I have an attribute table.name just to not create same block where's only table name is different.
My next block is FlattenJson. After this i'm using ReplaceText with search value (replacement value is empty string): (customers\\\/${client.customer.id}\\\/campaigns\\\/|customers\\\/${client.customer.id}\\\/adGroups\\\/).
Why this? From this line: "adGroup": "customers/5404177717/adGroups/110501283582" i only need last value 110501283582 as ad_group_id. And from this line: "campaign": "customers/5404177717/campaigns/11628802542" i only need 11628802542. ${client.customer.id} can be different, so i'm using EL features.
Also i need to change json value name adGroup to ad.group.id, for this i'm also using ReplaceText.
Can i do it faster without two ReplaceText processors?
Look at the following processors...I think using them can be an alternative:
JoltTransformJSON:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.JoltTransformJSON/
UpdateRecord:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/index.html
I have managed to create an import from Kafka to Elasticsearch using Kafka Connect.
Connector-config:
{
"name": "raw-customer-equipment",
"config": {
"connector.class": "io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
"tasks.max": 1,
"topics": "raw.customer.equipment",
"key.ignore": true,
"value.converter.schemas.enable": false,
"schema.ignore": true,
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"connection.url": "<elastic-url>",
"connection.username": "<user>",
"connection.password": "<pwd>",
"type.name": "_doc" }
}
However Elasticsearch doesnt seem to be able to map the imported (Json) data. When peeking on it in Kibana the imported data doesnt seem to be search'able.
{
"_index": "raw.customer.equipment",
"_type": "_doc",
"_id": "raw.customer.equipment+1+929943",
"_version": 1,
"_score": 0,
"_source": {
"ifstats_list": [
{
"Event Time": "1589212678436",
"AP_list": [
{
"AP ID": 1,
"AP Alias": "PRIV0"
},
{
"AP ID": 2,
"AP Alias": "VID1"
},
{
"AP ID": 5,
"AP Alias": "VID1_BH"
}
],
"Device Type": "<type>",
...
"Associated Stations": [
{
"Packets sent": 11056613,
"Packets received": 304744,
"Multiple Retries Count": 0,
"Channel STA": 6,
"MAC Address": "<mac>",
....
},
{
....
}]
....
I want to be able to query by for instance "MAC Address" but Elastic seem to just handle the imported data as a big text-chunk.
I guess It is something in the Kafka-connector setup that is missing or wrong but I fail to see what.
As you might have guessed Im new at Elastic, and Im not the one supposed to be able to use the data in the end
Any help appreciated
BR
Edit:
Added connector-config by request.
I am getting data from database using join query in laravel & pass to json and getting some result in array but I want object that I given below
Controller code:
$resultPastActivity= DB::table('table_user_create_activity')
->join('table_sub_category','table_user_create_activity.selected_activity_id', '=', 'table_sub_category.sub_category_id')
->select('sub_category_name','area','activity_type','activity_date','start_time','end_time')
->whereDate('activity_date', '<', $todayDate)
->where('user_id',$user_id)
->get();
return response()->json(['success' => '1','data' =>$resultPastActivity]);
The above code will give following json that is in array actually i want json in object
{
"success": "1",
"data": [
{
"sub_category_name": "Badminton",
"area": "Rankala lake",
"activity_type": "1",
"activity_date": "2018-01-12",
"start_time": "15:04:49",
"end_time": "20:05:69"
},
{
"sub_category_name": "Football",
"area": "Devakar panad",
"activity_type": "1",
"activity_date": "2018-01-15",
"start_time": "15:04:49",
"end_time": "20:05:69"
},
]
}
i want json as follows
{
"success": "1",
"data": {
{
"sub_category_name": "Badminton",
"area": "Rankala lake",
"activity_type": "1",
"activity_date": "2018-01-12",
"start_time": "15:04:49",
"end_time": "20:05:69"
},
{
"sub_category_name": "Football",
"area": "Devakar panad",
"activity_type": "1",
"activity_date": "2018-01-15",
"start_time": "15:04:49",
"end_time": "20:05:69"
},
}
}
try this, json_encode($json, JSON_FORCE_OBJECT) JSON_FORCE_OBJECT
$result=array();
$result["0"]=$resultPastActivity;
$json=json_encode((object)$result,JSON_FORCE_OBJECT);
return response()->json(['success' => '1','data' =>$json]);
The one you want is not a valid JSON, you can check it here: https://jsonformatter.curiousconcept.com/
So, you cannot create such an output, and, if you force it, the application that will receive it won't parse it correctly, so it doesn't make sense.
I am new to elastic search, I have created an index "cmn" with a type "mention". I am trying to import data from my existing solr to elasticsearch, so I want to map an existing field to the _id field.
I have created the following file under /config/mappings/cmn/,
{
"mappings": {
"mentions":{
"_id" : {
"path" : "docKey"
}
}
}
}
But this doesn't seem to be working, every time I index a record the following _id is created,
"_index": "cmn",
"_type": "mentions",
"_id": "k4E0dJr6Re2Z39HAIjYMmg",
"_score": 1
Also, the mapping is not reflects. I have also tried the following option,
{
"mappings": {
"_id" : {
"path" : "docKey"
}
}
}
SAMPLE DOCUMENT: Basically a tweet.
{
"usrCreatedDate": "2012-01-24 21:34:47",
"sex": "U",
"listedCnt": 2,
"follCnt": 432,
"state": "Southampton",
"classified": 0,
"favCnt": 468,
"timeZone": "Casablanca",
"twitterId": 473333038,
"lang": "en",
"stnostem": "#ootd #ootw #fashion #styling #photography #white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"sourceId": "tw",
"timestamp": "2014-04-09T22:58:00.396Z",
"sentiment": 0,
"updatedOnGMTDate": "2014-04-09T22:56:57.000Z",
"userLocation": "Southampton",
"age": 0,
"priorityScore": 57.4700012207031,
"statusCnt": 14612,
"name": "YazzyK",
"profilePicUrl": "http://pbs.twimg.com/profile_images/453578494556270594/orsA0pKi_normal.jpeg",
"mentions": "",
"sourceStripped": "Instagram",
"collectionName": "STREAMING",
"tags": "557/161/193/197",
"msgid": 1397084280396.33,
"_version_": 1464949081784713200,
"url2": "{\"urls\":[{\"url\":\"http://t.co/YbPFrXlpuh\",\"expandedURL\":\"http://instagram.com/p/mliZbgxVZm/\",\"displayURL\":\"instagram.com/p/mliZbgxVZm/\",\"start\":88,\"end\":110}]}",
"links": "http://t.co/YbPFrXlpuh",
"retweetedStatus": "",
"twtScreenName": "YazKader",
"postId": "454030232501358592",
"country": "Bermuda",
"message": "#ootd #ootw #fashion #styling #photography #white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"source": "Instagram",
"parentStatusId": -1,
"bio": "Live and breathe Fashion. Persian and proud- Instagram: #Yazkader",
"createdOnGMTDate": "2014-04-09T22:56:57.000Z",
"searchText": "#ootd #ootw #fashion #styling #photography #white #pink #playsuit #prada #sunny #spring http://t.co/YbPFrXlpuh",
"isFavorited": "False",
"frenCnt": 214,
"docKey": "tw_454030232501358592"
}
Also, how can we create unique mapping for each "TYPE" and not just the index.
Thanks
Do like this,
Put the mapping as,
PUT index_name/type_name/_mapping
{
"type_name": {
"_id": {
"path": "docKey"
},
"properties": {
"docKey": {
"type": "string"
}
}
}
}
And, it will work. (When you index docKey, then _id is set). You shouldn't have to provide all the mapping.
hihi i am doing a project regarding ajax (xmlHttRequest),
how am i going to call the title in the note session, because normally if you call start year is like, detail = eval....
then for loop it
inside should be
var start year=""
startyear += ...[i].startyear
something like this, but how am i going to call the title inside the note?
i try to call detail.notes.note.title it say is null or is not a object
this is json data:
{
"infos": {
"info": [
{
"startYear": "1900",
"endYear": "1930",
"timeZoneDesc": "daweerrewereopreproewropewredfkfdufssfsfsfsfrerewrBlahhhhh..",
"timeZoneID": "1",
"note": {
"notes": [
{
"id": "1",
"title": "Mmm"
},
{
"id": "2",
"title": "Wmm"
},
{
"id": "3",
"title": "Smm"
}
]
},
"links": [
{
"id": "1",
"title": "Red House",
"url": "http://infopedia.nl.sg/articles/SIP_611_2004-12-24.html"
},
{
"id": "2",
"title": "Joo Chiat",
"url": "http://www.the-inncrowd.com/joochiat.htm"
},
{
"id": "3",
"title": "Bake",
"url": "https://thelongnwindingroad.wordpress.com/tag/red-house-bakery"
}
]
}
]
}
}
Use http://jsonlint.com/ to help you out with JSON.
I got this...
Parse error on line 30:
... ] }
----------------------^
Expecting '}', ',', ']'