Transforming XML to JSON using dataweave - transformation

Request to please help me with the dataweave to achieve below output. Here we need to do the following.
We need to add default values when XML element is null.
We need to map all the repeating elements inside XML element to their corresponding parent XML element.
I tried writing the dataweave for the same but was not able to achieve the desired response.
Request
<PacketResponse>
<PackResult>
<Result>0</Result>
<packs>
<pack>
<SeqNo>1</SeqNo>
<PackNumber>2550082245045</PackNumber>
<packageDate>2022-07-08T13:31:55.917+02:00</packageDate>
<Values i:nil="true"/>
</pack>
<pack>
<SeqNo>2</SeqNo>
<PackNumber>2550256260676</PackNumber>
<packageDate>2022-07-27T08:36:18.12+02:00</packageDate>
<Values>
<GetPackValue>
<Code>000228</Code>
<Count>80</Count>
<Amount>1600.00</Amount>
</GetPackValue>
<GetPackValue>
<Code>000227</Code>
<Count>20</Count>
<Amount>200.00</Amount>
</GetPackValue>
<GetPackValue>
<Code>000229</Code>
<Count>40</Count>
<Amount>2000.00</Amount>
</GetPackValue>
</Values>
</pack>
<pack>
<SeqNo>3</SeqNo>
<PackNumber>2550326072025</PackNumber>
<packageDate>2022-08-09T13:17:43.197+02:00</packageDate>
<Values>
<GetPackValue>
<Code>000240</Code>
<Count>5</Count>
<Amount>260.00</Amount>
</GetPackValue>
</Values>
</pack>
</packs>
</PackResult>
</PacketResponse>
Expected Response
[
{
"seqNo": "1",
"PackNumber": "2550082245045",
"packageDate": "2022-07-08 01:31:55",
"code": "000000",
"count": 0,
"amount": 0
},
{
"seqNo": "2",
"PackNumber": "2550256260676",
"code": "000228",
"packageDate": "2022-07-27 08:36:18",
"count": "80",
"amount": 1600.00
},
{
"seqNo": "2",
"PackNumber": "2550256260676",
"code": "000227",
"packageDate": "2022-07-27 08:36:18",
"count": "20",
"amount": 200.00
},
{
"seqNo": "2",
"PackNumber": "2550256260676",
"code": "000229",
"packageDate": "2022-07-27 08:36:18",
"count": "40",
"amount": 2000.00
},
{
"seqNo": "3",
"PackNumber": "2550326072025",
"code": "000240",
"packageDate": "2022-08-09 01:17:43",
"count": "5",
"amount": 260.00
}
]

I've used a combination of flatMap() and multi-value selector:
%dw 2.0
output application/json
---
payload.PacketResponse.PackResult.packs.*pack flatMap ((pack, index) -> do {
var packResult = {
seqNo: pack.SeqNo,
"PackNumber": pack.PackNumber,
"packageDate": pack.packageDate,
}
---
pack.Values.*GetPackValue map ((packValue, index) -> packResult ++ packValue) default ([packResult ++ {"code": "000000", "count": 0,"amount": 0}])
}
)

Here's an alternative way to do the same.
The key here is that you need to iterate for each of the GetPackValues but make sure to generate a value for the cases where there's no value too.
Here the pack.Values.*GetPackValue default [{}] takes care of that.
Finally since you have a nested Array because some Values have multiple GetPackValues, the flatMap takes care of flattening those results into a simple Array.
%dw 2.0
output application/json
---
payload.PacketResponse.PackResult.packs.*pack flatMap (pack) ->
pack.Values.*GetPackValue default [{}] map (value) -> {
"seqNo": pack.SeqNo,
"PackNumber": pack.PackNumber,
"packageDate": pack.packageDate,
"code": value.Code default "000000",
"count": value.Count default 0,
"amount": value.Amount default 0
}

Use multivalue selector and apply map on it, like mentioned below:
' %dw 2.0
output application/json
---
payload.PacketResponse.PackResult.*packs.*pack map ((item, index) -> {
seqNo: item.SeqNo,
PackNumber: item.PackNumber,
packageDate: item.packageDate,
code: item.Values.GetPackValue.Code default "",
count: item.Values.GetPackValue.Count default "",
amount: item.Values.GetPackValue.Amount default "
} ) '

Related

How to filter traces by value in segments list in AWS X-Ray Traces

In AWS CloudWatch -> X-Ray -> Traces I can find traces like:
{
"Id": "1234",
"Duration": 26.753,
"LimitExceeded": false,
"Segments": [
{
"Id": "1qaz",
"Document": {
"id": "1qaz",
"name": "somesite.io",
"start_time": 221,
"trace_id": "1234",
"end_time": 222,
"parent_id": "qwerty",
"inferred": true,
"http": {
"request": {
"url": "https://somesite.io/api/SUPERID",
"method": "POST"
},
"response": {
"status": 200,
"content_length": 100
}
}
}
}
]
}
I wish to find trace with value SUPERID what is in above JSON in:
Segments[?].Document.http.request.url
To sum up: I'm looking for example query what should be entered to Find input to find above trace by SUPERID value
Refer to X-Ray filter expression
There are 2 ways:
filter by http.url
Add SUPERID in segment annotation, then filter it by annotation.key

Validation of each field in JSON response using karate API

I want to validate particular fields in the response whether it is integer or float(ex: fullbathrooms field). I tried below code but getting match failed error. Could you please help here ?.....Thanks
Given path '/property-client'
And request {"address": <address>,"city": <city>,"state": <state>,"zipCode": <zipCode>}
When method post
Then status 200
And print response
And match response == {fullbathrooms:'#number'}
Examples:
|read('testFile1.csv')|
Error : match failed: EQUALS
Actual response:
{
"success": true,
"message": {
"version": "1.0",
"response": {
"id": "94568859",
"type": "express",
"responseheader": null,
"reportdata": {
"property": {
"source": null,
"type": null,
"dom": null,
"propertytype": "Single Family Residence",
"standardtype": null,
"address": {
"documentid": null,
"number": "150",
"directional": null,
"street": "BRIDGE",
"suffix": "RD",
"postdirectional": null,
"unit": "",
"city": "HILLSBOROUGH",
"state": "CA",
"zip": "94010",
"zipplus4": "6908",
"fulladdress": "150 BRIDGE RD, HILLSBOROUGH, CA 94010"
},
"info": {
"type": null,
"fips": "6081",
"county": "San Mateo",
"bedrooms": "5",
"bathrooms": "6.50",
"fullbathrooms": "6.50",
"totalrooms": "0",
"livingarea": "7750",
"totallivingarea": "7750",
"landarea": "41382",
"landareatype": null,
"pool": "true",
"landvalue": "6904800",
"improvementvalue": "3284414",
"assessedvalue": "10189214",
"assessedyear": "2021",
"taxvalue": "11746898",
"taxyear": "2021",
"deliquentyear": null,
"yearbuilt": "2011",
"propertytax": null,
"approxage": "11",
"parcelnumber": "032-400-110",
"titlecompany": null,
"geocode": {
"latitude": "37.563272",
"longitude": "-122.334442",
"geoqualitycode": ""
}
}
Please take some time to read the documentation: https://github.com/karatelabs/karate#match-contains
I'm not going to refer to your response dump (which by the way is not valid JSON), but give you a simple example. Please pay attention to the structure of your JSON. And note that the 6.50 is a string not a number in your response.
* def response = { "foo": { "bar": { "fullbathrooms": "6.50" } } }
* match response.foo.bar == { fullbathrooms: '#string' }
If you want to validate numbers within strings, please refer other answers, for example: https://stackoverflow.com/search?q=%5Bkarate%5D+number+regex

NiFi: ReplaceText alternatives to modify JSON

My NiFi application receives two kinda different types of JSON's.
First of them looks like:
[
{
"campaign": {
"resourceName": "customers/8952771329/campaigns/11381694617",
"status": "ENABLED",
"name": "Saint_Spring_Active Minerals_oct-nov_2020_trueview_skip_5766500views",
"id": "11381694617"
},
"metrics": {
"interactionEventTypes": [
"VIDEO_VIEW"
],
"clicks": "6",
"videoQuartileP100Rate": 0.44493171079034244,
"videoQuartileP25Rate": 0.9747718298919024,
"videoQuartileP50Rate": 0.7339309987701469,
"videoQuartileP75Rate": 0.5337562301767105,
"videoViewRate": 0.4471109114825628,
"videoViews": "27872",
"viewThroughConversions": "0",
"contentBudgetLostImpressionShare": 0.0000013066088274492382,
"contentImpressionShare": 0.0999,
"contentRankLostImpressionShare": 0.9001,
"conversionsValue": 0,
"conversions": 0,
"costMicros": "9338700950",
"ctr": 0.00009624947864865732,
"currentModelAttributedConversions": 0,
"currentModelAttributedConversionsValue": 0,
"engagementRate": 0,
"engagements": "0",
},
"segments": {
"device": "CONNECTED_TV",
"date": "2020-12-20"
}
}
]
And second:
[
{
"adGroup": {
"resourceName": "customers/5404177717/adGroups/110501283582",
"campaign": "customers/5404177717/campaigns/11628802542"
},
"metrics": {
"interactionEventTypes": [
"CLICK"
],
"clicks": "1",
"averageCpm": 95497428.02172929,
"gmailForwards": "0",
"gmailSaves": "0",
"gmailSecondaryClicks": "0",
"impressions": "4418",
"interactionRate": 0.00022634676324128565,
"interactions": "1"
},
"adGroupAd": {
"resourceName": "customers/5404177717/adGroupAds/110501283582~480227690139",
"status": "ENABLED",
"ad": {
"resourceName": "customers/5404177717/ads/480227690139",
"id": "480227690139",
"name": "20 sec perek"
},
"adGroup": "customers/5404177717/adGroups/110501283582"
},
"segments": {
"device": "DESKTOP",
"date": "2020-11-21"
}
}
]
I already have 2 tables in my database to save this data. I have an attribute table.name just to not create same block where's only table name is different.
My next block is FlattenJson. After this i'm using ReplaceText with search value (replacement value is empty string): (customers\\\/${client.customer.id}\\\/campaigns\\\/|customers\\\/${client.customer.id}\\\/adGroups\\\/).
Why this? From this line: "adGroup": "customers/5404177717/adGroups/110501283582" i only need last value 110501283582 as ad_group_id. And from this line: "campaign": "customers/5404177717/campaigns/11628802542" i only need 11628802542. ${client.customer.id} can be different, so i'm using EL features.
Also i need to change json value name adGroup to ad.group.id, for this i'm also using ReplaceText.
Can i do it faster without two ReplaceText processors?
Look at the following processors...I think using them can be an alternative:
JoltTransformJSON:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.JoltTransformJSON/
UpdateRecord:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache.nifi.processors.standard.UpdateRecord/index.html

How to Get specific embedded array from mongodb and append it with new document using mongodb c# driver

I want to get specific embedded array from mongodb document and add new document in that embedded array using "mongodb.Driver" .net driver.
am inserting doc as:
{
"_id": "5c41b5c6b0ce0437dc576c53",
"ProjectId": "234",
"OwnerId": "62",
"ProjectName": "proj4h46m",
"FileDetails": [
{
"TotalWord": "-1",
"RepeatedWord": "-1",
"TMWordCount": "-1",
"TranslationRequired": "-1",
"ParentFileName": "test",
"ChildFileName": "test_AR-SA",
"Status": "Newly Uploaded"
}
]
}
I expect to get "FileDetails" array from it and add new doc and update to mongodb. as shown below:
{
"_id": "5c41b5c6b0ce0437dc576c53",
"ProjectId": "234",
"OwnerId": "62",
"ProjectName": "proj4h46m",
"FileDetails": [
{
"TotalWord": "-1",
"RepeatedWord": "-1",
"TMWordCount": "-1",
"TranslationRequired": "-1",
"ParentFileName": "test",
"ChildFileName": "test_AR-SA",
"Status": "Newly Uploaded"
},
{
"TotalWord": "10",
"RepeatedWord": "3",
"TMWordCount": "12",
"TranslationRequired": "1",
"ParentFileName": "test2",
"ChildFileName": "test_AR-KSA",
"Status": "Newly Uploaded"
}
]
}
I got this by using below method:-
var query2 = Query.EQ(""ProjectId", "234");
var document=#"{""TotalWord"": ""10"",""RepeatedWord"": "3",""TMWordCount"": ""12"",""TranslationRequired"": ""1"",""ParentFileName"": ""test2"",""ChildFileName"": ""test_AR-KSA"",""Status"": ""Newly Uploaded""}";
var update = Update.Push("FileDetails", document.ToBsonDocument());
collec.Update(query2, update);

How to extract multiple json values from a Json response

I am trying to extract multiple values from a JSON response on my jmeter script. Below is sample of my response:
{
"startDate": "2018-12-10T15:36:34.400+0000",
"userId": "7211111-2fa90",
"createdBy": "TEST",
"note": {
"content": "Application Submitted "
},
"Type": "SUBMITTED"
},
"currentEventState": "CLOSED",
{
"Xxxx": "test",
"Loc": null,
"Zipcode": [],
"Locality": 82,
"Address": {
"Add": 12302,
"Add2": "place",
"Zip": {
"Phone": "home",
"Email": "test#test.com"
}
},
"state": "MD",
"Cost": "E "
},
"AppID": "cd8d98e6-c2a79",
"Status": "CLOSED",
}
I am trying to extract userid and AppID for the case if the TYPE is Submitted and Status is Closed.I tried using the Json extractor with $.[?(#.Type=="SUBMITTED")].[*].?(#.Status=="CLOSED").userid,APPID, but couldn't get the expected result. Could anyone guide me on this.
You need to use an inline predicate to combine 2 clausees and a semicolon in order to store results into 2 separate JMeter Variables.
Add JSON Extractor as a child of the request which returns above JSON
Configure it as follows:
Names of created variables: userid;appid
JSON Path Expressions: $..[?(#.Type=='SUBMITTED' && #.Status == 'CLOSED')].userId; $..[?(#.Type=='SUBMITTED' && #.Status == 'CLOSED')].AppID
Default values: NA;NA
Here is the demo of single expression working fine:
And here are extracted values reported by the Debug Sampler:

Resources