I have a json array which looks like
{
[
{
"name": "ABCDEF",
"details": {
"code": null,
"description": null,
"range": {
"fromNo": "00174",
"toNo": "99999"
}
}
}, {
"name": "ABCDEF",
"details": {
"code": null,
"description": null,
"range": {
"fromSerialNo": "00001",
"toSerialNo": "00060"
}
}, {
"name": "ABCDEF",
"details": {
"code": null,
"description": null,
"range": {
"fromSerialNo": "00061",
"toSerialNo": "00173"
}
}]
}
and I want the output as
{
"name": "ABCDEF",
"details": {
"code": null,
"description": null,
"range": {
"fromSerialNo": "00001",
"toSerialNo": "99999"
}
But the problem is I cannot put the variable in EvaluateJsonPath as it does not accept nifi expression language. I have to loop through all the values and then get final range. Does anyone know if this can be done in nifi.
Using the out of the box processors there is currently no easy way to do this but NiFi was designed to be easily extensible. What you'll need to do it either write a custom java processor or script to solve this.
For an intro tutorial on writing custom java processors for NiFi check out this link: http://www.nifi.rocks/developing-a-custom-apache-nifi-processor-json/
As of version 0.5.0 there are the ExecuteScript and InvokeScripted processors. The creator of the processors has written a number of blogs on them here: http://funnifi.blogspot.com/
Related
I'm using JSONpath to try and find data with an array of JSON objects but I'm struggling to get to the information I want. The array contains many objects similar to below where there are values for RecID throughout. If I use $..RecID I get them all when I only want the first Key.RecID of each object (with a value 1338438 in this example). Is there a way to only extract the top level Key.RecID value?
BTW I'm trying to do this in jMeter and I'm assuming JSONpath is the best way to do what I want but if there is a better way I'd be happy to hear about it.
Thanks in advance
[{
"Key": {
"RecID": 1338438
},
"Users": [{
"FullName": "Miss Burns",
"Users": {
"Key": {
"Name": "Burns",
"RecID": 1317474
}
}
},
{
"FullName": "Mrs Fisher",
"Users": {
"Key": {
"Name": "Fisher",
"RecID": 1317904
}
}
}
],
"User": {
"FullName": "Mrs Fisher",
"Key": {
"Name": "Fisher",
"RecID": 1317904
}
},
"Organisation": {
"Key": {
"RecID": 1313881
}
}
}]
Trying to follow the example here: https://stripe.com/docs/issuing/cards/virtual
When I add params.AddExpand("number"), no number is returned, yet via the dashboard I was able to see the card numbers. Here's sample code and redacted info for the Req and Resp.
func (ac *appContext) CardRetrieve(id string) *stripe.IssuingCard {
stripe.Key = ac.Config.Stripe.SecretKey
params := stripe.IssuingCardParams{}
params.AddExpand("number")
params.AddExpand("cvc")
ic_num, _ := card.Get(id, ¶ms)
return ic_num
}
Returns:
{
"id": "ic_redacted",
"object": "issuing.card",
"brand": "Visa",
"cancellation_reason": null,
"cardholder": {
"id": "ich_redacted",
"object": "issuing.cardholder",
"billing": {
"address": {
"city": "A Beach",
"country": "US",
"line1": "404 Main St.",
"line2": "Suite #302",
"postal_code": "19001",
"state": "DE"
}
},
"company": null,
"created": 1613338532,
"email": "redacted#notreal.com",
"individual": {
"dob": {
"day": 20,
"month": 10,
"year": 1990
},
"first_name": "User",
"last_name": "Testing",
"verification": {
"document": {
"back": null,
"front": null
}
}
},
"livemode": false,
"metadata": {
},
"name": "User Testing",
"phone_number": "+15165551212",
"requirements": {
"disabled_reason": "under_review",
"past_due": [
]
},
"spending_controls": {
"allowed_categories": [
],
"blocked_categories": [
],
"spending_limits": [
{
"amount": 1,
"categories": [
],
"interval": "daily"
}
],
"spending_limits_currency": "usd"
},
"status": "active",
"type": "individual"
},
"created": 1613338532,
"currency": "usd",
"exp_month": 1,
"exp_year": 2024,
"last4": "0088",
"livemode": false,
"metadata": {
},
"replaced_by": null,
"replacement_for": null,
"replacement_reason": null,
"shipping": null,
"spending_controls": {
"allowed_categories": null,
"blocked_categories": null,
"spending_limits": [
{
"amount": 1,
"categories": [
],
"interval": "daily"
}
],
"spending_limits_currency": "usd"
},
"status": "inactive",
"type": "virtual"
}
What confuses me is the documentation found here:
https://stripe.com/docs/issuing/cards/virtual
It says: You can retrieve both the full unredacted card number and CVC from the API. For security reasons, these fields are only available for virtual cards and will be omitted unless you explicitly request them with the expand property. Additionally, they are only available through the Retrieve a card endpoint. That links to the issue card retrieval end point, but the params defined in the virtual cards example references the CardParams{} struct.
No of the examples show what imported module their aliasing for card to exec card.Get, but it stands to reason given the flow of the documentation that this should be IssuingCardParams{} and that the card alias is referencing: "github.com/stripe/stripe-go/issuing/card"
I also find it strange that we're defining params in the example but not passing it into the card.Get()
Edit:
I went digging through the module and it seems like to get the card details you have to call: details, _ := card.Details(id, params) but I get a 404 when trying to call that. The object returned is actually the right object and I see number and cvc, albeit nil.
I get the following error:
2021/02/15 00:33:06 Request error from Stripe (status 404): {"status":404,"message":"Unrecognized request URL (GET: /v1/issuing/cards/ic_redacted/details). Please see https://stripe.com/docs
So it seems you need to include a /v72 in the import:
"github.com/stripe/stripe-go/v72"
The documentation should be updated to show this and the virtual card example for go should also be updated.
In JMESPath with this query:
people[].{"index":#.index,"name":name, "state":state.name}
On this example data:
{
"people": [
{
"name": "a",
"state": {"name": "up"}
},
{
"name": "b",
"state": {"name": "down"}
},
{
"name": "c",
"state": {"name": "up"}
}
]
}
I get:
[
{
"index": null,
"name": "a",
"state": "up"
},
{
"index": null,
"name": "b",
"state": "down"
},
{
"index": null,
"name": "c",
"state": "up"
}
]
How do I get the index property to actually have the index of the array? I realize that #.index is not the correct syntax but have not been able to find a function that would return the index. Is there a way to include the current array index?
Use-case
Use Jmespath query syntax to extract the numeric index of the current array element, from a series of array elements.
Pitfalls
As of this writing (2019-03-22) this feature is not a part of the standard Jmespath specification.
Workaround
This is possible when running Jmespath from within any of various programming languages, however this must be done outside of Jmespath.
This is not exactly the form you requested but I have a possible answer for you:
people[].{"name":name, "state":state.name} | merge({count: length(#)}, #[*])
this request give this result:
{
"0": {
"name": "a",
"state": "up"
},
"1": {
"name": "b",
"state": "down"
},
"2": {
"name": "c",
"state": "up"
},
"count": 3
}
So each attribute of this object have a index except the last one count it just refer the number of attribute, so if you want to browse the attribute of the object with a loop for example you can do it because you know that the attribute count give the number of attribute to browse.
I have updated document in the elastic search. after update using I am fetching the same document by using their ID. It is giving me following response:
{
"_index": "b123456",
"_type": "documents",
"_id": "bltde56dd11ba998bab",
"_version": 3,
"found": true,
"_source": {
"title": "index.json",
"url": "/index1",
"tags": [],
"created_at": "2018-06-19T05:02:38.174Z",
"updated_at": "2018-06-19T05:07:57.155Z",
"version": 1,
"fields": [{
"uid": "fname",
"value": "john"
},
{
"uid": "lname",
"value": "test"
}
],
"class": "first"
}
}
After I am using update_by_query to update document I am sending following request to update_by_query:
{
"script": {
"source": "ctx._source.title = params.title;ctx._source.url = params.url;ctx._source.created_at = params.created_at;ctx._source.updated_at = params.updated_at;ctx._source.version = params.version;ctx._source.fields = params.fields",
"params": {
"title": "Demo title",
"url": "/demo",
"created_at": "2018-06-19T05:02:38.174Z",
"updated_at": "2018-06-19T05:07:57.155Z",
"version": 2,
"fields": [{
"uid": "fname",
"value": "vicky"
},
{
"uid": "lname",
"value": "test"
}
]
}
},
"query": {
"bool": {
"must": [{
"term": {
"_id": "bltde56dd11ba998bab"
}
},
{
"range": {
"version": {
"lt": 2
}
}
}
]
}
}
}
But it is giving me status code:409 and following error:
[documents][bltde56dd11ba998bab]: version conflict, current version
[3] is different than the one provided [2]
My document also contain custom version key.
Can anyone help me into this
For the sake of posterity, I'll submit an answer to this old question. The issue is occurring because ElasticSearch's internal version value in the _version field is actually 3 in your initial response, not 1.
You are then trying to update the document to using external version value 2, Elastic sees this as a conflict, as internally it thinks version 3 is the most up-to-date version, not version 1. Effectively, something as caused your external version scheme and Elastic's internal version scheme to become out-of-sync.
Also note, the following parameter should be included in your update calls to indicate that the operation should follow the rules for external versioning as opposed to Elastic's internal versioning scheme.
"version_type":external
There is a subtle but important distinction that needs to be made by specifying this parameter.
With version_type set to external, Elasticsearch will store the
version number as given and will not increment it. Also, instead of
checking for an exact match, Elasticsearch will only return a version
collision error if the version currently stored is greater or equal to
the one in the indexing command.
More information can be on Elastic's version can be found in their blog post
for me, it was document id. I am using node js elastic-search client, when I create a document I need to pass a document Id,
I was getting version conflict because I was trying to create multiple documents with the same id.
await elasticWrapper.client.create({
index: ElasticIndexs.Payments,
id: data.id, // <-- id should be unique
body: {
...data,
},
});
``
Suppose I have these documents in a Things table:
{
"name": "Cali",
"state": "CA"
},
{
"name": "Vega",
"state": "NV",
},
{
"name": "Wash",
"state": "WA"
}
My UI is a state-picker where the user can select multiple states. I want to display the appropriate results. The SQL equivalent would be:
SELECT * FROM Things WHERE state IN ('CA', 'WA')
I have tried:
r.db('test').table('Things').filter(r.expr(['CA', 'WA']).contains(r('state')))
but that doesn't return anything and I don't understand why that wouldn't have worked.
This works for getting a single state:
r.db('test').table('Things').filter(r.row('state').eq('CA'))
r.db('test').table('Things').filter(r.expr(['CA', 'WA']).contains(r.row('state')))
seems to be working in some versions and returns
[
{
"id": "b20cdcab-35ab-464b-b10b-b2f644df73e6" ,
"name": "Cali" ,
"state": "CA"
} ,
{
"id": "506a4d1f-3752-409a-8a93-83385eb0a81b" ,
"name": "Wash" ,
"state": "WA"
}
]
Anyway, you can use a function instead of r.row:
r.db('test').table('Things').filter(function(row) {
return r.expr(['CA', 'WA']).contains(row('state'))
})