For example we have: data_receipt with id = Ft9SKhvk7YTgMtZBQt741RHRpF9GDtrVUXbm7Txr1GYK
This receipt we can find only in chunk. We can easily match it with block, but not with transactions.
Transactions does not have data_receipt.
Can anyone explain how I can match it with transaction?
I found how it work near indexer github
if let Some(transaction_hash) =
tx_hashes_for_receipts.get(&receipt_or_data_id) {
Some(models::Receipt::from_receipt_view(
r,
block_hash,
transaction_hash,
chunk_hash,
index as i32,
block_timestamp,
))
}
It's mean that receipt is data, receipt have data_id in core of tree. And some transaction have receipt with field data_id included to output_data_receivers
data_id from output_data_receivers equal to data_id from data_receipt
data receipt: Ft9SKhvk7YTgMtZBQt741RHRpF9GDtrVUXbm7Txr1GYK with data_id : HWCSA1gNd7NviQ4ZmqNZ7P6fy5RA93NGbXjmQSNWpJxc
{
"jsonrpc": "2.0",
"id": "dontcare",
"method": "chunk",
"params": {
"chunk_id": "DBTgMT3NMnkof9pX5Lg1dZvfBB6REJD3vibEQThkjnWe"
}
}
transaction getting in parsing process: EToG7FhgJyDBg1VbFhyRSSxYTZ9SGPjy1tVnbBTPbdth this transaction have receipt : 5VDF57ehUt43oZ2yfULj8QL9wR9aojEXUkjD4XQB7rj2
{
"jsonrpc": "2.0",
"id": "dontcare",
"method": "EXPERIMENTAL_tx_status",
"params": [
"EToG7FhgJyDBg1VbFhyRSSxYTZ9SGPjy1tVnbBTPbdth",
"9nearapps.near"
]
}
this receipt have output_data_receivers and in tree have data_id: HWCSA1gNd7NviQ4ZmqNZ7P6fy5RA93NGbXjmQSNWpJxc
"output_data_receivers": [
{
"data_id": "HWCSA1gNd7NviQ4ZmqNZ7P6fy5RA93NGbXjmQSNWpJxc",
"receiver_id": "v1.nearapps.near"
}
]
Ps. Sorry for my english
Related
When using GraphQL federated resolvers to resolve an array, and one of the resolved items cannot be found:
Observed behaviour: null data returned for entire query, no error message.
Desired behaviour: the item which cannot be resolved is silently dropped from results array.
What works
Using Apollo Server federation, we are successfully able to resolve the following query:
query {
products {
name
reviews {
id
score
}
}
}
where products comes from the Product subgraph and score is resolved by the Review subgraph.
This works fine when all the review ids passed to the Review subgraph are resolvable.
Our Problem
Sometimes the Review subgraph deems that a review should not be returned. The business case could be, for example, it was automatically marked as spam and is blocked until a manual check is done.
In this situation, the Review subgraph returns null for this ID. Example query:
query {
_entities(representations: [
{id: 1, __typename: "Review"},
{id: 2, __typename: "Review"}
]) {
...on Review {
id
score
}
}
}
Result
{
"data": {
"_entities": [
{
"id": "1",
"score": "94"
},
null
]
}
}
When this happens, we would want the federated results to contain the Product and only review 1, the non-resolveable review id 2 having been dropped from the array. Like this:
{
"data": {
"products": {
"name": "Phone cover",
"reviews": [
{
"id": "1",
"score": "94"
}
]
}
}
}
However, Apollo Server returns null for the entire query, with no error message:
{
"data": null
}
Is this expected behaviour? Is there any way to gain the result I want, or otherwise control how Apollo reacts when one item in a reference array is not resolved?
I need to query the partial fields for courseWork.list and courseWork.get so I am passing this value in fields as described in the documentation. But courseWork(individualStudentsOptions) API call returns a error:
{
"error": {
"code": 400,
"message": "field_mask: Unknown field mask values: individual_students_options",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "field_mask",
"description": "Unknown field mask values: individual_students_options"
}
]
}
]
}
}
In other experiments, for example courseWork(id) everything is fine and the API call returns this:
{
"courseWork": [
{
"id": "93359557635"
},
{
"id": "93359557700"
},
...
]
}
So please help me how to fill in the field of individualStudentsOptions ?
How exactly are you calling this api?
This is how we do it here:
String studentUser = "student#gmail.com";
IndividualStudentsOptions individualStudentsOptions = new IndividualStudentsOptions();
ArrayList<String> studentIdList = new ArrayList<>();
studentIdList.add(studentUser);
individualStudentsOptions.setStudentIds(studentIdList);
CourseWork courseWork = new CourseWork()
.setCourseId(course.getId())
.setTitle("My course work")
.setDescription("desc")
.setMaxPoints(100.0)
.setDueDate(date)
.setDueTime(timeOfDay)
.setAssigneeMode("INDIVIDUAL_STUDENTS")
.setIndividualStudentsOptions(individualStudentsOptions)
.setWorkType("ASSIGNMENT")
.setState("PUBLISHED");
courseWork = service.courses().courseWork().create(course.getId(), courseWork).execute();
I'm using Elasticsearch DSL, I'm trying to use a query result as a parameter for another query like below:
{
"query": {
"bool": {
"must_not": {
"terms": {
"request_id": {
"query": {
"match": {
"processing.message": "OUT Followup Synthesis"
}
},
"fields": [
"request_id"
],
"_source": false
}
}
}
}
}
}
As you can see above I'm trying to search for sources that their request_id is not one of the request_idswith processing.message equals to OUT Followup Synthesis.
I'm getting an error with this query:
Error loading data [x_content_parse_exception] [1:1660] [terms_lookup] unknown field [query]
How can I achieve my goal using Elasticsearch DSL?
Original question extracted from the comments
I'm trying to fetch data with processing.message equals to 'IN Followup Sythesis' with their request_id doesn't appear in data with processing.message equals to 'OUT Followup Sythesis'. In SQL language:
SELECT d FROM data d
WHERE d.processing.message = 'IN Followup Sythesis'
AND d.request_id NOT IN (SELECT request_id FROM data WHERE processing.message = 'OUT Followup Sythesis');
Answer: generally speaking, neither application-side joins nor subqueries are supported in Elasticsearch.
So you'll have to run your first query, take the retrieved IDs and put them into a second query — ideally a terms query.
Of course, this limitation can be overcome by "hijacking" a scripted metric aggregation.
Taking these 3 documents as examples:
POST reqs/_doc
{"request_id":"abc","processing":{"message":"OUT Followup Synthesis"}}
POST reqs/_doc
{"request_id":"abc","processing":{"message":"IN Followup Sythesis"}}
POST reqs/_doc
{"request_id":"xyz","processing":{"message":"IN Followup Sythesis"}}
you could run
POST reqs/_search
{
"size": 0,
"query": {
"match": {
"processing.message": "IN Followup Sythesis"
}
},
"aggs": {
"subquery_mock": {
"scripted_metric": {
"params": {
"disallowed_msg": "OUT Followup Synthesis"
},
"init_script": "state.by_request_ids = [:]; state.disallowed_request_ids = [];",
"map_script": """
def req_id = params._source.request_id;
def msg = params._source.processing.message;
if (msg.contains(params.disallowed_msg)) {
state.disallowed_request_ids.add(req_id);
// won't need this particular doc so continue looping
return;
}
if (state.by_request_ids.containsKey(req_id)) {
// there may be multiple docs under the same ID
// so concatenate them
state.by_request_ids[req_id].add(params._source);
} else {
// initialize an appendable arraylist
state.by_request_ids[req_id] = [params._source];
}
""",
"combine_script": """
state.by_request_ids.entrySet()
.removeIf(entry -> state.disallowed_request_ids.contains(entry.getKey()));
return state.by_request_ids
""",
"reduce_script": "return states"
}
}
}
}
which'd return only the correct request:
"aggregations" : {
"subquery_mock" : {
"value" : [
{
"xyz" : [
{
"processing" : { "message" : "IN Followup Sythesis" },
"request_id" : "xyz"
}
]
}
]
}
}
⚠️ This is almost guaranteed to be slow and goes against the suggested guidance of not accessing the _source field. But it also goes to show that subqueries can be "emulated".
💡 I'd recommend to test this script on a smaller set of documents before letting it target your whole index — maybe restrict it through a date range query or similar.
FYI Elasticsearch exposes an SQL API, though it's only offered through X-Pack, a paid offering.
I am new to elastic search spring data, Today I was trying to get In query working with Spring data ES repository.
I have to do a lookup for list of user names, and if its exactly match in the index, need to get those users back as result.
I tried to use the built in repository 'In' method to do so, but it returns partial matches, please help me to make this working like SQL IN query.
Here is my repository code:
public interface UserRepository extends ElasticsearchRepository<EsUser, String>
{
public List<EsUser> findByUserAccountUserNameIn(Collection<String> terms);
}
REQUEST:
{"terms":["vijay", "arun"], "type":"NAME"}
RESPONSE:
[
{
"userId": "236000",
"fbId": "",
"userAccount": {
"userName": "arun",
"urlFriendlyName": "arun",
},
"userProfile": {
},
"userStats": {
}
},
{
"userId": "6228",
"userAccount": {
"userName": "vijay",
"urlFriendlyName": "vijay",
},
"userProfile": {
},
"userStats": {
}
},
{
"userId": "236000",
"fbId": "",
"userAccount": {
"userName": "arun singh",
"urlFriendlyName": "arun-singh",
},
"userProfile": {
},
"userStats": {
}
}
{
"userId": "236000",
"fbId": "",
"userAccount": {
"userName": "vijay mohan",
"urlFriendlyName": "vijay-mohan",
},
"userProfile": {
},
"userStats": {
}
}
]
This is because your userAccount.userName field is an analyzed string, and thus, the two tokens arun and singh have been indexed. Your query then matches the first token, which is normal.
In order to prevent this and guarantee an exact match you need to declare your field as not_analyzed, like this:
#Field(index = FieldIndex.not_analyzed)
private String userName;
Then you'll need to delete your index and the associated template in /_template, restart your application so a new template and index are created with the proper field mapping.
Then your query will work.
I'm trying to decide upon the best format of response for my API. I need to return a reports response which provides information on the report itself and the fields contained on it. Fields can be of differing types, so there can be: SelectList; TextArea; Location etc..
They each use different properties, so "SelectList" might use "Value" to store its string value and "Location" might use "ChildItems" to hold "Longitude" "Latitude" etc.
Here's what I mean:
"ReportList": [
{
"Fields": [
{
"Id": {},
"Label": "",
"Value": "",
"FieldType": "",
"FieldBankFieldId": {},
"ChildItems": [
{
"Item": "",
"Value": ""
}
]
}
]
}
The problem with this is I'm expecting the users to know when a value is supposed to be null. So I'm expecting a person looking to extract the value from "Location" to extract it from "ChildItems" and not "Value". The benefit to this however, is it's much easier to query for things than the alternative which is the following:
"ReportList": [
{
"Fields": [
{
"SelectList": [
{
"Id": {},
"Label": "",
"Value": "",
}
]
"Location": [
{
"Id": {},
"Label": "",
"Latitude": "",
"Longitude": "",
"etc": "",
}
]
}
]
}
So this one is a reports list that contains a list of fields which on it contains a list of fieldtype for every fieldtype I have (15 or something like that). This is opposed to just having a list of reports which has a list of fields with a "fieldtype" enum which I think is fairly easy to manipulate.
So the Question: Which format is best for a response? Any alternatives and comments appreciated.
EDIT:
To query all fields by fieldtype in a report and get values with the first way it would go something like this:
foreach(field in fields)
{
switch(field.fieldType){
case FieldType.Location :
var locationValue = field.childitems;
break;
case FieldType.SelectList:
var valueselectlist = field.Value;
break;
}
The second one would be like:
foreach(field in fields)
{
foreach(location in field.Locations)
{
var latitude = location.Latitude;
}
foreach(selectList in field.SelectLists)
{
var value= selectList.Value;
}
}
I think the right answer is the first one. With the switch statement. It makes it easier to query on for things like: Get me the value of the field with the id of this guid. It just means putting it through a big switch statement.
I went with the first one because It's easier to query for the most common use case. I'll expect the client code to put it into their own schema if they want to change it.