how to store nested fields in elasticsearch - elasticsearch

I am using Java High Level REST client. Here is the link to its documentation
I have created a client.
trait HighLevelRestClient {
def elasticSearchClient(): RestHighLevelClient = {
new RestHighLevelClient(
RestClient.builder(
new HttpHost("localhost", ElasticSearchPort, "http")))
}
}
While indexing the data, the nested fields are being stored as String. The following code explains how the index is being created:
val indexRequest = new IndexRequest("my-index", "test-type").source(
"person", person,
"date", DateTime.now()
)
where, person is a case class, represented as:
Person(personId: String, name: String, address: Address)
and Address is itself a case class, represented as:
Address(city: String, zip: Int)
My application requires person to be stored as key-value pair, so that it's fields are searchable. But, when I am using the above code, it is being stored as String.
{
"person" : "Person(my-id, my-name, Address(my-city, zip-value))",
"date" : "2017-12-12"
}
and required structure is:
{
"person" : {
"personId" : "my-id",
"name" : "person-name",
"address": {
"city" : "city-name",
"zip" : 12345
}
},
"date" : "2017-12-12"
}
I hope I have framed the question well. Any help would be appreciated. Thanks!

You are almost there. To achieve your goal you need to:
Serialize the object to JSON on your side
Specify the content type of the request
It is actually described in the page of the Index API.
A convenient library to serialize case classes into JSON is for example json4s (you can see some examples of serialization here).
Your code might look like the following:
import org.apache.http.HttpHost
import org.elasticsearch.action.index.IndexRequest
import org.elasticsearch.client.{RestClient, RestHighLevelClient}
import org.elasticsearch.common.xcontent.XContentType
import org.joda.time.DateTime
import org.json4s.NoTypeHints
import org.json4s.jackson.Serialization
import org.json4s.jackson.Serialization.write
case class Address(city: String, zip: Int)
case class Person(personId: String, name: String, address: Address)
case class Doc(person: Person, date: String)
object HighClient {
def main(args: Array[String]): Unit = {
val client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("localhost", 9206, "http")))
implicit val formats = Serialization.formats(NoTypeHints)
val doc = Doc(
Person("blah1", "Peter Parker", Address("New-York", 33755)),
DateTime.now().toString
)
val indexRequest = new IndexRequest("my-index", "test-type").source(
write(doc), XContentType.JSON
)
client.index(indexRequest)
client.close()
}
}
Note that in this case:
new IndexRequest("my-index", "test-type").source(
write(doc), XContentType.JSON
)
this function will be used: public IndexRequest source(String source, XContentType xContentType)
While in your case:
new IndexRequest("my-index", "test-type").source(
"person", person,
"date", DateTime.now()
)
it will call public IndexRequest source(Object... source).
Hope that helps!

Related

Kotlin OpenapiGenerator Any type generates into Map<String, JsonObject>

I'm struggling to find correct way to define Any object in Kotlin so that OpenApiGenerator would generate it as Object type.
I have a simple DTO object, payload is basically a map of object fields and values:
data class EventDto(
val payload: Map<String, Any>,
...other fields
)
Which gets converted to OpenAPI Spec and looks like this:
ommited code
...
"payload": {
"type": "object",
"additionalProperties": {
"type": "object"
}
},
But when I execute open-api-generator
this get's converted into Kotlin class looking like this:
#Serializable
public data class EventPayloadDto(
#SerialName(value = "payload")
val payload: kotlin.collections.Map<kotlin.String, kotlinx.serialization.json.JsonObject>? = null,
)
Which is not so nice convert into because each object value needs to be converted to JsonObject, is it possible to retain "Any" object when generating from OpenAPI docs or I must use Map<String, String>?
I tried using objectMapper.convert
objectMapper.convertValue(event, object : TypeReference<Map<String, JsonObject>>() {})
but since there are no serializers into JsonObject it had no effect and ended up in an error.

passing json in json to spring controller

I am trying to pass json object to spring controller and I manage to do that, but value of one property is in json and I think that I have problem because of it. But there is no other way to pass that data. Code is below,
data class:
#Entity
data class Section(
#Id
#GeneratedValue
val id: Long = 0L,
val name: String = "",
var text: String,
#ManyToOne
var notebook: Notebook
)
Controller code:
#PutMapping("/sections/{id}")
fun updateSection(#RequestBody section: Section, #PathVariable id: Long): Section =
sectionRepository.findById(id).map {
it.text = section.text
it.notebook = section.notebook
sectionRepository.save(it)
}.orElseThrow { SectionNotFoundException(id) }
javascript sending post to api:
function updateApi(data) {
axios.put(MAIN_URL + 'sections/' + data.id, {
data
})
.then(showChangesSaved())
.catch(ShowErrorSync());
}
function saveSection() {
var data = JSON.parse(window.sessionStorage.getItem("curr-section"));
data.text = JSON.stringify(element.editor).toString();
updateApi(data);
}
I get error like this:
2020-11-18 15:06:24.052 WARN 16172 --- [nio-8080-exec-2] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Instantiation of [simple type, class org.dn.model.Section] value failed for JSON property text due to missing (therefore NULL) value for creator parameter text which is a non-nullable type; nested exception is com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class org.dn.model.Section] value failed for JSON property text due to missing (therefore NULL) value for creator parameter text which is a non-nullable type
at [Source: (PushbackInputStream); line: 1, column: 375] (through reference chain: org.dn.model.Section["text"])]
so text in element.editor is JSON formatted string and I need to pass it as it is to controller. Is there any way to do that? I tried searching, but I can't find json in json help...
Whole project is available on github
What does your json looks like? If I check out your project and run the following two tests:
one with Section as an object as request body
one with Section as json
Both will succeed. So the problem might lie in your JSON:
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
class HttpRequestTest {
#LocalServerPort
private val port = 0
#Autowired
private val restTemplate: TestRestTemplate? = null
#Test
fun sectionAsObject() {
val section = Section(0L, "2L", "text", Notebook(1L, "1", "2"))
assertThat(restTemplate!!.put("http://localhost:$port/sections/123", section
)).isNotNull
}
#Test
fun sectionAsJson() {
val sectionAsJson = """
{
"id": 0,
"name": "aName",
"text": "aText",
"noteBook": {
"id": 0,
"name": "aName",
"desc": "2"
}
}
""".trimIndent()
assertThat(restTemplate!!.put("http://localhost:$port/sections/123", sectionAsJson
)).isNotNull
}
}
BTW: it is not a pretty good habit to expose your database ids, which is considered to be a security risk as it exposes your database layer. Instead, you might want to use a functional unique key ;)

Abstract object not mapped correctly in Elasticsearch using Nest 7.0.0-alpha1

I am using NEST (.NET 4.8) to import my data, and I have a problem getting the mapping to work in NEST 7.0.0-alpha1.
I have the following class structure:
class LinkActor
{
public Actor Actor { get; set; }
}
abstract class Actor
{
public string Description { get; set; }
}
class Person : Actor
{
public string Name { get; set; }
}
I connect to Elasticsearch this way:
var connectionSettings = new ConnectionSettings(new Uri(connection));
connectionSettings.DefaultIndex(indexName);
var client = new ElasticClient(connectionSettings);
The actual data looks like this:
var personActor = new Person
{
Description = "Description",
Name = "Name"
};
var linkActor = new LinkActor
{
Actor = personActor
};
And the data is indexed like this:
result = client.IndexDocument(linkActor);
Using NEST 6.6 I am getting the following data in Elasticsearch 6.5.2:
"actor": {
"name": "Name",
"description": "Description"
}
However when using NEST 7.0.0-alpha1 I get the following data in Elasticsearch 7.0.0:
"actor": {
"description": "Description"
}
So the data from the concrete class is missing. I am obviously missing / not understanding some new mapping feature, but my attempts with AutoMap has failed:
client.Map<(attempt with each of the above classes)>(m => m.AutoMap());
Is is still possible to map the data from the concrete class in NEST 7.0.0-alpha1?
I found a workaround using the NEST.JsonNetSerializer (remember to install this), which allows me to pass a JObject directly:
Connect to Elasticsearch using a pool so you can add the JsonNetSerializer.Default:
var pool = new SingleNodeConnectionPool(new Uri(connection));
var connectionSettings = new ConnectionSettings(pool, JsonNetSerializer.Default);
connectionSettings.DefaultIndex(indexName);
var client = new ElasticClient(connectionSettings);
Convert the linkActor object from above to a JObject (JsonSerializerSettings omitted for clarity, add them to get CamelCasing):
var linkActorSerialized = JsonConvert.SerializeObject(linkActor);
var linkActorJObject = JObject.Parse(linkActorSerialized);
result = client.IndexDocument(linkActorJObject);
This gives the desired result:
"actor": {
"name": "Name",
"description": "Description"
}
It is a workaround, hopefully someone will be able to explain the mapping in the question.

How to return nested objects in GraphQL schema language

I was going through the documentation for GraphQl and realized that the new Schema Langugage supports only default resolvers. Is there a way I can add custom resolvers while using the new Schema Language?
let userObj = {
id: 1,
name: "A",
homeAddress: {
line1: "Line1",
line2: "Line2",
city: "City"
}
};
let schema = buildSchema(`
type Query {
user(id: ID): User
}
type User {
id: ID
name: String
address: String
}
`);
//I would like User.address to be resolved from the fields in the json response eg. address = Line1, Line2, City
This is the schema that I have defined. I would like to add some behavior here that would allow me to parse the address object and return a concatenated string value.
As mentioned by HagaiCo and in this github issue, the right way would be to go with graphql-tools.
It has a function called makeExecutableSchema, which takes a schema and resolve functions, and then returns an executable schema
It seems like you have a confusion in here, since you defined that address is String but you send a dictionary to resolve it.
what you can do, is to define a scalar address type:
scalar AddressType if you use buildSchema and then attach parse functions to it. (or use graphql-tools to do it easily)
or build the type from scratch like shown in the official documentations:
var OddType = new GraphQLScalarType({
name: 'Odd',
serialize: oddValue,
parseValue: oddValue,
parseLiteral(ast) {
if (ast.kind === Kind.INT) {
return oddValue(parseInt(ast.value, 10));
}
return null;
}
});
function oddValue(value) {
return value % 2 === 1 ? value : null;
}
and then you can parse the dictionary into a string (parseValue) and otherwise

MongoDB how update element in array using Spring Query Update

In my project I'm using SpringBoot 1.3.2 and org.springframework.data.mongodb.core.query.*
I'm trying to update element in array, in my main object i have array looking like this:
"sections" : [
{
"sectionId" : "56cc3c908f5e6c56e677bd2e",
"name" : "Wellcome"
},
{
"sectionId" : "56cc3cd28f5e6c56e677bd2f",
"name" : "Hello my friends"
}
]
Using Spring I want to update name of record with sectionId 56cc3c908f5e6c56e677bd2e
I was trying to to this like that but it didn't work
Query query = Query.query(Criteria
.where("sections")
.elemMatch(
Criteria.where("sectionId").is(editedSection.getId())
)
);
Update update = new Update().set("sections", new BasicDBObject("sectionId", "56cc3c908f5e6c56e677bd2e").append("name","Hi there"));
mongoTemplate.updateMulti(query, update, Offer.class);
It create something like:
"sections" : {
"sectionId" : "56cc3c908f5e6c56e677bd2e",
"name" : "Hi there"
}
But this above is object { } I want an array [ ], and I don't want it remove other elements.
Can any body help me how to update name of record with sectionId 56cc3c908f5e6c56e677bd2e using Spring
You essentially want to replicate this mongo shell update operation:
db.collection.update(
{ "sections.sectionId": "56cc3c908f5e6c56e677bd2e" },
{
"$set": { "sections.$.name": "Hi there" }
},
{ "multi": true }
)
The equivalent Spring Data MongoDB code follows:
import static org.springframework.data.mongodb.core.query.Criteria.where;
import static org.springframework.data.mongodb.core.query.Query;
import static org.springframework.data.mongodb.core.query.Update;
...
WriteResult wr = mongoTemplate.updateMulti(
new Query(where("sections.sectionId").is("56cc3c908f5e6c56e677bd2e")),
new Update().set("sections.$.name", "Hi there"),
Collection.class
);
Can use BulkOperations approach to update list or array of document objects
BulkOperations bulkOps = mongoTemplate.bulkOps(BulkMode.UNORDERED, Person.class);
for(Person person : personList) {
Query query = new Query().addCriteria(new Criteria("id").is(person.getId()));
Update update = new Update().set("address", person.setAddress("new Address"));
bulkOps.updateOne(query, update);
}
BulkWriteResult results = bulkOps.execute();
Thats my solution for this problem:
public Mono<ProjectChild> UpdateCritTemplChild(
String id, String idch, String ownername) {
Query query = new Query();
query.addCriteria(Criteria.where("_id")
.is(id)); // find the parent
query.addCriteria(Criteria.where("tasks._id")
.is(idch)); // find the child which will be changed
Update update = new Update();
update.set("tasks.$.ownername", ownername); // change the field inside the child that must be updated
return template
// findAndModify:
// Find/modify/get the "new object" from a single operation.
.findAndModify(
query, update,
new FindAndModifyOptions().returnNew(true), ProjectChild.class
)
;
}

Resources