I am working on an application that has messages and I want to store all the messages. But my problem is the message has a from first name and last name which could change. So if for example my JSON was
{
"subject": "Hello!",
"message": "Hello there",
"from": {
"user_id": 1,
"firstname": "George",
"lastname": "Lastgeorge"
}
}
The user could potentially change their last name or even first name. Which would require basically looping over every record in elasticsearch and updating everyone with the user_id.
Is there a better way to go about doing this?
I feel you should use parent mapping.
Keep the user info as parent with userID as key.
/index/userinfo/userID
{
"name" : "George",
"last" : "Lastgeorge"
}
Next , you need to maintain each chat as a child document and map the parent to the userindo type.
This way , whenever you want to make some change to the user information , simply make the change in userInfo type.
With this feature intact , you can search your logs based on user information , or search users based on chat records.
Link - http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/parent-child.html
Related
I am saving logs to Elasticsearch for analysis but I found there are mixed types of a particular field which causing error when indexing the document.
For example, I may save below log to the index where uuid is an object.
POST /index-000001/_doc
{
"uuid": {"S": "001"}
}
but from another event, the log would be:
POST /index-000001/_doc
{
"uuid": "001"
}
the second POST will fail because the type of uuid is not an object. so I get this error: object mapping for [uuid] tried to parse field [uuid] as object, but found a concrete value
I wonder what the best solution for that? I can't change the log because they are from different application. The first log is from the data of dynamodb while the second one is the data from application. How can I save both types of logs into ES?
If I disable dynamic mapping, I will have to specify all fields in the index mapping. For any new fields, I am not able to search them. so I do need dynamic mapping.
There will be many cases like that. so I am looking for a solution which can cover all conflict fields.
It's perfectly possible using ingest pipelines which are run before the indexing process.
The following would be a solution for your particular use case, albeit somewhat onerous:
create a pipeline
PUT _ingest/pipeline/uuid_normalize
{
"description" : "Makes sure uuid is a hash map",
"processors" : [
{
"script": {
"source": """
if (ctx.uuid != null && !(ctx.uuid instanceof java.util.HashMap)) {
ctx.uuid = ['S': ctx.uuid]; // hash map init
}
"""
}
}
]
}
run the pipeline when ingesting a new doc
POST /index-000001/_doc
{
"uuid": {"S": "001"}
}
POST /index-000001/_doc?pipeline=uuid_normalize <------
{
"uuid": "001"
}
You could now extend this to be as generic as you like but it is assumed that you know what you expect as input in each and every doc. In other words, unlike dynamic templates, you need to know what you want to safeguard against.
You can read more about painless script operators here.
You just cannot.
You should either normalize all your field in a way or another.
Or use 2 separate field.
I can suggest to use a field like this :
"uuid": {"key": "S", "value": "001"}
and skip the key when not necessary.
But you will have to preprocess your value before ingestion.
I am trying to attach a contact to campaign response.
I am using rest API for that.
https://learn.microsoft.com/en-us/dynamics365/customer-engagement/web-api/campaignresponse?view=dynamics-ce-odata-9
Post Data :
{
"firstname": "TestFirst",
"lastname": "TestLast",
"emailaddress": "test#test.com",
"telephone": "1234567890",
"prioritycode": 0,
"responsecode": 1,
"subject": "Test Subject",
"statuscode": 1,
"regardingobjectid_campaign#odata.bind": "/campaigns(xxxx90c-11ef-e811-a966-000d3ad24a0d)",
"regardingobjectid_contact#odata.bind": "/contacts(xxxxfa2e-c3b5-e811-a962-000d3ad24a0d)"
}
Here is my JSON.
I am getting Error : “Campaign as RegardingObject must be supplied”. Without contact, it works fine.
I had the same problem and the documentation is not very clear about it, I had to check all the relationships of the CampaignResponse in order to understand how to solve this.
If you want to create a CampaignResponse linked to both a Campaign and a Contact you need to do the following:
Create a CampaignResponse with the "regardingobjectid_campaign#odata.bind" in the params sent.
POST "https://some_subdomain.crm6.dynamics.com/api/data/v9.0/campaignresponses"
{
"regardingobjectid_campaign#odata.bind": "/campaigns(CAMPAIGN_ID_HERE)",
"description": "some desc",
"subject": "some subject "
}
Then find the CampaignResponse you just created to get its activityid (every CampaignResponse is an Activity)
Finally, you need to create a new ActivityParty, that will link the Contact to the CampaignResponse.
POST "https://some_subdomain.crm6.dynamics.com/api/data/v9.0/campaignresponses(CAMPAIGN_ID_HERE)/activitypointer_activity_parties"
{
"partyid_contact#odata.bind": "/contacts(CONTACT_ID_HERE)",
"participationtypemask": 11 //this is the code for customers
}
The "Regarding" lookup field can only be set to a single "regarding" record. Even though it appears that there are different Regarding fields, one for each entity type, those are "helper" fields that let you easily set the main Regarding field by setting one of those regardingobjectid_xxx fields.
You must choose to use either a campaign or a contact as your Regarding field. You can of course create other lookups, so you could use the Regarding field for your campaign and then add an additional Contact lookup field, for example.
I am developing google home supported api, here I have facing issue with session. I like to add some object values with response of conversation.
example : {last_intent: 'sampleintetn'}
I want to get back this data in request while user continue that same conversion.
I'm setting the contextOut parameter with an array of values such as
[
{
"name": "my_session_data",
"lifespan": 0,
"parameters": {
"myprop": "sample property",
"orbitaSession": {}
}
}
]
Contexts are slightly different than Alexa properties. The biggest difference that is relevant to what you're trying to do is that a Context can have a lifetime, expressed in number of user responses during the conversation.
A lifespan of 0 means to clear this Context. Sending the context again in your next response resets the lifetime counter.
Parameter values must also be strings, so you can't store another object in there. You can, however, convert that object to a string and store it, and convert it back to an object when you read it again later.
So something like this is more valid and will more likely do what you want:
[
{
"name": "my_session_data",
"lifespan": 5,
"parameters": {
"myprop": "sample property",
"orbitaSession": "{}"
}
}
]
I am looking to make some sort of "GenericModel" class extending Eloquent's Model class, that can load database configuration (like connection, table name, primary key column) as well as relationships at runtime based on a configuration JSON file.
My reasons for wanting this are as follows: I'm going to have a lot of database tables and thus a lot of models, but most don't really have any complicated logic behind them. I've developed a generic CRUD API and front-end interface to interact with them. Each model has a "blueprint" JSON file associated with it that describes things like its attributes and relationships. This lets me automatically generate, say, a view to create a new model and it knows what attributes I need to fill in, what input elements to use, what to label them, which are mandatory, how to validate, whether to check for uniqueness, etc. without ever needing code specific to that model. Here's an example, project.json:
{
"db_table": "projects",
"primary_key": "projectid",
"display_attr": "title", // Attribute to display when picking row from list, etc
"attributes": {
"projectid": { // Attribute name matches column name
"display": "ID", // Display this to user instead of db column name
"data_type": "integer" // Can be integer, string, numeric, bool...
},
"title": {
"data_type": "string",
"unique": true // Check for uniqueness when validating field
},
"customer": {
"data_type": "integer", // Data type of local key, matches customer PK
"relationship": { // Relationship to a different model
"type": "manytoone",
"foreign_model": "customer"
},
"user": "autocomplete" // User input element/widget to use, queries customer model for matches as user types
},
"description": {
"data_type": "string",
"user": "textarea" // Big string, use <textarea> for user input
"required": false // Can be NULL/empty, default true
}
},
"views": {
"table": [ // Show only these attributes when viewing table
"customer",
"title"
],
"edit_form": [ // Show these when editing
"customer",
"title",
"description"
],
...
}
}
This works extremely well on the front end, I don't need any more information than this to describe how my models behave. Problem is I feel like I just end up writing this all over again in most of my Model classes and it seems much more natural to have them just pull information from the blueprint file as well. This would result in the information being in one place rather than two, and would avoid extra effort and possible mistakes when I change a database table and only need to update one file to reflect it.
I'd really just like to be able to do something like GenericModel::blueprint('project.json')->find($id) and get a functioning "product" instance. Is this possible, or even advisable? Or is there a better way to do this?
Have you looked at Migrations (was Schema Builder)? It allows you to programatically build models (from JSON if necessary).
Then you could leverage Eloquent on your queries...
Looking for a little guidance in setting up a MongoDB schema. Here's the scenario:
I'm creating a save bookmark feature for people. In the DB, all I need to store is the username, a title, and a link. From this, I would need to create a service that outputs JSON and queries a particular bookmark or a person's entire feed. Which of the two set ups makes more sense from both an implementation and performance stand point?
A) Each bookmark is its own object:
{
"_id": ObjectId("abcd1234"),
"username": "Choy",
"title": "This is my first link",
"url": "http://www.google.com"
},
{
"_id": ObjectId("abcd1234"),
"username": "Choy",
"title": "This is my second link",
"url": "http://www.bing.com"
}
B) Each user is its own object:
{
"_id": "Choy",
"bookmarks": {
"abcd1234": {
"title": "This is my first link",
"url": "http://www.google.com"
},
"abcd12345": {
"title": "This is my second link",
"url": "http://www.bing.com"
}
}
}
Initially (A) made more sense to me, as I could easily query a specific bookmark, update, and remove it. But from the application point of view, (B) would be easier when I want to list all the bookmarks for a person as I could just do a findOne(username) on the _id instead of having to iterate through each record after doing a find(username) and convert to an array and then JSON (which I believe is a bit memory intensive).
On the other hand, it would be an extra step in (B) to add a new bookmark, as I would have to get the record, push a new bookmark into it and then save.
When you have a has-a relation in MongoDB, it is usually the best decision to embed the data in the object which owns it.
Your goal is to fulfill the needs of the user with as few searches as possible, because every single document lookup costs time. When you don't need all the bookmarks from a user but only specific ones, you can always use the dot notation to reach into objects and retrieve subsets of fields.
Aggregation instead of relation is also useful when you delete or rename a user. MongoDB can't do auto-cascade like SQL databases, so you have to deal with any orphaned data yourself in that case. But when the user document is self-contained, this won't be a problem.
So I would recommend you to go for solution B).