I am looking to make some sort of "GenericModel" class extending Eloquent's Model class, that can load database configuration (like connection, table name, primary key column) as well as relationships at runtime based on a configuration JSON file.
My reasons for wanting this are as follows: I'm going to have a lot of database tables and thus a lot of models, but most don't really have any complicated logic behind them. I've developed a generic CRUD API and front-end interface to interact with them. Each model has a "blueprint" JSON file associated with it that describes things like its attributes and relationships. This lets me automatically generate, say, a view to create a new model and it knows what attributes I need to fill in, what input elements to use, what to label them, which are mandatory, how to validate, whether to check for uniqueness, etc. without ever needing code specific to that model. Here's an example, project.json:
{
"db_table": "projects",
"primary_key": "projectid",
"display_attr": "title", // Attribute to display when picking row from list, etc
"attributes": {
"projectid": { // Attribute name matches column name
"display": "ID", // Display this to user instead of db column name
"data_type": "integer" // Can be integer, string, numeric, bool...
},
"title": {
"data_type": "string",
"unique": true // Check for uniqueness when validating field
},
"customer": {
"data_type": "integer", // Data type of local key, matches customer PK
"relationship": { // Relationship to a different model
"type": "manytoone",
"foreign_model": "customer"
},
"user": "autocomplete" // User input element/widget to use, queries customer model for matches as user types
},
"description": {
"data_type": "string",
"user": "textarea" // Big string, use <textarea> for user input
"required": false // Can be NULL/empty, default true
}
},
"views": {
"table": [ // Show only these attributes when viewing table
"customer",
"title"
],
"edit_form": [ // Show these when editing
"customer",
"title",
"description"
],
...
}
}
This works extremely well on the front end, I don't need any more information than this to describe how my models behave. Problem is I feel like I just end up writing this all over again in most of my Model classes and it seems much more natural to have them just pull information from the blueprint file as well. This would result in the information being in one place rather than two, and would avoid extra effort and possible mistakes when I change a database table and only need to update one file to reflect it.
I'd really just like to be able to do something like GenericModel::blueprint('project.json')->find($id) and get a functioning "product" instance. Is this possible, or even advisable? Or is there a better way to do this?
Have you looked at Migrations (was Schema Builder)? It allows you to programatically build models (from JSON if necessary).
Then you could leverage Eloquent on your queries...
Related
I'm an absolute FHIR newbie and I'm trying to create a set of StructureDefinitions and examples for an upcoming medical project.
For this project, we need a very specific resource, which is not supported by any FHIR resource yet. Here's our use case:
We are placing sensors on our Patients while they execute certain exercises (e.g. a leg squat) - we capture the sensor measurements and based on those we assign a pre-calculated bio-mechanical body model to each individual Patient. Those body models are calculated and assigned somewhere else in our system (this process is not relevant here). In a first step, I would like to add all the pre-calculated body models itself to our FHIR dataset as resources - so that I'm able to output all existing body models in our system.
Such a body model consists of an unique identifier, a human readable title and a set of attributes which describe the body model. The crucial part are the attributes - those might vary for each body model and we don't know the set of possible attributes beforehand, hence I need a dynamic format representing key and value of each attribute. If I were to represent this in a simple json structure I'd look as follows:
{
"id": "0",
"title": "SAMPLE_BODY_MODEL",
"attributes": [
{
"key": "ATTRIBUTE_1",
"value": "EXAMPLE_1"
},
{
"key": "ATTRIBUTE_2",
"value": "EXAMPLE_2"
}
]
}
My goal now is to create a StructureDefinition corresponding to the custom resource I've described above.
Hence I looked up the topic of "custom resources" and found this article on the HL7 site: https://hl7.org/fhir/basic.html - explaining that the Basic resource should be used for custom resources.
So I went ahead and tried to create a basic resource and extending it:
{
"resourceType": "StructureDefinition",
...
"type": "Basic",
"differential": {
"element": [
{
"id" : "Basic",
"path": "Basic",
"definition": "This element describes a general body model captured during an exercise or a movement, e.g. whilst doing leg squats."
},
{
"id" : "Basic.id",
"path": "Basic.id",
"definition": "ID of the body model"
}
{
"id": "Basic.extension:title",
"path": "Basic.extension",
"sliceName": "definition",
"definition": "Title of the body model",
"min": 0,
"max": "1",
"type": [
{
"code": "string" // I know that's wrong, but I somehow would like to restrict this to a string only
}
]
},
{
"id": "Basic.extension:attributes",
"path": "Basic.extension",
"sliceName": "attributes",
"definition": "Attributes of the body model",
// This is where I'm stuck - how do I define this to be a list of objects consisting of attributes key and value?
}
]
}
}
To sum it all up: How do I create a new StructureDefinition from a basic resource allowing me to specify a new required attribute named "attributes", which consists of one-to-many elements, which again contain the attributes key and value for the key and value of the body model attributes?
Hope this makes sense - otherwise please feel free to let me know and I'll try to rephrase my question.
Many thanks in advance!
First, for a newbie, you're doing really well :) (And nice job on framing the question well too!)
Your first extension slice has a few issues:
sliceName should be "title", not "definition" - essentially the 'extra' bit in the id is the slicename
The 'type' needs to be Extension. (The type of all extensions is always Extension.) However, you should also specify a specific profile on Extension that indicates the canonical URL the StructureDefinition you've used to define the 'title' extension. That extension will have a context of Basic and will constrain extension.value[x] to be of type string and will also establish a fixed URL for extension.url.
Your second slice will be similar. However, the profile on extension it points to won't constrain extension.value. Instead, it'll slice extension.extension to have two slices, one with a fixed url of "name" and the other with a fixed url of "value". There's an example here of a 2-element complex extension. Your slice names and data types will differ, as will the context, but it should make a good model for you.
If you still have issues, add your revised version to your question and we'll see if we can help further.
We are a multi-tenant platform.
The platform has a construct called Entity.
Users can create entities to model any real-life object eg: Customers, Orders, Payment, Inventory, Cart, pretty much anything.
Each entity will have its set of attributes, for example, a customer entity can have: name, email, phone, address (can be another nested entity), etc.
The requirement is to provide query/OLAP capabilities on these entities. For example, find all customers where name = 'john'.
The requirement includes all types of queries such as DATE RANGE, CONTAINS, LIKE, NUMERIC RANGE, FULL-TEXT Queries, etc. We also need Sorting, Aggregation, Pagination features.
Current design
We use elasticsearch to store entity data.
Each tenant is assigned a separate index.
When an entity is created in a tenant, the corresponding mappings are created inside the associated index. The mappings have roughly the following form:
{
"properties": {
"Customer": {
"properties": {
"name": {
"values": {
"type": "text"
}
},
"age": {
"values": {
"type": "integer"
}
}
}
},
"Order": {
"properties": {
"id": {
"values": {
"type": "text"
}
},
"eta": {
"values": {
"type": "integer"
}
}
}
}
//... other entities of this tenant
}
}
Major Problems with this design
Ever-growing mappings.
Frequent updates to mappings and hence the nodes are busy circulating cluster update information, leading to search/indexing latencies and occasional timeouts.
Existing mappings can't be altered if required. We have to go for the entire re-index procedure.
The current design was able to serve us for a few years until recently when the issues started popping up.
What would be a good design to model the above multi-tenancy requirement? Which database solution and schema modeling will be appropriate?
If you decide to stick with ES, you'll need to give your tenants more than one index, i.e. one index per tenant/entity instead of one index per tenant. The reasons for this are exactly the ones you mentioned, i.e. ever-growing mapping and the difficulty to update existing mappings.
You'll end up with more indexes for sure (N tenants x M entities) and the challenge will be to properly size those indexes in terms of how many primary shards you need for each of their entities. But I think it's even more difficult now that all entities are stored in a single index per tenant, so it'll turn out to be easier to fan out your tenant index into several.
Another option is to come up with a very generic mapping, which only contains typed fields like int_field_1, int_field_2, text_field_1, text_field_2, etc, and you keep a per-tenant mapping between the generic field names and the tenant specific field names:
name -> text_field_1
age -> int_field_1
...
That way you've less mappings to manage, it's more flexible in terms of what kind of data you can accommodate, but it comes at the price of keeping the above field mapping up-to-date.
In any case, you need to end up having more indexes for your tenants in order to make it easier to manage their mappings and keep their size at bay. it will also allow you to scale better, because it's easier to spread several smaller indices over your data nodes, than very big indexes, especially given the new sizing recommendation made by Elastic.
I'm writing a controller that handles an array of object, something like:
#PostMapping("/post")
public void saveEmployeeArray(#RequestBody Emoloyee[] employeeArray)
{
// Method body
}
Keep in mind that, in this case, employees are usually created, update, or deleted in bulk, meaning, I usually need to handle multiple employees at the same time.
A colleague told me that although it works I should only handle one employee in the controller as, according to him, it's rest best practice. But he didn't present an alternative to the issue of having to handle multiple employees most of the time, other than making multiple requests.
My question is, how is the best way to handle multiple objects?
If your list of Employee is inside another data structure (e.g. company), you can offer PATCH operation for outer resource to add or change a list of Employees. JSON PATCH is specified in RFC 6902, see https://datatracker.ietf.org/doc/html/rfc6902/
The JSON body of a PATCH request would look like this:
[
{ "op": "add", "path": "/employees", "value": { "name": "Employee 1", ...} },
{ "op": "add", "path": "/employees", "value": { "name": "Employee 2", ...} },
...
]
The body is a list of PATCH operations to change the addressed data structure at several places within one request, so it fits to your requirement.
The specification in RFC 6902 offers more than just adding elements, you can also remove, replace, move, copy and test. Have a look in the RFC for good and simple examples.
I've been taking baby steps into using Elasticsearch, and while researching a separate issue I ran into this question. Here, swatkins asked about querying nested objects, and a responder pointed out that nested objects weren't necessary given his model. I've copied the model here, and made some changes to reflect my particular question:
[{
id:4635,
description:"This is a test description",
author:"John",
author_id:51421,
meta: {
title:"This is a test title for a video",
description:"This is my video description",
url:"/url_of_video"
awesomeness-level: "pretty-awesome"
kung-fu: true
}
},
{
id:4636,
description:"This is a test description 2",
author:"John",
author_id:51421,
meta: {
title:"This is an example title for a video",
description:"This is my video description2",
url:"/url_of_video2"
kung-fu:false
monsters:true
monsters-present: "Dracula, The Mummy"
}
}]
Our application allows users to define custom metadata, so we're using a nested object to represent that data. At first glance, it looks similar to swatkins' model, so I thought that maybe we shouldn't be using a nested object.
The big difference is each objects meta might be different, note the second video has meta specifically about "monster movies", while the first video references an "awesomeness-level". So, should I be using a nested object, or just mapping metadata as a normal field? If we do the latter, will the first video have empty metadata fields? Does that really even matter? Thanks in advance!
Assuming that your example represents two elasticsearch documents, it doesn't look like you need to make meta a nested object. It makes sense to use nested objects when one parent object has multiple nested objects and your searches involve several fields of the nested objects. For example, if you have a record like this:
{
"name": "apple",
"attributes": [
{
"color": "yellow",
"size": "big"
},
{
"color": "red",
"size": "small"
}
]
}
and you want this record to be found when you search for color:yellow AND shape:big or color:red AND shape:small but don't want it to be returned when you search for color:yellow AND shape:small, it makes sense to make attributes a nested object. It will allow you to index and search each attribute independently and then get parent object of matching attribute.
Looking for a little guidance in setting up a MongoDB schema. Here's the scenario:
I'm creating a save bookmark feature for people. In the DB, all I need to store is the username, a title, and a link. From this, I would need to create a service that outputs JSON and queries a particular bookmark or a person's entire feed. Which of the two set ups makes more sense from both an implementation and performance stand point?
A) Each bookmark is its own object:
{
"_id": ObjectId("abcd1234"),
"username": "Choy",
"title": "This is my first link",
"url": "http://www.google.com"
},
{
"_id": ObjectId("abcd1234"),
"username": "Choy",
"title": "This is my second link",
"url": "http://www.bing.com"
}
B) Each user is its own object:
{
"_id": "Choy",
"bookmarks": {
"abcd1234": {
"title": "This is my first link",
"url": "http://www.google.com"
},
"abcd12345": {
"title": "This is my second link",
"url": "http://www.bing.com"
}
}
}
Initially (A) made more sense to me, as I could easily query a specific bookmark, update, and remove it. But from the application point of view, (B) would be easier when I want to list all the bookmarks for a person as I could just do a findOne(username) on the _id instead of having to iterate through each record after doing a find(username) and convert to an array and then JSON (which I believe is a bit memory intensive).
On the other hand, it would be an extra step in (B) to add a new bookmark, as I would have to get the record, push a new bookmark into it and then save.
When you have a has-a relation in MongoDB, it is usually the best decision to embed the data in the object which owns it.
Your goal is to fulfill the needs of the user with as few searches as possible, because every single document lookup costs time. When you don't need all the bookmarks from a user but only specific ones, you can always use the dot notation to reach into objects and retrieve subsets of fields.
Aggregation instead of relation is also useful when you delete or rename a user. MongoDB can't do auto-cascade like SQL databases, so you have to deal with any orphaned data yourself in that case. But when the user document is self-contained, this won't be a problem.
So I would recommend you to go for solution B).