I need to create a new table dynamically from within the code. The current proof of concept is currently working, but what I was given as POC has several issues and I have some questions about whether this is the best way. I cannot change the architecture, I can only implement as best I can.
Assume I have a controller called WorldController with accompanying Model WorldTable and accompanying views to list all Worlds and also add new Worlds. Also assume every user has their own universe, called $universe. Lastly, note that we are using Postgres and subsequently, schema.
Right now,I can add a new World to the Database in the World table and I can successfully create the required tables that the World requires, using $universe.'world_geography' syntax. So if $universe = 'milkyway', then that tables are milkyway.world_climate and milkyway.world_geography etc.
The additional tables are created in the WorldTable model by using SQL statements:
$sql = "
CREATE TABLE $universe.world_geography (
id serial4 NOT NULL,
landmasses_id int4 NOT NULL,
layers_id int4 NOT NULL,
materials_id int4 NOT NULL
);
";
$connection = ConnectionManager::get('default');
$submit = $connection->execute($sql);
This is obviously problematic due to SQL code injection. What would
be a better way to do this?
Second question: Seeing that this code is creating Tables that
aren't part of this model, should this code then rather be run from
the Controller?
And lastly, does CakePHP have any better way of doing this that I
might have missed?
Comment: I have read many, many other questions here on SO and around the net and none of them ask this question or have sufficient answers to my question.
Related
I have a Laravel 8 application and am wondering how to solve the problem of how to solve a typical polymorphic issue. I have an Employee model. That Employee can be an ExecutiveEmployee or EntryLevelEmployee. There will be methods an ExecutiveEmployee has that an EntryLevelEmployee doesn't have and the inverse is also true.
Using Laravel 8, is it right to create a base Employee model (without a corresponding table?) and then create two models named ExecutiveEmployee and EntryLevelEmployee that inherit from Employee? This would also imply that both employee types will have two different database tables, even though there will be a lot of overlapping data.
Does it make sense to just have one Employee model and create a migration that has the employee type listed in the model? I am assuming that it's ok if an EntryLevelEmployee has some database attributes which are relevant to it that may or may not be relevant to an ExecutiveEmployee type here, or is that an incorrect assumption?
What's the correct way to model this in Laravel 8? I prefer to keep everything in one table because of how similar the models are. I do have to keep in mind that there will be data that one has that the other doesn't. There will be different accessor methods as well.
Is it possible to have everything in one employees table while utilizing multiple models? Meaning, if I create two models named ExecutiveEmployee and EntryLevelEmployee they would both query the underlying table employees?
UPDATE 1
The more I research, the more I think polymorphism is the incorrect approach here and what I might need is Single-Table Inheritance. This package seems to bring the capability to Eloquent. Would there be a good reason to not use this?
I would use polymorphic relationships in this case, because you are more flexible and have less coupling.
Using the Single Table Inheritance (STI), you can add type specific columns in the employees table and make them nullable. But think about adding/removing types in the future.
executive_employees
id - integer
executive_specific - string
entry_level_employees
id - integer
entry_level_specific - string
employees
id - integer
name - string
email - string
employable_id - integer
employable_type - string
As for the STI the same would be
employees
id - integer
name - string
email - string
type - string
executive_specific - nullable string
entry_level_specific - nullable string
So STI would be suitable when you don't have type specific columns. But you want to add specific behavior in your code. For example a User type (Admin, Author).
Even so, it's a matter of preferences.
It really depends on the state and behavior of your employee object.
Below are few points I will consider to make a decision
If your objects' states/properties are different then definitely you will create different models as your data will be stored in different tables.
If most states/properties are same and some are different, you can
consider storing all in one table/model and for the difference in
behavior create separate table like Ron Van Der Heijden has
suggested and you can consider query scope with that to make
transaction with database.
And another view will be
How many JOINs you will create if you will create different tables,
will that impact the performance and other stuffs, will it make your
code complex?
Can you make simpler relations and handle stuffs independently?
When you are making an API, will your
code make the api overworking? or you need to create too many request
for any operation?
These stuffs will decide how you will make a decision.
Update 1:
Another point I would like to add about the package you are thinking to use, consider using a parent key in table and you can define relationships in a single model.I do not think you need to use a package, you can define it yourself, I guess.
I don't understand why you don't create a simple one-to-many relation. Based on the information you provided, the polymorphic relation looks unnecessary. I think the right way is to create employee_roles table and relations. Then you can give different permissions to different employee types. There are several ways to do that. You can create a middleware to create route restrictions. You can check the role before executing a function in the controller, and run only if the employee has permission. You can use if-else in blade not to render the parts that can't be used by auth user etc.
If you have different “types” of employees, and each employee type should have different logic then yeah, that sounds like a polymorphic relationship.
is there any method for inserting new record or update when record exists in database using DB facade ?
Note: there is no model but only a controller , so using model related methods is impossible.my question is regards to query builder
The most straightforward approach is to look into your database's syntax for upserting. It will be different depending on the database and the specific situation, and may require defining a UNIQUE key of some kind.
Here is an example for MySQL:
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1;
By the way, the more idiomatic approach would be to define a model. Model's allow you to handle this situation more gracefully and (in many cases) prevent you from tying your code to a specific RDBMS.
I need to load model validations from a table and validate my model. e.g. I have a database table called validations, which has rows like :
validation_action validation_condition
---------------- --------------------
validates_presence_of if answer_name is name
validates_format_of if answer_type is date
In my model I want:
class Model < ActiveRecord::Base
load validation_actions , lambda {if validation_condition is true}
Ok more detail:
I am creating an app for taking surveys. I am storing questions in a table and answers in another table. I need to store the validation for each answer in the question table and validate each answer before I accept it. I can query validation for each question and run it in controller but I want to do it in model instead as it is much cleaner.
so two models :
Questions -> table questions sas code and details about questions
Answers -> table answers stores answers with a foreign key to Questions.
I want to validate the input in Answers model depending upon conditions defined in questions database table.
Please let me know if more detail is needed?
If I understand correctly, you have Questions which have many associated Answers, and also have associated ValidationActions. When an Answer is being saved, you want to run validations against it based on evaluating code stored in the ValidationActions belonging to it's associated Question object.
This seems like a bad idea. Not so much that validations are conditional on the state of an associated object (the Question), which is the sort of thing that does happen, but that you're doing what amounts to executing code from the contents of the database. This is one of those things that makes web developers twitchy - the security issue is the most obvious reason (that if someone can edit your database's fields, they can cause arbitrary code to run on your system), but it's definitely a questionable practice even outside that (I'm having trouble articulating the reason more convincingly than 'code is code and data is data and never the twain shall meet', but there it is).
All that said, it should be possible to write a custom validator as explained here to handle this. It will doubtless be inefficient, might well be prone to breakage, and is probably a security hole unless it's written very carefully. But it's possible.
I trying to use Grails Scaffolding to throw a quick CRUD application together around some legacy database tables. It is an Oracle database, and the primary key value is intended to be populated by Oracle's GUID() function.
Based on this earlier StackOverflow question, I tried specifying "guid" as the Hibernate generator for this column in my Grails domain class:
...
static mapping = {
table name: "OWNER"
version false
columns {
id column: "OWNER_OID", generator: "guid"
name column: "NAME"
...
}
}
...
When I run my Grails app, viewing and even editing records works just fine. However, when I try to create a new record, things blow up with the Oracle error message "ORA-02289: sequence does not exist".
I enabled SQL logging for my datasource, and see Grails/Hibernate trying to execute the following during a save operation:
select hibernate_sequence.nextval from dual
This doesn't look right at all, and doesn't match the generated SQL from that earlier StackOverflow question linked above. Does anyone see something I am missing here, or otherwise know how to make Grails/Hibernate populate a primary key column with Oracle GUID values?
Whew... after another day of wrestling with this, I think I have my arms around the thing. This answer covers a bit more ground than the original question description, but that's because I found yet more problems after getting past the Hibernate generator issue.
Issue #1: Getting an Oracle GUID() value
As covered by Adam Hawkes' answer, the "guid" Hibernate generator is unmaintained and only works for older versions of the Oracle dialect.
However, if you use the Hibernate generator "assigned" (meaning that you want to set primary keys manually rather than have Hibernate auto-generate them), then you can insert values pulled from an Oracle SYS_GUID() call.
Even though Hibernate's newer Oracle dialects don't support "guid" seamlessly, they still understand the SQL necessary to generate these values. If you are inside of a Controller, you can fetch that SQL query with the following:
String guidSQL = grailsApplication.getMainContext().sessionFactory.getDialect().getSelectGUIDString()
If you are inside of a domain class instead, you can still do this... but you will need to first inject a reference to grailsApplication. You probably want to do this in a Controller, though... more on this below.
If you're curious, the actual String returned here (for Oracle) is:
select rawtohex(sys_guid()) from dual
You can execute this SQL and fetch the generated ID value like this:
String guid = grailsApplication.getMainContext().sessionFactory.currentSession.createSQLQuery(guidSQL).list().get(0)
Issue #2: Actually using this value in a Grails domain object
To actually use this GUID value in your Grails domain class, you need to use the Hibernate generator "assigned". As mentioned earlier, this declares that you want to set your own ID's manually, rather than letting Grails/GORM/Hibernate generate them automatically. Compare this modified code snippet to the one in my original question above:
...
static mapping = {
table name: "OWNER"
version false
id column: "OWNER_OID", generator: "assigned"
name column: "NAME"
...
}
...
In my domain class, I changed "guid" to "assigned". I also found that I needed to eliminate the "columns {}" grouping block, and move all my column information up a level (weird).
Now, in whichever Controller is creating these domain objects... generate a GUID as described above, and plug it into the object's "id" field. In a Controller generated automatically by Grails Scaffolding, the function will be "save()":
def save() {
def ownerInstance = new Owner(params)
String guidSQL = grailsApplication.getMainContext().sessionFactory.getDialect().getSelectGUIDString()
ownerInstance.id = grailsApplication.getMainContext().sessionFactory.currentSession.createSQLQuery(guidSQL).list().get(0)
if (!ownerInstance.save(flush: true, insert: true)) {
render(view: "create", model: [ownerInstance: ownerInstance])
return
}
flash.message = message(code: 'default.created.message', args: [message(code: 'owner.label', default: 'Owner'), ownerInstance.id])
redirect(action: "show", id: ownerInstance.id)
}
You might think to try putting this logic directly inside the domain object, in a "beforeInsert()" function. That would definitely be cleaner and more elegant, but there are some known bugs with Grails that prevent ID's from being set in "beforeInsert()" properly. Sadly, you'll have to keep this logic at the Controller level.
Issue #3: Make Grails/GORM/Hibernate store this properly
The plain truth is that Grails is primarily intended for virgin-new applications, and its support for legacy databases is pretty spotty (in fairness, though, it's a bit less spotty than other "dynamic" frameworks I've tried). Even if you use the "assigned" generator, Grails sometimes gets confused when it goes to persist the domain object.
One such problem is that a ".save()" call sometimes tries to do an UPDATE when it should be doing an INSERT. Notice that in the Controller snippet above, I have added "insert: true" as a parameter to the ".save()" call. This tells Grails/GORM/Hibernate explicitly to attempt an INSERT operation rather than an UPDATE one.
All of the stars and planets must be in alignment for this to work right. If your domain class "static mapping {}" block does not set the Hibernate generator to "assigned", and also set "version false", then Grails/GORM/Hibernate will still get confused and try to issue an UPDATE rather than an INSERT.
If you are using auto-generated Grails Scaffolding controllers, then it is safe to use "insert: true" in the Controller's "save()" function, because that function in only called when saving a new object for the first time. When a user edits an existing object, the Controller's "update()" function is used instead. However, if you are doing your own thing in your own custom code somewhere... it will be important to check on whether a domain object is already in the the database before you make a ".save()" call, and only pass the "insert: true" parameter if it really is a first-time insert.
Issue #4: Using natural keys with Grails/GORM/Hibernate
One final note, not having to do with Oracle GUID values, but related to these Grails issues in general. Let's say that in a legacy database (such as the one I've been dealing with), some of your tables use a natural key as their primary key. Say you have an OWNER_TYPE table, containing all the possible "types" of OWNER, and the NAME column is both the human-readable identifier as well as the primary key.
You'll have to do a couple of other things to make this work with Grails Scaffolding. For one thing, the auto-generated Views do not show the ID field on the screen when users are creating new objects. You will have to insert some HTML to the relevant View to add a field for the ID. If you give the field a name of "id", then the auto-generated Controller's "save()" function will receive this value as "params.id".
Secondly, you have to make sure that the auto-generated Controller's "save()" function properly inserts the ID value. When first generated, a "save()" starts off by instantiating a domain object from the CGI parameters passed by the View:
def ownerTypeInstance = new OwnerType.get( params )
However, this does not handle the ID field you added to your View. You will still need to set that manually. If on the View you gave the HTML field a name of "id", then it will be available in "save()" as "params.id":
...
ownerTypeInstance = new OwnerType()
ownerTypeInstance.id = params.id
// Proceed to the ".save()" step, making sure to pass "insert: true"
...
Piece of cake, huh? Perhaps "Issue #5" is figuring out why you put yourself through all this pain, rather than just writing your CRUD interface by hand with Spring Web MVC (or even vanilla JSP's) in the first place! :)
Support for using SYS_GUID() is dependent upon the Oracle dialect that you are using. Looking at the hibernate source on GitHub it appears that the dialect was only setup to use the Oracle-generated guid in Oracle9Dialect.java and Oracle8iDialect.java. Therefore, it won't work with the 9i or 10g dialects.
You should submit a patch to hibernate which will add the required function(s) to enable the same functionality as the other dialects.
I am creating a laboratory database which analyzes a variety of samples from a variety of locations. Some locations want their own reference number (or other attributes) kept with the sample.
How should I represent the columns which only apply to a subset of my samples?
Option 1:
Create a separate table for each unique set of attributes?
SAMPLE_BOILER: sample_id (FK), tank_number, boiler_temp, lot_number
SAMPLE_ACID: sample_id (FK), vial_number
This option seems too tedious, especially as the system grows.
Option 1a: Class table inheritance (link): Tree with common fields in internal node/table
Option 1b: Concrete table inheritance (link): Tree with common fields in leaf node/table
Option 2: Put every attribute which applies to any sample into the SAMPLE table.
Most columns of each entry would most likely be NULL, however all of the fields are stored together.
Option 3: Create _VALUE_ tables for each Oracle data type used.
This option is far more complex. Getting all of the attributes for a sample requires accessing all of the tables below. However, the system can expand dynamically without separate tables for each new sample type.
SAMPLE:
sample_id*
sample_template_id (FK)
SAMPLE_TEMPLATE:
sample_template_id*
version *
status
date_created
name
SAMPLE_ATTR_OF
sample_template_id* (FK)
sample_attribute_id* (FK)
SAMPLE_ATTRIBUTE:
sample_attribute_id*
name
description
SAMPLE_NUMBER:
sample_id* (FK)
sample_attribute_id (FK)
value
SAMPLE_DATE:
sample_id* (FK)
sample_attribute_id (FK)
value
Option 4: (Add your own option)
To help with Googling, your third option looks a little like the Entity-Attribute-Value pattern, which has been discussed on StackOverflow before although often critically.
As others have suggested, if at all possible (eg: once the system is up and running, few new attributes will appear), you should use your relational database in a conventional manner with tables as types and columns as attributes - your option 1. The initial setup pain will be worth it later as your database gets to work the way it was designed to.
Another thing to consider: are you tied to Oracle? If not, there are non-relational databases out there like CouchDB that aren't constrained by up-front schemas in the same way as relational databases are.
Edit: you've asked about handling new attributes under option 1 (now 1a and 1b in the question)...
If option 1 is a suitable solution, there are sufficiently few new attributes that the overhead of altering the database schema to accommodate them is acceptable, so...
you'll be writing database scripts to alter tables and add columns, so the provision of a default value can be handled easily in these scripts.
Of the two 1 options (1a, 1b), my personal preference would be concrete table inheritance (1b):
It's the simplest thing that works;
It requires fewer joins for any given query;
Updates are simpler as you only write to one table (no FK relationship to maintain).
Although either of these first options is a better solution than the others, and there's nothing wrong with the class table inheritance method if that's what you'd prefer.
It all comes down to how often genuinely new attributes will appear.
If the answer is "rarely" then the occasional schema update can cope.
If the answer is "a lot" then the relational DB model (which has fixed schemas baked-in) isn't the best tool for the job, so solutions that incorporate it (entity-attribute-value, XML columns and so on) will always seem a little laboured.
Good luck, and let us know how you solve this problem - it's a common issue that people run into.
Option 1, except that it's not a separate table for each set of attributes: create a separate table for each sample source.
i.e. from your examples: samples from a boiler will have tank number, boiler temp, lot number; acid samples have vial number.
You say this is tedious; but I suggest that the more work you put into gathering and encoding the meaning of the data now will pay off huge dividends later - you'll save in the long term because your reports will be easier to write, understand and maintain. Those guys from the boiler room will ask "we need to know the total of X for tank grouped by this set of boiler temperature ranges" and you'll say "no prob, give me half an hour" because you've done the hard yards already.
Option 2 would be my fall-back option if Option 1 turns out to be overkill. You'll still want to analyse what fields are needed, what their datatypes and constraints are.
Option 4 is to use a combination of options 1 and 2. You may find some attributes are shared among a lot of sample types, and it might make sense for these attributes to live in the main sample table; whereas other attributes will be very specific to certain sample types.
You should really go with Option 1. Although it is more tedious to create, Option 2 and 3 will bite you back when trying to query you data. The queries will become more complex.
In fact, the most important part of storing the data, is querying it. You haven't mentioned how you are planning to use the data, and this is a big factor in the database design.
As far as I can see, the first option will be most easy to query. If you plan on using reporting tools or an ORM, they will prefer it as well, so you are keeping your options open.
In fact, if you find building the tables tedious, try using an ORM from the start. Good ORMs will help you with creating the tables from the get-go.
I would base your decision on the how you usually see the data. For instance, if you get 5-6 new attributes per day, you're never going to be able to keep up adding new columns. In this case you should create columns for 'standard' attributes and add a key/value layout similar to your 'Option 3'.
If you don't expect to see this, I'd go with Option 1 for now, and modify your design to 'Option 3' only if you get to the point that it is turning into too much work. It could end up that you have 25 attributes added in the first few weeks and then nothing for several months. In which case you'll be glad you didn't do the extra work.
As for Option 2, I generally advise against this as Null in a relational database means the value is 'Unknown', not that it 'doesn't apply' to a specific record. Though I have disagreed on this in the past with people I generally respect, so I wouldn't start any wars over it.
Whatever you do option 3 is horrible, every query will have join the data to create a SAMPLE.
It sounds like you have some generic SAMPLE fields which need to be join with more specific data for the type of sample. Have you considered some user_defined fields.
Example:
SAMPLE_BASE: sample_id(PK), version, status, date_create, name, userdata1, userdata2, userdata3
SAMPLE_BOILER: sample_id (FK), tank_number, boiler_temp, lot_number
This might be a dumb question but what do you need to do with the attribute values? If you only need to display the data then just store them in one field, perhaps in XML or some serialised format.
You could always use a template table to define a sample 'type' and the available fields you display for the purposes of a data entry form.
If you need to filter on them, the only efficient model is option 2. As everyone else is saying the entity-attribute-value style of option 3 is somewhat mental and no real fun to work with. I've tried it myself in the past and once implemented I wished I hadn't bothered.
Try to design your database around how your users need to interact with it (and thus how you need to query it), rather than just modelling the data.
If the set of sample attributes was relatively static then the pragmatic solution that would make your life easier in the long run would be option #2 - these are all attributes of a SAMPLE so they should all be in the same table.
Ok - you could put together a nice object hierarchy of base attributes with various extensions but it would be more trouble than it's worth. Keep it simple. You could always put together a few views of subsets of sample attributes.
I would only go for a variant of your option #3 if the list of sample attributes was very dynamic and you needed your users to be able to create their own fields.
In terms of implementing dynamic user-defined fields then you might first like to read through Tom Kyte's comments to this question. Now, Tom can be pretty insistent in his views but I take from his comments that you have to be very sure that you really need the flexibility for your users to add fields on the fly before you go about doing it. If you really need to do it, then don't create a table for each data type - that's going too far - just store everything in a varchar2 in a standard way and flag each attribute with an appropriate data type.
create table sample (
sample_id integer,
name varchar2(120 char),
constraint pk_sample primary key (sample_id)
);
create table attribute (
attribute_id integer,
name varchar2(120 char) not null,
data_type varchar2(30 char) not null,
constraint pk_attribute primary key (attribute_id)
);
create table sample_attribute (
sample_id integer,
attribute_id integer,
value varchar2(4000 char),
constraint pk_sample_attribute primary key (sample_id, attribute_id)
);
Now... that just looks evil doesn't it? Do you really want to go there?
I work on both a commercial and a home-made system where users have the ability to create their own fields/controls dynamically. This is a simplified version of how it works.
Tables:
Pages
Controls
Values
A page is just a container for one or more controls. It can be given a name.
Controls are linked to pages and represents user input controls.
A control contains what datatype it is (int, string etc) and how it should be represented to the user (textbox, dropdown, checkboxes etc).
Values are the actual data that the users have typed into the controls, a value contains one column for every datatype that it can represent (int, string, etc) and depending on the control type, the relevant column is set with the user input.
There is an additional column in Values which specifies which group the value belong to.
Each time a user fills in a form of controls and clicks save, the values typed into the controls are saved into the same group so that we know that they belong together (incremental counter).
CodeSpeaker,
I like you answer, it's pointing me in the right direction for a similar problem.
But how would you handle drop-downlist values?
I am thinking of a Lookup table of values so that many lookups link to one UserDefinedField.
But I also have another problem to add to the mix. Each field must have multiple linked languages so each value must link to the equivilant value for multiple languages.
Maybe I'm thinking too hard about this as I've got about 6 tables so far.