Currently Liquibase has some limitations when you try to generate change logs on an existing database. It does not export the following types of objects:
Stored procedures, functions, packages
Triggers
Types
Reference: http://www.liquibase.org/documentation/generating_changelogs.html
As far as I understand I need to develop my own liquibase.snapshot.SnapshotGenerator implementation. I know how to obtain these types of objects from Oracle but I'm a bit lost on how to implement such interface from Liquibase.
Ideally I guess the liquibase.database.Database interface should be also extended to add the following extra methods:
public abstract boolean supportsPackages();
public abstract boolean supportsFunctions();
public abstract boolean supportsStoredProcedures();
public abstract boolean supportsTriggers();
public abstract boolean supportsTypes();
You are right that the general strategy is to create a new class that implements SnapshotGenerator, but there are a couple other steps you need to do as well. The general snapshot process is:
Liquibase searches for SnapshotGenerator implemetnations and calls the addsTo() for each object it finds in the database. For your types, you probably want a quick "if passed object instanceof Schema" because they are types that are part of a schema.
You will need to create new Package, StoredProcedure, etc objects that implement DatabaseObject. They will be lik ehte liquibase.structure.core.Table class and capture the state of the object. They are created in your SnapshotGenerator.addsTo() method to the point of being identifiable (name, schema, etc set)
All objects that are added by the addsTo() method are then ran through your SnapshotGenerator.snapshotObject() method which will pull any additional metadata that you didn't get originally, such as stored procedure text, etc.
Once liquibase has a snapshot containing your objects it compares the snapshot to another (in generateChangeLog case an empty snapshot) to determine what objects are missing, unexpected, and changed in the second snapshot. It then looks for implementations of MissingObjectChangeGenerator, UnexpectedObjectChangeGenerator, and ChangedObjectChangeGenerator. For generateChangeLog there will only be "missing" objects so you would implement MissingTriggerChangeGenerator, MissingPackagedChangeGenerator etc. Their job is to create Change instances to create the missing objects
The Msising*ChangeGenerator classes could return RawSqlChange instances or you could create new implementations of Change such as CreateTriggerChange.
Related
I am "playing" with graphql-java and the related kickstarter servlet, modifying it to build the schema programmatically based on the types I discover automatically. The documentation is (almost) there but is outdated in spots and references deprecated methods. My particular issue is that the GraphQLFieldDefinition.Builder has four variants of the type(...) method - three accepting various (other) builders and one accepting the GraphQLOutputType, none of which I may have. Furthermore, those that accept builder actually invoke the build() method on those builders so I am anxious to pass any of my "unfinished" type builders either. I thought of defining all types upfront with no fields at all, then add fields to them after that but if build() gets called on these at that point in time, even if they don't get into an unwanted state today I can see how a future update may demand at most a single build() invocation.
So the question is how does one declare a field to yet-undefined or not fully defined type? For example, an Employee's manager field being itself of the Employee type such as:
type Employee {
name: String!
manager: Employee
reports: [Employee!]
buddy: Employee
...
}
As I noted in my comments, instead of attempting to (directly) use the builder or the built type one should create an instance of GraphQLTypeReference instead like so:
GraphQLTypeReference.typeRef(typeName)
This will be resolved automatically when the schema is built.
Should we want to indicate that this is not nullable even if the referenced type is, we further need to wrap into a GraphQLNonNull wrapper like so:
GraphQLNonNull.nonNull(typeRef)
... and use that as the type.
I use spring-data-cassandra, and have entity like this:
#Table("users")
public class User {
#Column("permissions")
#CassandraType(type = DataType.Name.SET, typeArguments = {DataType.Name.TEXT})
public Set<String> permissions = new HashSet<>();
}
In cassandra I have table users with field permissions of type Set. It works fine when I store some values in the set, but when I try to store empty set, it becomes null when I read such entity from the repository.
Is there a way to force spring-data-cassandra to change null to empty HashSet? Or can I somehow add custom reader for this specific property of the entity?
TL;DR;
That's Cassandra's default behavior to return null for empty Collection and Map-typed columns.
Further Read
Cassandra returns null values for lists, sets, and maps, which do not contain any items. This is especially unfortunate when using classes with pre-initialized fields as seen in your question. There's an open ticket (DATACASS-266 - Loading empty collection-typed properties overwrites pre-initialized fields) in the issue tracker - as of now, without comments or votes.
We're not exactly sure whether it's a good idea to skip setting properties or apply some sort of defaulting when dealing with empty (null) collections as this raises follow-up questions what to do when:
Creating an instance through constructor creation: A value is required in such case. For property access, we could omit to set the property, for constructor creation we must provide a value.
The pre-initialized collection contains items but the one received from Cassandra is null.
We assume, the change would be applied, what will happen with already existing code that assumes empty collections default to null.
A possibility to address this behavior could be configuration on MappingCassandraConverter or an extension point to override so users can apply their own empty collection behavior.
I've been trying to eliminate the null collections in my model objects as well, and while it may not be possible to do that at the Spring Data level currently (version 2.1.x), there are some options you can consider:
Use property access for the field in question (i.e. use the annotation #AccessType(PROPERTY)), and in the setter method, set the field to an empty collection when the argument is null.
Define a compatible (see below) constructor that sets the field to an empty collection when a null is provided (and if the model is mutable, you may still want to provide the setter as above).
There are some caveats to ensure Spring Data Cassandra uses the desired constructor (e.g. don't provide a no argument constructor), so it's critical to review the "Object Mapping Fundamentals" section of the reference guide (https://docs.spring.io/spring-data/cassandra/docs/current/reference/html/#mapping.fundamentals).
Among the recommendations in that reference guide (as of version 2.1 at least) is to use an all argument constructor and make model objects immutable, which would work well with the constructor-based approach to handling nulls. Though it does mean writing and maintaining the constructor to handle the nulls rather than relying on Lombok's #AllArgsConstructor.
I have used the property access approach in one case, but not the constructor approach. However I do intend go the constructor route when adding new or model classes (I'm a fan of immutable objects, and will explore that route even without any collection fields)
I believe Spring Data Cassandra 2.0 also added persistence lifecycle callbacks which is another possible option I suppose, but I ruled that out, mainly because the logic would not reside in the model class itself (as well as going against the recommendations from the creators of the framework)
Our database (which uses slick datamapper pattern, so our models are simple case classes which is ideal for commands) has many ADT's as fields (i.e. abstract sealed class with many case classes/case objects as children), and the idea is that our parameters (which is obviously going to be a string from within the params object) is going to converted to one of these types with a custom method
How would you go about creating a custom asType[T] for some type T from the string that you get from params when using Scalatra commands?
For more info on Scalatra commands, you can read here http://www.scalatra.org/guides/formats/commands.html
I am having a problem with a circular dependency. Similar question have been asked and I have read a lot of answers. Most deal with a work-around but I would like to refactor so what I have it correct and I would like some input on where I have gone wrong. I can change what I am doing but not the entire project architecture.
I am using VB.Net in Visual Studio 2012.
I have two class libraries:
DataLayer for accessing the database.
DataObject which contains classes that represents my business objects.
My Presentation Layer calls methods in the DataLayer which returns objects from the DataObject class library.
(I have simplified somewhat – I actually have a controller layer but it needs references to the two class libraries above. This is an existing architecture that is from before my time.)
In the DataObject class library I have an abstract class that represents a file. It has properties such as filename, userID, etc. It also has a method, GetFile(), that I code in the derived classes because there are different ways of getting the file. A DataLayer method returns a collection of these file objects, but I don't want to get the actual file until it's needed.
So far, I have a derived class that calls a webService (using properties from the baseClass) and a derived class that accesses the fileSystem. Both return a byte array representing the file. The calling class does not need to know how the file is retrieved.
Now I have a new requirement to build the file on the fly using data from the database. I can get all the data I need using the properties in the base class.
My issue is that my GetFile() method will need to access my DataLayer class library to pull data from the database which causes a circular dependency. The DataLayer class library has a reference to DataObject since that is what it returns. But now I need to call the DataLayer from a class in DataObjects.
I could call the DataLayer from presentation and pass the result to
my DataObject’s GetFile() method, but then my presentation layer
needs to do something special for this derived class. My goal is
that the derived class handles GetFile without presentation knowing
about the implementation.
I could create a new class library for this DataLayer code but I
don't like a special case.
I could access the DB directly in the DataObject class but that
circumvents the layered architecture.
I can’t change our architecture, but I can change my approach.
Any opinions?
I think I have the answer.
In my concrete class, when I am loading the data initially (in the DataLayer), I will get all the data I need to create the file. I'll store it in a new property in my concrete class which my GetFile() method will use to build the file.
This has a little more overhead - I make DB calls and put all this data in memory when it may not be needed. I'll give it a try and see how performance is.
Any critiques of this approach?
I have a semi complicated question regarding Entity Framework4, Lambda expressions, and Data Transfer Objects (DTO).
So I have a small EF4 project, and following established OO principles, I have a DTO to provide a layer of abstraction between the data consumers (GUI) and the data model.
VideoDTO = DTO with getters/setters, used by the GUI
VideoEntity = Entity generated by EF4
My question revolves around the use of the DTO by the GUI (and not having the GUI use the Entity at all), combined with a need to pass a lambda to the data layer. My data layer is a basic repository pattern with Add. Change, Delete, Get, GetList, etc.
Trying to implement a Find method with a signature like so:
public IEnumerable<VideoDTO> Find(Expression<Func<VideoEntity, bool>> exp)
...
_dataModel.Videos.Where(exp).ToList<Video>()
---
My problem/concern is the "exp" needing to be of type VideoEntity instead of VideoDTO. I want to preserve the separation of concerns so that the GUI does not know about the Entity objects. But if I try to pass in
Func<VideoDTO, bool>
I cannot then do a LINQ Where on that expression using the actual data model.
Is there a way to convert a Func<VideoDTO,bool> to a Func<VideoEntity, bool>
Ideally my method signature would accept Func<VideoDTO, bool> and that way the GUI would have no reference to the underlying data entity.
Is this clear enough? Thanks for your help
Thanks for the repliesto both of you.
I'll try the idea of defining the search criteria in an object and using that in the LINQ expression. Just starting out with both EF4 and L2S, using this as a learning project.
Thanks again!
In architectures like CQRS there isn't need for such a conversion at all cause read & write sides of app are separated.
But in Your case, You can't runaway from translation.
First of all - You should be more specific when defining repositories. Repository signature is thing You want to keep explicit instead of generic.
Common example to show this idea - can You tell what indexes You need in Your database when You look at Your repository signature (maybe looking at repository implementation, but certainly w/o looking at client code)? You can't. Cause it's too generic and client side can search by anything.
In Your example it's a bit better cause expression genericness is tied with dto instead of entity.
This is what I do (using NHibernate.Linq, but the idea remains)
public class Application{
public Project Project {get;set;}
}
public class ApplicationRepository{
public IEnumerable<Application> Search(SearchCriteria inp){
var c=Session.Linq<Application>();
var q=c.AsQueryable();
if(!string.IsNullOrEmpty(inp.Acronym))
q=q.Where(a=>a.Project.Acronym.Contains(inp.Acronym));
/*~20 lines of similar code snipped*/
return q.AsQueryable();
}
}
//used by client
public class SearchCriteria{
public string Acronym{get;set;}
/*some more fields that defines how we can search Applications*/
}
If You do want to keep Your expressions, one way would be to define dictionary manually like this:
var d=new Dictionary<Expression<Func<VideoDTO,object>>,
Expression<Func<VideoEntity,object>>{
{x=>x.DtoPropNumberOne,x=>x.EntityPropNumberOne} /*, {2}, {3}, etc.*/
};
And use it later:
//can You spot it?
//client does not know explicitly what expressions dictionary contains
_dataModel.Videos.Where(d[exp]).ToList<Video>();
//and I'm not 100% sure checking expression equality would actually work
If You don't want to write mapping dictionary manually, You will need some advanced techniques. One idea would be to translate dto expression to string and then back to entity expression. Here are some ideas (sorting related though) that might help. Expressions are quite complicated beasts.
Anyway - as I said, You should avoid this. Otherwise - You will produce really fragile code.
Perhaps your design goal is to prevent propagation of the data model entities to the client tier rather than to prevent a dependency between the presentation layer and data model. If viewed that way then there would be nothing wrong with the query being formed the way you state.
To go further you could expose the searchable fields from VideoEntity via an interface (IVideoEntityQueryFields) and use that as the type in the expression.
If you don't want to add an interface to your entities then the more complicated option is to use a VideoEntityQuery object and something that translates an Expression<Func<VideoEntityQuery,bool>> to an Expression<Func<VideoEntity,bool>>.