I am currently using Scriptella ETL in Spring. I want to change the db schema and perform an upgrade of the db when i execute my program. I did not find any reference to do this except for a way using ANT.
db-upgrade example
Wanted to know if we can write a separate sql like v1-v2.xml with the new alters and create statements and have it referenced in spring context.xml?
My current spring context xml referencing etl..
<bean id="etlProgress" class="scriptella.interactive.ConsoleProgressIndicator"/>
<bean id="etlExecutor" class="scriptella.driver.spring.EtlExecutorBean">
<property name="configLocation" value="etl.xml"/>
<property name="progressIndicator"><ref local="etlProgress"/></property>
</bean>
Any help here pls?
Found an answer myself. In etl.xml you can add a query that queries the "MetaInf" table. In my case, I had a metainf table like this.
CREATE TABLE Metainf (
buildnum int primary key
);
INSERT INTO Metainf VALUES (1);
That means, i have to query "buildnum" to check the version of the db. This can be done the following way
<query>
<!-- Selects current DB build -->
SELECT * FROM Metainf
<!-- Check if upgrade is necessary -->
<script if="buildnum lt 1">
<!--Upgrades DB to build 1 -->
<!--...-->
<!-- Update Metainf to confirm successful upgrade -->
UPDATE Metainf SET buildnum=1;
</script>
<script if="buildnum eq 1">
<!-- upgrade scripts for subsequent builds -->
UPDATE Metainf SET buildnum=2;
</script>
</query>
I placed my old db file and ran after this change. Voila! I have my new changes in the database.
People who are looking for this answer, do try and let me know if it worked for you also.
Thanks
Related
We are using JPA with Hibernate 4.3.8 and Oracle 11. Our entities doesn't have explicit DB identifier names declared in annotations and so we rely upon Hibernate to generate them correctly.
For MySQL it works properly but after we had switched to Oracle then we've encountered couple of the problems.
One of them is that Hibernate-generated schema contains identifiers longer than Oracle supports. We thought that Oracle10gDialect handles it but it seems we were wrong.
What is the best approach to add support of Oracle to our application? Must we explicitly declare all the database tables/columns/indexes... in annotations (#Table, #Column...)? Shouldn't it be Hibernate who takes a care of this dialect-specific task?
Another problem is that Hibernate also doesn't escape the keywords (eg. column name code must be escaped in Oracle). What if we decide to support another database in the future? Must we choose all the identifier names so that all of them suit for every DB? This seems to be very overhelming. (Note that the property hibernate.globally_quoted_identifiers can solve it partially but then #UniqueConstraint columns doesn't match.)
Or maybe... does Eclipselink handle this?
In such cases you can use Hibernate's NamingStrategy prepared for Oracle.
In simple words: NamingStrategy tells Hibernate how to generate names for columns or tables.
You can provide oracle-aware NamingStrategy as I do for each of my Oracle/Hibernate projects.
Here is sample JPA configuration:
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"
...
<property name="jpaPropertyMap">
<map>
<entry key="hibernate.ejb.naming_strategy" value="com.mycompany.OracleNamingStrategy"/>
<entry key="hibernate.dialect" value="org.hibernate.dialect.Oracle10gDialect"/>
...
</map>
</property>
</bean>
Here is the NamingStrategy I have been using for many projects:
https://code.google.com/p/hibernate-naming-strategy-for-oracle/source/browse/trunk/src/de/schauderhaft/hibernate/OracleNamingStrategy.java
look at smart abbreviating method that guarantees to produce Oracle-compliant name:
public static String abbreviateName(String someName) {
if (someName.length() <= MAX_LENGTH)
return someName;
String[] tokens = splitName(someName);
shortenName(someName, tokens);
return assembleResults(tokens);
}
There are many articles about applying NamingStratety with Hibernate, for example:
1.
http://www.petrikainulainen.net/programming/tips-and-tricks/implementing-a-custom-namingstrategy-with-hibernate/
2.
http://sudhirmongia.blogspot.com/2010/08/naming-strategy-in-hibernate.html
I hope this helps.
I adapt Spring+Hibernate+Tomcat+Oracle application to distributed form and need to make schema names customizable in JPA annotations, something like with Spring EL:
#Entity
#Table(name = "LOSS", schema="${app.dataSchema}")
public class Loss { ... }
I look for support of placeholders in annotation like ${app.dataSchema} in above example. Or any other possibility...
Idea - to have application that can be deployed on another site without recompilation, where preserved DB relations/hierarchy, but some names are changed and can be configured by system properties, JNDI or deploy context descriptor.
One solution that I see - to create view in selected schema - to provide bridge between actual schema/table/column and application hard-coded names.
But I hope that JPA/Hibernate/Spring have some instruments for such configuration...
Use orm.xml. That is what JPA provides it for. Putting deployment info in annotations is a bad idea if you ever need to change deployment, so just having a different orm.xml file means you can easily achieve that. Note that you can set a default schema in the persistence-unit-defaults part of orm.xml too
Externalize Schema
By EntityManagerFactory
<bean id="domainEntityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceUnitName" value="pun"/>
<property name="dataSource" ref="domainDataSource"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="generateDdl" value="false"/>
<property name="showSql" value="false"/>
<property name="databasePlatform" value="${hibernate.dialect}"/>
</bean>
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.hbm2ddl.auto">create-update</prop>
<prop key="hibernate.default_schema">${yourSchema}</prop>
</props>
</property>
</bean>
By orm.xml
...
<persistence-unit name="MySchemaPU" transaction-type="JTA">
<provider>
<mapping-file>META-INF/orm.xml</mapping-file>
...
orm.xml below:
<?xml version="1.0" encoding="UTF-8"?>
<entity-mappings xmlns="http://java.sun.com/xml/ns/persistence/orm"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence/orm http://java.sun.com/xml/ns/persistence/orm_1_0.xsd"
version="1.0">
<persistence-unit-metadata>
<persistence-unit-defaults>
<schema>myschema</schema>
</persistence-unit-defaults>
</persistence-unit-metadata>
</entity-mappings>
By persistence.xml
<persistence-unit name="myPersistenceUnit" transaction-type="RESOURCE_LOCAL">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<jta-data-source>myDatasource</jta-data-source>
<properties>
.....
<property name="hibernate.default_schema" value="MYSCHEMA"/>
</properties>
</persistence-unit>
For Columns and Tables the deployment must be changed. You cannot replace placeholders at runtime in compiled classes.
Hibernate has org.hibernate.cfg.Configuration which is a point where packages are scanned and config files are read.
It uses org.hibernate.cfg.AnnotationBinder.bindClass(...) which read annotations on classes.
These classes (8 kLOC) and methods so long that I can't read them easy so I have no hope to subclass Configuration...
Ever if note that all classes and initialisation code binded to concrete classes without possibility to inject own, seems my task can't be solved easy with current Hibernate 4.x architecture.
I start looking for alternatives and ask about externaliztion of config in EBean mailing list and got answer:
public class MySchemaNamingConvention
extends com.avaje.ebean.config.UnderscoreNamingConvention {
#Override
protected TableName getTableNameFromAnnotation(Class<?> beanClass) {
final Table t = findTableAnnotation(beanClass);
if (t != null && !isEmpty(t.name())) {
String catalog = t.catalog();
String schema = t.schema();
////// CUSTOM schema processing code HERE.
////// CUSTOM schema processing code HERE.
////// CUSTOM schema processing code HERE.
return new TableName(quoteIdentifiers(catalog),
quoteIdentifiers(schema),
quoteIdentifiers(t.name()));
}
return null;
}
So damn easy that I say good bye to Hibernate!
UPDATE #Xstian. Before I migrate to Ebean I set default schema through hibernate.default_schema and extensively use views and synonyms to control which table will be accessible (Oracle):
grant select on ANOTHER_SCHEMA.TBL to MY_SCHEMA;
create or replace view MY_SCHEMA.TBL as select * from ANOTHER_SCHEMA.TBL;
create or replace synonym MY_SCHEMA.TBL for ANOTHER_SCHEMA.TBL;
During migration to views/synonyms I have issue only with DB constraints.
In case if you use Oracle - FK constraint on view not work unless you add constraint with disable novalidate keyword and same to referenced PK:
alter view MY_SCHEMA.XXX add constraint PK_XXX primary key(ID) disable novalidate;
alter table/view MY_SCHEMA.TBL add constraint FK_XXX foreign key (XXX_ID)
references MY_SCHEMA.XXX (ID) disable novalidate;
And synonyms don't allow FK constraint at all for Oracle!
Seems that Hibernate dictate of data layout and I think to switch to more flexible framework - Ebean, but I also evaluate sql2o and jOOQ.
I am using Visual Studio 2010 and ADO.Net Entity Data Model to create my database schema. Since i am still drafting and discussing the schema i would like to be able to add example data to the edmx-file and be able to show it in the designer like in the following screenshot:
Source code of EDMX file with added sample data
The underlying xml-code would contain the normal edmx-output and on top the sample data like below:
<?xml version="1.0" encoding="utf-8"?>
....
<EntityType Name="Title">
<Documentation>
<Summary>Table titles contains all the dvd-title we have in our store</Summary>
</Documentation>
<Key>
<PropertyRef Name="Id" />
</Key>
<Property Type="Int32"
Name="Id" Nullable="false"
annotation:StoreGeneratedPattern="Identity" />
<Property Type="String" Name="Title" Nullable="false" >
<Documentation>
<Summary>The title of the movie</Summary>
<LongDescription>
The translated title of the movie and the original title
</LongDescription>
</Documentation>
</Property>
</EntityType>
<Titles>
<Title>
<id>810</id>
<title>Pulp Fiction</title>
<year>1994</year>
</Title>
<Title>
<id>940</id>
<title>Lock, Stock and Two Smoking Barrels</title>
<year>1998</year>
</Title>
</Titles>
The sample data should be displayed and should be editable in design view.
Does anyone know a tool or a way to combine the edmx-file and to display and add sample data?
Bonus Question: When i generate the sql-statement the documentation is not added to the sql statement. Is there a way to include the documentation of the fields and the tables so that they would be added to Microsoft SQL Server?
I don't think it is possible. One of the reasons is that what you see is a model and not the database schema. What you don't see is how entity is mapped to the database. In the basic scenario there is 1:1 mapping indeed but once you start customizing your model you may end up having columns in the database with names different from the property names in the model, you may have complex properties you would not even have corresponding properties for on the diagram, you may put multiple entities to one table or one entity to multiple tables. How are you going to show this? And since it is just entity model and not table model data does not seem to be a good fit. I don't know of any tool that could do what you need. Edmx is an Xml file and the designer itself is extensible and based on DSL. You can try cooking something yourself but it will be a lot of work.
Our application has the concept of Stories and the concept of Tags. A story can have many tags applied to it, and a tag can be applied to many stories, making the relationship many-to-many. The two tables, Stories and Tags are bridged with a third, StoriesToTags.
The relevant pieces of the mapping files are as follows:
Here's the mapping from Story to Tag:
<class name="Story" table="Stories">
...
<set fetch="subselect" name="Tags" table="StoriesToTags">
<key>
<column name="StoryId" />
</key>
<many-to-many class="Tag">
<column name="TagId" />
</many-to-many>
</set>
</class>
And the inverse relationship from Tag to Story:
<class name="Tag" table="Tags">
...
<set fetch="subselect" inverse="true" name="Stories" table="StoriesToTags">
<key>
<column name="TagId" />
</key>
<many-to-many class="Story">
<column name="StoryId" />
</many-to-many>
</set>
</class>
As you can see, we're using the subselect fetch strategy to avoid the N+1 query problem. Everything works great, until you attempt to page a result using LINQ:
IQueryable<Story> stories = GetStories(...).TakePage(pageNumber, pageSize);
After running this query, NHibernate deletes the relationships (records in StoriesToTags) for all stories that were not loaded in the query. It only seems to occur when the tags are specifically loaded (that is, the subselect is triggered). The relationships are not deleted if we switch to a join or select fetch strategy, but that causes N+1 queries to be executed.
My best guess is that NHibernate thinks the tags have been orphaned, but we haven't set any cascades on the collections. Also, as far as I can tell, setting a cascade has no effect.
This process worked great under NHibernate 2.x and NHibernate.Linq. We didn't see the issue with deletion occur until we moved to NHibernate 3.x, which has LINQ support built-in. I'm not sure it makes a difference, but for what it's worth, we're using SQL Server with identity keys.
Any thoughts? I initially thought I was doing something insanely stupid, but I've tried basically every permutation of mapping and we can't seem to eliminate the issue.
Edit: Another interesting piece of information. If you call session.IsDirty() before closing the session, the issue doesn't occur. I suspect that this is because collection changes aren't persisted between flushes, but I can't decipher NHibernate's source well enough to know for certain.
have you set up in mapping of the entity : Cascade.None() this will stop deleting anything else except the entity.
This might help: http://ayende.com/blog/1890/nhibernate-cascades-the-different-between-all-all-delete-orphans-and-save-update
Can you give us some clue at what you are trying to achieve here ? I never experimented with a specified fetch on a many-to-many, but me thinks it got something to do with some sort of explicit cascade= all for a many to many.
Has anyone heard of this error before:
I have a Java 5 application, that uses Hibernate 3.3.2 to access an Oracle Database 10g, the JDBC driver is the Oracle light driver version 10.2.0.4.0 (the most recent version, I think).
The database access happens during a transaction that is managed via the AOP-tools of spring.
There is one database table mapped to one Java entity, the mapping is done via annotations. There are two attributes in this entity that are CLOB in the database. They are annotated according to JPA with "Lob" and are Java Strings. Everthing works fine unless both attributes have values with more than 4000 characters: After the commit the values are switched on the database, that is attribute A contains the value of attribute B and vice versa.
This is not a joke!
The Eclipse debugger tells me that the Java entity has the correct values until the transaktion closes (I did not debug into the spring transaction handling).
This error does not occur if both Clobs are annotated with the Hibernate annotation 'Type(type = "clob")' (in which case they have to be of the type java.sql.Clob and not String, of course).
I'm not 100% sure but could you try to set the connection property SetBigStringTryClob to true (see How To Handle CLOBs Easily in JDBC?).
If you are using Spring, something like this:
<property name="hibernateProperties">
<props>
<prop key="hibernate.connection.SetBigStringTryClob">true</prop>
</props>
</property>
Meanwhile I found a solution to the problem: The SQL-dialect was wrong, someone set it to Oracle9 instead of the correct version, which is this:
name="hibernate.dialect" value="org.hibernate.dialect.Oracle10gDialect"
Setting the correct SQL-dialect solves the problem :-)
I have exactly the same problem. Two CLOB on a table. On the insert only one is set and everything is fine. On the update I set the second one and after commit the values on table are swapped. Both contains more than 4000 characters. Debugged and the values on the bean are correct.
The sad thing is that i have already org.hibernate.dialect.Oracle10gDialect in my properties.
(Hibernate version 4.2.2.Final)
//UPDATE
for the moment as a workaround I have set the first column as updatable=false in the JPA annotation so that during the generation of the update query by hibernate there is only one CLOB column now. I can do this as that in this particular case I don't need to change the value after the insert, but it's not a general solution.
The only thing that worked for us was to use hql queries for the update. Each query updated a single CLOB (so we needed two separate queries):
Query query = em.createQuery("update SomeTable as someTable set value = :value where id = :id");
query.setParameter("value", value);
query.setParameter("id", id);
if (query.executeUpdate() != 1) {
throw new Exception("Value update had no effect");
}