Hibernate select batch-size - performance

Hello,
i have simple J2EE application that shows first 100 Employee records from db (Oracle DB) with Hibernate (version 3). All of the tables have about 10000 records. There are Country(id, name), City(id, name, country_id), Address(id, address, city_id), Company(id, name), Office(id, company_id, address_id), Employee(id, name, address_id), Position(id, name) and Employee_Position_Office(employee_id, position_id, office_id) in db. And i have some problems with batch-size. I need to do it in 4 sql-request (generated by Hibernate). Let's show you what i have:
In my class-mappings i set batch-size=100 and Hibernate creates 3 sql-request for Employee, OfficePosition and Address loading, look:
select * from ( select employee0_.ID as ID6_, employee0_.NAME as NAME6_, employee0_.ADDRESS as ADDRESS6_ from EMPLOYEE employee0_ ) where rownum <= ?
select officeposi0_.EMPLOYEE_ID as EMPLOYEE1_6_1_, officeposi0_.POSITION_ID as POSITION2_1_, officeposi0_.OFFICE_ID as OFFICE3_1_, position1_.ID as ID5_0_, position1_.NAME as NAME5_0_ from EMPLOYEE_POSITION_OFFICE officeposi0_ inner join POSITION position1_ on officeposi0_.POSITION_ID=position1_.ID where officeposi0_.EMPLOYEE_ID in (100x ?)
select address0_.ID as ID2_2_, address0_.ADDRESS as ADDRESS2_2_, address0_.CITY_ID as CITY3_2_2_, city1_.ID as ID1_0_, city1_.NAME as NAME1_0_, city1_.COUNTRY_ID as COUNTRY3_1_0_, country2_.ID as ID0_1_, country2_.NAME as NAME0_1_ from ADDRESS address0_ left outer join CITY city1_ on address0_.CITY_ID=city1_.ID left outer join COUNTRY country2_ on city1_.COUNTRY_ID=country2_.ID where address0_.ID in (100x ?)
And if i change batch-size to number more than 100 nothing changes - this is good.
But then (after these requests) Hibernate generates request(s) for Office loading, and i have some problems with it, let's show you:
My 100 first Employees work in 397 offices (randomly generated) and Hibernate should loads they with 1 request when batch-size is more than 397, when i set batch-size to 397 it works fine (in 1 sql-request),
select office0_.ID as ID4_4_, office0_.COMPANY_ID as COMPANY2_4_4_, office0_.ADDRESS_ID as ADDRESS3_4_4_, (SELECT COUNT(*) FROM EMPLOYEE_POSITION_OFFICE
E WHERE
E.OFFICE_ID=office0_.ID)
as formula0_4_, company1_.ID as ID3_0_, company1_.NAME as NAME3_0_, address2_.ID as ID2_1_, address2_.ADDRESS as ADDRESS2_1_, address2_.CITY_ID as CITY3_2_1_, city3_.ID as ID1_2_, city3_.NAME as NAME1_2_, city3_.COUNTRY_ID as COUNTRY3_1_2_, country4_.ID as ID0_3_, country4_.NAME as NAME0_3_ from OFFICE office0_ left outer join COMPANY company1_ on office0_.COMPANY_ID=company1_.ID left outer join ADDRESS address2_ on office0_.ADDRESS_ID=address2_.ID left outer join CITY city3_ on address2_.CITY_ID=city3_.ID left outer join COUNTRY country4_ on city3_.COUNTRY_ID=country4_.ID where office0_.ID in (397x ?)
but when i set any another number (less or more than 397) Hibernate generates several sql-requests with same "body" but others "tails".
For example, there are results when batch-size = 400.
...in (200x ?)
...in (100x ?)
...in (50x ?)
...in (25x ?)
...in (12x ?)
...in (10x ?)
Tell, please, what i do wrong and can it be fixed. Thx.
My mappings:
<class name="---.Address" table="ADDRESS"
batch-size="100">
<id name="id" type="long">
<column name="ID" />
<generator class="increment" />
</id>
<property name="address" type="java.lang.String">
<column name="ADDRESS" />
</property>
<many-to-one name="city" class="---.City"
fetch="join">
<column name="CITY_ID" />
</many-to-one>
</class>
<class name="---.City" table="CITY">
<id name="id" type="long">
<column name="ID" />
<generator class="increment" />
</id>
<property name="name" type="java.lang.String">
<column name="NAME" />
</property>
<many-to-one name="country" class="---.Country"
fetch="join">
<column name="COUNTRY_ID" />
</many-to-one>
</class>
<class name="---.Company" table="COMPANY">
<id name="id" type="long">
<column name="ID" />
<generator class="increment" />
</id>
<property name="name" type="java.lang.String">
<column name="NAME" />
</property>
</class>
<class name="---.Country" table="COUNTRY">
<id name="id" type="long">
<column name="ID" />
<generator class="increment" />
</id>
<property name="name" type="java.lang.String">
<column name="NAME" />
</property>
</class>
<class name="---.Employee" table="EMPLOYEE"
batch-size="100">
<id name="id" type="long">
<column name="ID" />
<generator class="increment" />
</id>
<property name="name" type="java.lang.String">
<column name="NAME" />
</property>
<many-to-one name="address" class="---.Address"
fetch="join">
<column name="ADDRESS" unique="true" />
</many-to-one>
<map name="officePositions" table="EMPLOYEE_POSITION_OFFICE" lazy="false"
fetch="join" batch-size="100">
<key>
<column name="EMPLOYEE_ID"></column>
</key>
<map-key-many-to-many class="---.Office">
<column name="OFFICE_ID">
</column>
</map-key-many-to-many>
<many-to-many column="POSITION_ID"
class="---.Position" />
</map>
</class>
<class name="---.Office" table="OFFICE"
batch-size="400">
<id name="id" type="long">
<column name="ID" />
<generator class="increment" />
</id>
<many-to-one name="company" class=---.Company"
fetch="join">
<column name="COMPANY_ID" />
</many-to-one>
<many-to-one name="address" class="---.Address"
fetch="join">
<column name="ADDRESS_ID" />
</many-to-one>
<property name="collegues">
<formula>
SELECT COUNT(*) FROM EMPLOYEE_POSITION_OFFICE
E WHERE
E.OFFICE_ID=ID
</formula>
</property>
</class>
<class name="---.Position" table="POSITION">
<id name="id" type="long">
<column name="ID" />
<generator class="increment" />
</id>
<property name="name" type="java.lang.String">
<column name="NAME" />
</property>
</class>
Hibernate cfg:
<hibernate-configuration>
<session-factory>
<property name="connection.url">jdbc:oracle:thin:#localhost:1521:xe</property>
<property name="connection.driver_class">oracle.jdbc.driver.OracleDriver</property>
<property name="connection.username">---</property>
<property name="connection.password">----</property>
<property name="connection.pool_size">10</property>
<property name="current_session_context_class">thread</property>
<property name="dialect">org.hibernate.dialect.Oracle10gDialect</property>
<property name="hibernate.use_sql_comments">false</property>
<property name="hibernate.format_sql">false</property>
<property name="hibernate.show_sql">true</property>
<mapping resource="---/Country.hbm.xml" />
<mapping resource="---/City.hbm.xml" />
<mapping resource="---/Address.hbm.xml" />
<mapping resource="---/Company.hbm.xml" />
<mapping resource="---/Office.hbm.xml" />
<mapping resource="---/Position.hbm.xml" />
<mapping resource="---/Employee.hbm.xml" />
</session-factory>

Related

java.sql.SQLSyntaxErrorException: ORA-02289: sequence does not exist

In this method I am getting error java.sql.SQLSyntaxErrorException: ORA-02289: sequence does not exist I have gone through posts and I m not getting any Idea to fix it
public void saveOrUpdateProcessRun(ProcessRun argProcessRun) {
LOGGER.info(METHOD_START);
getHibernateTemplate().saveOrUpdate(argProcessRun);
LOGGER.info(METHOD_END);
}
This is Mapping I have XML Based Configurations Generator class is not Working(PROC_RUN_ID_SEQ)
<?xml version="1.0"?>
<!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
<hibernate-mapping package="com.uhg.esbdb.common.beans">
<class name="ProcessRun" table="PROC_RUN">
<id name="processRunID" type="integer" column="PROC_RUN_ID">
<generator class="sequence">
<param name="sequence">PROC_RUN_ID_SEQ</param>
</generator>
</id>
<property name="processID" type="integer" column="PROC_ID" />
<property name="processRunStartDatetime" type="timestamp"
column="PROC_RUN_STRT_DTTM" />
<property name="processRunEndDatetime" type="timestamp"
column="PROC_RUN_END_DTTM" />
<property name="processRunStatusCode" type="integer" column="PROC_RUN_STS_CD" />
<property name="createdByID" type="string" column="CRE_BY_ID" />
<property name="createdDatetime" type="timestamp" column="CRE_DTTM" />
<property name="modifiedByID" type="string" column="MOD_BY_ID" />
<property name="modifiedDatetime" type="timestamp" column="MOD_DTTM" />
</class>
<class name="FileLoad" table="FL_LOAD">
<id name="fileLoadID" type="integer" column="FL_LOAD_ID">
<generator class="sequence">
<param name="sequence">FL_LOAD_ID_SEQ</param>
</generator>
</id>
<property name="dataFileName" type="string" column="DATA_FL_NM" />
<property name="dataFileSizeByteNumber" type="integer"
column="DATA_FL_SZ_BYTE_NBR" />
<property name="fileLoadStatusCode" type="integer" column="FL_LOAD_STS_CD" />
<property name="loadStartDatetime" type="timestamp" column="LOAD_STRT_DTM" />
<property name="loadEndDatetime" type="timestamp" column="LOAD_END_DTM" />
<property name="createdByID" type="string" column="CRE_BY_ID" />
<property name="createdDatetime" type="timestamp" column="CRE_DTTM" />
<property name="modifiedByID" type="string" column="MOD_BY_ID" />
<property name="modifiedDatetime" type="timestamp" column="MOD_DTTM" />
<many-to-one name="processRun" class="ProcessRun" column="PROC_RUN_ID" />
</class>
<class name="ControlTotal" table="CTL_TOT">
<composite-id name="id" class="ControlTotalID">
<key-many-to-one name="fileLoad" class="FileLoad"
column="FL_LOAD_ID" />
<key-property name="controlTotalTypeCode" type="integer"
column="CTL_TOT_TYP_CD" />
</composite-id>
<property name="controlTotal" type="string" column="CTL_TOT" />
<property name="createdByID" type="string" column="CRE_BY_ID" />
<property name="createdDatetime" type="timestamp" column="CRE_DTTM" />
<property name="modifiedByID" type="string" column="MOD_BY_ID" />
<property name="modifiedDatetime" type="timestamp" column="MOD_DTTM" />
</class>
<!-- Added for loading wfg transaction ids from BE017 and BNKACH feeds to the EDB table EDBREF.WFG_ACH_PAYMENTS-->
<class name="WfgTransactionIdBean" table="EDBREF.WFG_ACH_PAYMENTS">
<id name="traceNumber" type="string" column="TRACE_NUMBER"></id>
<property name="sourceSystem" type="string" column="SOURCE_SYSTEM" />
<property name="transactionDate" type="timestamp" column="TRANSACTION_DATE" />
<property name="processed" type="string" column="PROCESSED" />
</class>
</hibernate-mapping>
Instead of PROC_RUN_ID_SEQ I am getting-> DEBUG main SQL logStatement:92 - select hibernate_sequence.nextval from dual
The causes can be different:
The name of the sequence you have mapped in your pojo is different
You have no permission to use sequence
You have no defined a sequence on your database
Solutions
Fix the sequence name on your pojo
Grant the sequence for your database user
Add the sequence on your database
EDIT AFTER UPDATED QUESTION
Try as follow:
<id name="propName" type="long" unsaved-value="null">
<column name="columnName" not-null="true" unique="true"
index="pkName" />
<generator
class="org.hibernate.id.enhanced.SequenceStyleGenerator">
<param name="optimizer">none</param>
<param name="increment_size">1</param>
<param name="sequence_name">PROC_RUN_ID_SEQ</param>
</generator>
</id>

Liquibase SYSDATE with loadData

I am trying to load in a CSV file using loadData, two of my columns are date times. I don't want to have to put the date time into the csv file, but have the current system time used --> SYSDATA (using Oracle).
I tried a couple things detailed below which didn't work:
File:
CONFIGURATION_ID~SERVICE_NAME~CATEGORY~CONFIGURATION_KEY~CONFIGURATION_VALUE~CREATE_TS~CREATED_BY~UPDATED_BY
3590~MobileCloudWallet~SYSTEM_PROPERTIES~VTS.Wallet.Provider.Type~XX,UGO,APPLE~26-SEP-16 09.52.05.708089000 AM~SYSTEM~
3591~MobileCloudWallet~SYSTEM_PROPERTIES~VTS.Wallet.Provider.Type.UGO.IneligibleCardList~4085869-4085869,4085868-4085868,4085860-4085860~26-SEP-16 09.52.05.730864000 AM~SYSTEM~
3592~MobileCloudWallet~SYSTEM_PROPERTIES~VTS.Credentials.Purpose.Type~Payment~26-SEP-16 09.52.05.740717000 AM~SYSTEM~
What I tried
A
<loadData encoding="UTF-8"
file="src/main/resources/data/configuration.tsv" quotchar=""
separator="~" tableName="CONFIGURATION">
<column name="CONFIGURATION_ID" type="NUMERIC" />
<column name="SERVICE_NAME" type="STRING" />
<column name="CATEGORY" type="STRING" />
<column name="CONFIGURATION_KEY" type="STRING" />
<column name="CONFIGURATION_VALUE" type="STRING" />
<column name="CREATE_TS" type="DATETIME" valueDate="SYSDATE" />
<column name="CREATED_BY" type="STRING" />
<column name="UPDATED_TS" type="DATE" valueDate="SYSDATE"/>
<column name="UPDATED_BY" type="STRING" />
</loadData>
B
<property name="now" value="sysdate" dbms="oracle" />
<changeSet...>
<loadData encoding="UTF-8"
file="src/main/resources/data/configuration.tsv" quotchar=""
separator="~" tableName="CONFIGURATION">
<column name="CONFIGURATION_ID" type="NUMERIC" />
<column name="SERVICE_NAME" type="STRING" />
<column name="CATEGORY" type="STRING" />
<column name="CONFIGURATION_KEY" type="STRING" />
<column name="CONFIGURATION_VALUE" type="STRING" />
<column name="CREATE_TS" type="DATETIME" defaultValueComputed="${now}" />
<column name="CREATED_BY" type="STRING" />
<column name="UPDATED_TS" type="DATE" defaultValueComputed="${now}"/>
<column name="UPDATED_BY" type="STRING" />
</loadData>
</changeset>
I was over-thinking things. I set the value of the data column in my CSV to CURRENT_TIMESTAMP, keep the column def in loadUpdateData as date and get rid of defaultValueComputed/valueDate. Worked great.

EF 5 with oracle edmx StoreGeneratedPattern="Identity" issue

We are using EF5 with Oracle[Oracle.manageddataaccess.client].Whenever I create/refresh edmx.I loose the property StoreGeneratedPattern="Identity". I had to set this manually for each and every Entity.
Is there any way to automate it or any other work around ? am I missing something here ?
We use VS2013,Below is the sample entity in the edmx after create.
<EntityType Name="ADDRESS">
<Key>
<PropertyRef Name="ADDRESS_ID" />
</Key>
<Property Name="ADDRESS_ID" Type="number" Precision="38" Scale="0" Nullable="false" />
<Property Name="CLIENT_ID" Type="number" Precision="38" Scale="0" Nullable="false" />
<Property Name="USER_ID" Type="number" Precision="38" Scale="0" />
<Property Name="ADDRESS_ACT_FLG" Type="number" Precision="38" Scale="0" />
<Property Name="ADDRESS_1" Type="varchar2" MaxLength="50" />
<Property Name="ADDRESS_2" Type="varchar2" MaxLength="50" />
<Property Name="ADDRESS_3" Type="varchar2" MaxLength="50" />
<Property Name="ADDRESS_4" Type="varchar2" MaxLength="50" />
<Property Name="ADDRESS_SUFFIX" Type="char" MaxLength="2" />
<Property Name="COUNTY" Type="varchar2" MaxLength="50" />
<Property Name="CITY" Type="varchar2" MaxLength="50" />
<Property Name="STATE_PROVINCE" Type="varchar2" MaxLength="75" />
<Property Name="COUNTRY" Type="varchar2" MaxLength="50" />
<Property Name="POSTAL_CODE" Type="varchar2" MaxLength="10" />
<Property Name="ADDRESS_TYPE" Type="varchar2" MaxLength="2" />
<Property Name="PRIMARY_ADDRESS_IND" Type="number" Precision="38" Scale="0" />
<Property Name="CREATED_USER_ID" Type="number" Precision="38" Scale="0" Nullable="false" />
<Property Name="CREATED_DATE" Type="timestamp" Precision="6" Nullable="false" />
<Property Name="MODIFIED_USER_ID" Type="number" Precision="38" Scale="0" Nullable="false" />
<Property Name="MODIFIED_DATE" Type="timestamp" Precision="6" Nullable="false" />
</EntityType>
I have written a short blog post about it: http://blog.aitgmbh.de/2014/06/02/patch-for-entity-framework-models-based-on-oracle-databases/
There I also refer to a NuGet package that I have created: http://bit.ly/1hbxIsO
This adds MSBuild scripts that manipulate the edmx file automatically.
It basically lets you define which columns should be identity columns and ensures at every build that the StoreGeneratedPattern property of these identity columns is set to "Identity".
Update: The patch is now also available on GitHub.

Sub query in Nhibernate

I have following structure.
1) GroupMember.cs
public class GroupMember
{
public virtual int Id { get; set; }
public virtual SystemUser User { get; set; }
public virtual Group Group { get; set; }
public virtual IList<Group> GroupDetail { get; set; }
public virtual IList<SystemUser> SystemUserDetail { get; set; }
// Few more Properties are there
}
2) SystemUser.cs
public class SystemUser
{
public virtual int Id{get;set;}
public virtual string DisplayName{get;set;}
// Few more Properties are there
}
Nhibernate files
GroupMembers
<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="NSP.DataModel"
namespace="NSP.DataModel.Account">
<class name="GroupMember" entity-name="SysGroupMember" table="SYS_DEF_GROUP_MEMBERS">
<id name="Id" column="id" type="Int32">
<generator class="identity"/>
</id>
<many-to-one entity-name="SysGroup" name="Group" column="GroupID" not-null="true" cascade="none"/>
<many-to-one entity-name="SysUser" name="User" column="UserID" not-null="false" cascade="none"/>
<property name="Status" type="int" not-null="false">
<column name="Status" not-null="false"/>
</property>
<property name="CreatedDate" type="datetime" not-null="false">
<column name="CreatedDate"/>
</property>
<property name="CreatedBy" type="int" not-null="false">
<column name="CreatedBy"/>
</property>
<property name="UpdatedDate" type="datetime" not-null="false">
<column name="UpdatedDate"/>
</property>
<property name="UpdatedBy" type="int" not-null="false">
<column name="UpdatedBy"/>
</property>
<bag name="GroupDetail" inverse="true">
<key column="Id"/>
<one-to-many entity-name="SysGroup"/>
</bag>
<bag name="SystemUserDetail">
<key column="Id"/>
<one-to-many entity-name="SysUser"/>
</bag>
</class>
</hibernate-mapping>
SysUser
<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="NSP.DataModel" namespace="NSP.DataModel.Authentication">
<class name="SystemUser" entity-name="SysUser" table="SYS_DEF_USER" abstract="true">
<id name="Id" column="id" type="Int32">
<generator class="identity"/>
</id>
<many-to-one entity-name="SysUserTypes" name="UserTypeId" column="UserTypeId" not-null="true" cascade="none" />
<property name="IsActive" column="IsActive" type="Boolean" not-null="true"/>
<property name="IsLicensed" column="IsLicensed" type="Boolean" not-null="true"/>
<property name="DisplayName" type="string" not-null="false">
<column name="DisplayName" length="128"/>
</property>
<property name="Email" column="Email" type="string" not-null="true" length="200"/>
<property name="PasswordMD5HexHash" column="PasswordMD5HexHash" type="string" not-null="false"/>
<bag name="UserTypeList" inverse="true">
<key column="UserTypeId"/>
<one-to-many entity-name="SysUserTypes"/>
</bag>
</class>
</hibernate-mapping>
I want to get the result using this query
select * from sys_def_user where id not in (select UserId from SYS_DEF_GROUP_MEMBERS where GroupID =5)
What can be the nhibernate syntax for this? Please its urget...
You can use NHibernate's LINQ provider for this. The examples listed here, http://code.msdn.microsoft.com/101-LINQ-Samples-3fb9811b, are a good starting point for learning LINQ. Most of those examples are written in the from ... select syntax, but I prefer the extension method syntax:
var subquery = session.Query<GroupMember>()
.Where(gm => gm.Group.Id == 5)
.Select(gm => gm.User.Id);
var users = session.Query<User>()
.Where(u => !subquery.Contains(u.Id));
Hopefully this example will get you started on the right track, and you'll be writing your own LINQ queries in no time.

NHibernate sluggish performance in FindAll(criteria) query

I have a very simple mapping file (se below) and a simple class.
<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="Domain"
namespace="Domain" default-access="field.camelcase-underscore"
default-lazy="true">
<class name="PricePropStateView" table="V_PRICE_PROP_STATES">
<id name="PriceId" column="PRICE_ID" type="long" />
<property name="DetailId" column="DETAILS_ID" type="long" />
<property name="Moe" column="MOE" type="string" />
<property name="PropId" column="PROP_ID" type="long" />
<property name="PoQteId" column="PO_QTE_ID" type="string" />
<property name="PoLineItemId" column="LINE_ITEM_ID" type="string" />
<property name="PropState" column="PROP_STATE" type="string" />
</class>
</hibernate-mapping>
The class represents a data set returned by a View in Oracle. The performance is very slow. When executed in Toad for Oracle, the result set is returned in less than a second. When using
DetachedCriteria criteria =
DetachedCriteria.For<PricePropStateView>()
.Add(Restrictions.Eq("PoQteId", aQuoteName));
return FindAll(criteria).ToList();
It is very slow...almost 29 seconds. Any ideas> Thanks

Resources