Trying to make my spring boot JPA application compliant with Oracle DB, already running with MySQL and H2.
The data generated by liquibase unfortunately uses some of Oracle's reserved keywords as table or column names.
The good news is that hibernate and liquibase implementations can detect these keywords and "quote" them when querying database (using objectQuotingStrategy="QUOTE_ONLY_RESERVED_KEYWORDS" for liquibase, and spring.jpa.properties.hibernate.auto_quote_keyword: true for hibernate).
The bad news is hibernate and liquibase do not share the same list of reserved keywords for Oracle.
For example, value is not recognized as a reserved keyword by liquibase, but is by hibernate (which uses ANSI SQL:2003 keywords).
One of my liquibase changeSets creates a table with a lower case value column, so Liquibase creates the table with an unquoted lowercase value column, and Oracle DB turns it automatically in an upper case VALUE column. Now when hibernate tries to fetch
that column, it recognizes value and quotes it (`SELECT "value" from ...), which makes it case-sensitive, so the column is not found (ORA-00904).
I thought I found a workaround for it by extending SpringLiquibase and adding my custom keywords, as described here : https://liquibase.jira.com/browse/CORE-3324. The problem is that this does not seem to work with OracleDatabase implementation, which overwrites SpringLiquibase's set of reserved keywords (and of course, the isReservedWord() method uses OracleDatabase's set).
For now, I'll use the QUOTE_ALL_OBJECTS quoting strategy for liquibase and hibernate.globally_quoted_identifiers.
But, just out of curiosity, I wanted to know if the set of reserved keywords used by liquibase for Oracle could be appended.
spring boot version: 2.3.9.RELEASE.
hibernate-core version (spring boot dependency): 5.4.28
liquibase-core version (spring boot dependency): 3.8.9
Hmm in case of Oracle you have keywords and reserved words.
reserved words can not be used as identifiers
keywords can be used as identifiers but it is not recommened.
You can get list of them directly from database:
select KEYWORD, RESERVED from v$reserved_words;
...
1864 rows selected
What about using uppercase names everywhere in the source code?
It looks like Liqubase depends on some JDBC driver functionally - which does not work.
OracleDatabase.java:
public void setConnection(DatabaseConnection conn) {
//noinspection HardCodedStringLiteral,HardCodedStringLiteral,HardCodedStringLiteral,HardCodedStringLiteral,
// HardCodedStringLiteral
reservedWords.addAll(Arrays.asList("GROUP", "USER", "SESSION", "PASSWORD", "RESOURCE", "START", "SIZE", "UID", "DESC", "ORDER")); //more reserved words not returned by driver
Connection sqlConn = null;
if (!(conn instanceof OfflineConnection)) {
try {
/*
* Don't try to call getWrappedConnection if the conn instance is
* is not a JdbcConnection. This happens for OfflineConnection.
* see https://liquibase.jira.com/browse/CORE-2192
*/
if (conn instanceof JdbcConnection) {
sqlConn = ((JdbcConnection) conn).getWrappedConnection();
}
} catch (Exception e) {
throw new UnexpectedLiquibaseException(e);
}
if (sqlConn != null) {
tryProxySession(conn.getURL(), sqlConn);
try {
//noinspection HardCodedStringLiteral
reservedWords.addAll(Arrays.asList(sqlConn.getMetaData().getSQLKeywords().toUpperCase().split(",\\s*")));
} catch (SQLException e) {
//noinspection HardCodedStringLiteral
Scope.getCurrentScope().getLog(getClass()).info("Could get sql keywords on OracleDatabase: " + e.getMessage());
//can not get keywords. Continue on
}
If Liquibase calls sqlConn.getMetaData().getSQLKeywords() and this does not return proper output, then your chances are limited. It might be a bug in JDBC drivers, or your application does not have SELECT_CATALOG_ROLE privilege and does not see v$reserved_words view (if JDBC queries this internally).
Related
I currently use the spring data solr library and implement its repository interfaces, I'm trying to add functionality to one of my custom queries that uses a Solr template with a SimpleQuery. it currently uses paging which appears to be working well, however, I want to use a Group field so sibling products are only counted once, at their first occurrence. I have set the group field on the query and it works well, however, it still seems to be using the un-grouped number of documents when constructing the page attributes.
is there a known work around for this?
the query syntax provides the following parameter for this purpose, but it would seem that Spring Data Solr isn’t taking advantage of it. &group.ngroups=true should return the number of groups in the result and thus give a correct page numbering.
any other info would be appreciated.
There are actually two ways to add this parameter.
Queries are converted to the solr format using QueryParsers, so it would be possible to register a modified one.
QueryParser modifiedParser = new DefaultQueryParser() {
#Override
protected void appendGroupByFields(SolrQuery solrQuery, List<Field> fields) {
super.appendGroupByFields(solrQuery, fields);
solrQuery.set(GroupParams.GROUP_TOTAL_COUNT, true);
}
};
solrTemplate.registerQueryParser(Query.class, modifiedParser);
Using a SolrCallback would be a less intrusive option:
final Query query = //...whatever query you have.
List<DomainType> result = solrTemplate.execute(new SolrCallback<List<DomainType>>() {
#Override
public List<DomainType> doInSolr(SolrServer solrServer) throws SolrServerException, IOException {
SolrQuery solrQuery = new QueryParsers().getForClass(query.getClass()).constructSolrQuery(query);
//add missing params
solrQuery.set(GroupParams.GROUP_TOTAL_COUNT, true);
return solrTemplate.convertQueryResponseToBeans(solrServer.query(solrQuery), DomainType.class);
}
});
Please feel free to open an issue.
I'm trying to update my MongoDB libraries to the latest, and now have a couple of errors, which might or might not be related. The first is in my applicationContext.xml, where I have the error "Factory bean 'customConversions' not found" next to this section:
<mongo:mapping-converter base-package="com.my.model">
<mongo:custom-converters base-package="com.my.model.converters">
</mongo:custom-converters>
</mongo:mapping-converter>
I can't see from the docs anything I might be missing. What could be causing this, and what can I do to fix?
If I try to run the app, I now get:
org.springframework.data.mapping.model.MappingException: No mapping metadata found for java.util.Date
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:206) ~[spring-data-mongodb-1.1.1.RELEASE.jar:na]
I'm using the following Maven Dependencies:
org.springframework.data: spring-data-mongodb: 1.1.1.RELEASE
org.springframework: core, spring-context, etc.: 3.2.1.RELEASE
Is this just a broken release, or am I doing something else wrong? I had no issues using java.util.Date in my model classes before.
Did you add it to the MongoTemplate? http://static.springsource.org/spring-data/data-mongo/docs/1.0.0.M5/reference/html/#d0e2718
OK, this is some time later, but it might benefit people upgrading code and database from legacy MongoDB versions.
I think that Mongo changed the way some data was stored internally. Or it could be that we exported data to JSON and then imported it again. Either way, we were left with data that had both incorrect Date and incorrect ObjectId representations. spring-data-mongo used to handle this but for whatever reason don't any longer. The fix for us was to run the following type of script in the Mongo shell:
db.entity.find().forEach(
function(o){
delete o._id;
if (typeof(o.createdTs) !== 'undefined' && typeof(o.createdTs.sec) !== 'undefined'){
o.createdTs = ISODate(o.createdTs.sec);
}
if (typeof(o.updatedTs) !== 'undefined' && typeof(o.updatedTs.sec) !== 'undefined'){
o.updatedTs = ISODate(o.updatedTs.sec);
}
try{
db.entity2.insert( o );
} catch (err){
print("Following node conversion failed. Error is: " + err);
printjson(o);
}
}
);
db.entity2.renameCollection('entity', true);
Now this worked for us because we weren't using the Mongo Object ID at all - we've been using a different, uniquely indexed UUID field as an ID instead. If you're referring to objectId at all anywhere else, you will need to create an objectId from the old string id and use that.
This has enabled us to upgrade to spring-data-1.1.0 and beyond, and has meant that we can now introduce spring-data-neo4j, which we were previously unable to do with this project due to this issue.
I had the same mapping exception (org.springframework.data.mapping.model.MappingException). One of the dates in the MongoDB records somehow had a date in the following format that could not be decoded by java.util.date:
"createdTime": {
"dateTime": ISODate("2016-09-15T02:01:00.560Z"),
"offset": {
"_id": "Z",
"totalSeconds": 0
},
"zone": {
"_class": "java.time.ZoneOffset",
"_id": "Z",
"totalSeconds": 0
}
}
Everything worked fine after I deleted that record.
I'm using spring's JdbcDaoSupport for making data base call. I want to execure native query (sql query) for retrieving data. Do we have any API available in JdbcTemplate for native query? I used queryForObject but it throws exception if there is no data whereas i was expecting it to return back null if it couldn't find data.
There are many options available for executing native sql with JdbcTemplate. The linked documentation contains plenty of methods that take native sql, and usually some sort of callback handler, which will accomplish exactly what you are looking for. A simple one that comes to mind is query(String sql, RowCallbackHandler callback).
jdbcTemplate.query("select * from mytable where something > 3", new RowCallbackHandler() {
public void processRow(ResultSet rs) {
//this will be called for each row. DO NOT call next() on the ResultSet from in here...
}
});
Spring JdbcTemplate's queryForObject method expects your SQL to return exactly one row. If the there are no rows returned or if there are more than 1 row returned it will throw a org.springframework.dao.IncorrectResultSizeDataAccessException. You will have to wrap the call to queryForObject with a try catch block to handle IncorrectResultSizeDataAccessException and return null if the exception is thrown
e.g.
try{
return jdbcTemplate.queryForObject(...);
}catch(IncorrectResultSizeDataAccessException e){
return null;
}
I'm using LLBLGen and I have some code like so:
if (onlyRecentMessages)
{
messageBucket.PredicateExpression.Add(MessageFields.DateEffective >= DateTime.Today.AddDays(-30));
}
var messageEntities = new EntityCollection<MessageEntity>();
using (var myAdapter = PersistenceLayer.GetDataAccessAdapter())
{
myAdapter.FetchEntityCollection(messageEntities, messageBucket);
}
I'm currently getting a SqlException on the FetchEntityCollection line. The error is:
System.Data.SqlClient.SqlException: The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Too many parameters were provided in this RPC request. The maximum is 2100.
but that's a side note. What I actually want to be able to do is include the generated SQL in a custom exception in my code. So for instance something like this:
using (var myAdapter = PersistenceLayer.GetDataAccessAdapter())
{
try
{
myAdapter.FetchEntityCollection(messageEntities, messageBucket);
}
catch (SqlException ex)
{
throw new CustomSqlException(ex, myAdapter.GeneratedSqlFromLastOperation);
}
}
Of course, there is no such property as GeneratedSqlFromLastOperation. I'm aware that I can configure logging, but I would prefer to have the information directly in my stack track / exception so that my existing exception logging infrastructure can provide me with more information when these kinds of errors occur.
Thanks!
Steve
You should get an ORMQueryExecutionException, which contains the full query in the description. The query's execute method wraps all exceptions in an ORMQueryExecutionException and stores the query in the description.
ps: please, if possible ask llblgen pro related questions on our forums, as we don't monitor stackoverflow frequently. Thanks. :)
What new features in java 7 is going to be implemented?
And what are they doing now?
Java SE 7 Features and Enhancements from JDK 7 Release Notes
This is the Java 7 new features summary from the OpenJDK 7 features page:
vm JSR 292: Support for dynamically-typed languages (InvokeDynamic)
Strict class-file checking
lang JSR 334: Small language enhancements (Project Coin)
core Upgrade class-loader architecture
Method to close a URLClassLoader
Concurrency and collections updates (jsr166y)
i18n Unicode 6.0
Locale enhancement
Separate user locale and user-interface locale
ionet JSR 203: More new I/O APIs for the Java platform (NIO.2)
NIO.2 filesystem provider for zip/jar archives
SCTP (Stream Control Transmission Protocol)
SDP (Sockets Direct Protocol)
Use the Windows Vista IPv6 stack
TLS 1.2
sec Elliptic-curve cryptography (ECC)
jdbc JDBC 4.1
client XRender pipeline for Java 2D
Create new platform APIs for 6u10 graphics features
Nimbus look-and-feel for Swing
Swing JLayer component
Gervill sound synthesizer [NEW]
web Update the XML stack
mgmt Enhanced MBeans [UPDATED]
Code examples for new features in Java 1.7
Try-with-resources statement
this:
BufferedReader br = new BufferedReader(new FileReader(path));
try {
return br.readLine();
} finally {
br.close();
}
becomes:
try (BufferedReader br = new BufferedReader(new FileReader(path)) {
return br.readLine();
}
You can declare more than one resource to close:
try (
InputStream in = new FileInputStream(src);
OutputStream out = new FileOutputStream(dest))
{
// code
}
Underscores in numeric literals
int one_million = 1_000_000;
Strings in switch
String s = ...
switch(s) {
case "quux":
processQuux(s);
// fall-through
case "foo":
case "bar":
processFooOrBar(s);
break;
case "baz":
processBaz(s);
// fall-through
default:
processDefault(s);
break;
}
Binary literals
int binary = 0b1001_1001;
Improved Type Inference for Generic Instance Creation
Map<String, List<String>> anagrams = new HashMap<String, List<String>>();
becomes:
Map<String, List<String>> anagrams = new HashMap<>();
Multiple exception catching
this:
} catch (FirstException ex) {
logger.error(ex);
throw ex;
} catch (SecondException ex) {
logger.error(ex);
throw ex;
}
becomes:
} catch (FirstException | SecondException ex) {
logger.error(ex);
throw ex;
}
SafeVarargs
this:
#SuppressWarnings({"unchecked", "varargs"})
public static void printAll(List<String>... lists){
for(List<String> list : lists){
System.out.println(list);
}
}
becomes:
#SafeVarargs
public static void printAll(List<String>... lists){
for(List<String> list : lists){
System.out.println(list);
}
}
New Feature of Java Standard Edition (JSE 7)
Decorate Components with the JLayer Class:
The JLayer class is a flexible and powerful decorator for Swing components. The JLayer class in Java SE 7 is similar in spirit to the JxLayer project project at java.net. The JLayer class was initially based on the JXLayer project, but its API evolved separately.
Strings in switch Statement:
In the JDK 7 , we can use a String object in the expression of a switch statement. The Java compiler generates generally more efficient bytecode from switch statements that use String objects than from chained if-then-else statements.
Type Inference for Generic Instance:
We can replace the type arguments required to invoke the constructor of a generic class with an empty set of type parameters (<>) as long as the compiler can infer the type arguments from the context. This pair of angle brackets is informally called the diamond.
Java SE 7 supports limited type inference for generic instance creation; you can only use type inference if the parameterized type of the constructor is obvious from the context. For example, the following example does not compile:
List<String> l = new ArrayList<>();
l.add("A");
l.addAll(new ArrayList<>());
In comparison, the following example compiles:
List<? extends String> list2 = new ArrayList<>();
l.addAll(list2);
Catching Multiple Exception Types and Rethrowing Exceptions with Improved Type Checking:
In Java SE 7 and later, a single catch block can handle more than one type of exception. This feature can reduce code duplication. Consider the following code, which contains duplicate code in each of the catch blocks:
catch (IOException e) {
logger.log(e);
throw e;
}
catch (SQLException e) {
logger.log(e);
throw e;
}
In releases prior to Java SE 7, it is difficult to create a common method to eliminate the duplicated code because the variable e has different types.
The following example, which is valid in Java SE 7 and later, eliminates the duplicated code:
catch (IOException|SQLException e) {
logger.log(e);
throw e;
}
The catch clause specifies the types of exceptions that the block can handle, and each exception type is separated with a vertical bar (|).
The java.nio.file package
The java.nio.file package and its related package, java.nio.file.attribute, provide comprehensive support for file I/O and for accessing the file system. A zip file system provider is also available in JDK 7.
Source: http://ohmjavaclasses.blogspot.com/
Java Programming Language Enhancements # Java7
Binary Literals
Strings in switch Statement
Try with Resources (How it works) or ARM (Automatic Resource Management)
Multiple Exception Handling
Suppressed Exceptions
underscore in literals
Type Inference for Generic Instance Creation using Diamond Syntax
Improved Compiler Warnings and Errors When Using Non-Reifiable Formal Parameters with Varargs Methods
Official reference
Official reference with java8
wiki reference
In addition to what John Skeet said, here's an overview of the Java 7 project. It includes a list and description of the features.
Note: JDK 7 was released on July 28, 2011, so you should now go to the official java SE site.
Language changes:
-Project Coin (small changes)
-switch on Strings
-try-with-resources
-diamond operator
Library changes:
-new abstracted file-system API (NIO.2) (with support for virtual filesystems)
-improved concurrency libraries
-elliptic curve encryption
-more incremental upgrades
Platform changes:
-support for dynamic languages
Below is the link explaining the newly added features of JAVA 7 , the explanation is crystal clear with the possible small examples for each features :
http://radar.oreilly.com/2011/09/java7-features.html
Using Diamond(<>) operator for generic instance creation
Map<String, List<Trade>> trades = new TreeMap <> ();
Using strings in switch statements
String status= “something”;
switch(statue){
case1:
case2:
default:
}
Underscore in numeric literals
int val 12_15;
long phoneNo = 01917_999_720L;
Using single catch statement for throwing multiple exception by using “|” operator
catch(IOException | NullPointerException ex){
ex.printStackTrace();
}
No need to close() resources because Java 7 provides try-with-resources statement
try(FileOutputStream fos = new FileOutputStream("movies.txt");
DataOutputStream dos = new DataOutputStream(fos)) {
dos.writeUTF("Java 7 Block Buster");
} catch(IOException e) {
// log the exception
}
binary literals with prefix “0b” or “0B”
I think ForkJoinPool and related enhancement to Executor Framework is an important addition in Java 7.
The following list contains links to the the enhancements pages in the Java SE 7.
Swing
IO and New IO
Networking
Security
Concurrency Utilities
Rich Internet Applications (RIA)/Deployment
Requesting and Customizing Applet Decoration in Dragg able Applets
Embedding JNLP File in Applet Tag
Deploying without Codebase
Handling Applet Initialization Status with Event Handlers
Java 2D
Java XML – JAXP, JAXB, and JAX-WS
Internationalization
java.lang Package
Multithreaded Custom Class Loaders in Java SE 7
Java Programming Language
Binary Literals
Strings in switch Statements
The try-with-resources Statement
Catching Multiple Exception Types and Rethrowing Exceptions with Improved Type Checking
Underscores in Numeric Literals
Type Inference for Generic Instance Creation
Improved Compiler Warnings and Errors When Using Non-Reifiable Formal Parameters with Varargs Methods
Java Virtual Machine (JVM)
Java Virtual Machine Support for Non-Java Languages
Garbage-First Collector
Java HotSpot Virtual Machine Performance Enhancements
JDBC
Reference 1 Reference 2