Encoding issue in CloudFoundry - spring-boot

I uploaded my web application to CloudFoundry, but my native language was broken as below.
enter image description here
I have checked that the DB has broken data.
DB uses ClearDB MySQL Database, which CloudFoundry provides as a service.
I ran the following query.
SHOW VARIABLES LIKE 'c%'
character_set_client utf8
character_set_connection utf8
character_set_database utf8
character_set_filesystem binary
character_set_results utf8
character_set_server latin1
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/
collation_connection utf8_general_ci
collation_database utf8_general_ci
collation_server latin1_swedish_ci
completion_type NO_CHAIN
concurrent_insert AUTO
connect_timeout 10
I suspect that the part that is 'latin' is guessing, but I do not know how to change that.
Is there someone you can help with this problem?
Thank you.

Depending on the particular ClearDB database server, there will be cases when your database's characters set is latin1 (the default configuration for MySQL older than 8.1).
We have contacted the ClearDB support team, but they were not willing to change this configuration because it could affect other customers hosted on the same server and they offered us a more expensive solution: to have our own dedicated servers.
The good point is that the character set can be configured per connection bases, so we decided to override the default DataSource, autoconfigured by the Java buildpack used by the Cloud Foundry.
In order to override the DataSource, we needed to do several things:
Add Maven dependency for Spring Cloud connectors,
Turn-off the default autoconfiguration of the DataSource bean,
Configure the DataSource bean by using connection parameters provided by the environment.
To add Spring Cloud connectors, add this dependency to your pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cloud-connectors</artifactId>
</dependency>
In order to turn off the auto-configuration, be sure to create Cloud bean in spring-boot cloud profile:
#Configuration
#Profile("cloud")
public class DataSourceConfiguration {
#Bean
public Cloud cloud() {
return new CloudFactory().getCloud();
}
}
Finally, to use database connection parameters hosted within VCAP_SERVICES environment variables, you can reference them in the application-cloud.yml in this way:
spring.datasource.url: ${cloud.services.mysql.connection.jdbcurl}&characterEncoding=utf-8
spring.datasource.username: ${cloud.services.mysql.connection.username}
spring.datasource.password: ${cloud.services.mysql.connection.password}
spring.datasource.testOnBorrow: true
When it comes to customizing DataSource configuration, it is important to pay attention to spring.datasource.url that ends with characterEncoding=utf-8: this is where connections are configured to use the UTF-8 encoding.
Also, mysql is the name of the PCF service instance in our case and can be different in your environment.
Finally, to learn more about binding to data services with Spring Boot, you can read this excellent article: https://spring.io/blog/2015/04/27/binding-to-data-services-with-spring-boot-in-cloud-foundry.

Related

Springboot: Can springboot access server container configuration files to read needed variables instaed from application.properties file

I know Springboot applications can use application.properties or application.yaml files to retrieve variables like database connectivity setting, username, password etc.
However, and because of sensitive information, our development team has these only for test environment but not for production environments. These are not available to developers.
These are provided by security team and set up directly on server in server configuration files on Liberty server (these are server.xml files located in the server installation directory).
Developers have their own instance of Liberty server running where they have their own versions of server.xml files. If we could make Springboot read these files, then we could make it mimic production server environments and make transition easier instead of reading local application.properties files.
Is this possible?
If not, what would be a workaround?
First, usually developer/application doesn't need direct access to props like database connectivity setting, username, password because all that is configured in server in data source configuration, so application just needs JNDI name of the datasource to connect to it.
Second, if you use technology that cannot be configured in server, developers and security team should utilize environment variables for such props.
Liberty can read env variables, or also define them via server.env and then utilize in server.xml config for example:
<dataSource jndiName="jdbc/myDS">
...
<properties.db2.jcc serverName="${JDBC_HOST}" portNumber="${JDBC_PORT}" databaseName="${JDBC_DB}" user="${JDBC_USER}" password="${JDBC_PASSWORD}"/>
</dataSource>
Similarly you can use env vars in your springboot either configuring it in application.properties like this:
jdbc.user=${JDBC_USER}
or directly in code:
#Value("${JDBC_USER}")
private String jdbcUser;
Utilizing env variables has additional benefit that you can override them later if deploying in containers for example.
I will talk about alternative solution.
Firstly I do not understand how developer will access production server properties.
And for securing sensitive properties you could use property encryptor tool. Sensitive properties will be in encrypted format in application.properties and during server startup it will decrypt encrypted properties and use accordingly.
Here is a such library for property encrytion library

How to pass database name explicitly in Spring Boot Configuration?

I've a Spring Boot application of mine, which connects to a Postgres database. I've specified in application.properties the datasource url as -
spring.datasource.url=jdbc:postgresql://< server ip here >:5432/mydb
The jdbc url (jdbc:postgresql://< server ip here >:5432/) is actually stored in a separate external location which my application is able to read. Therefore, I want to specify only the database name in my properties file.
I don't want to pass the database name as some environment variable since it's not going to change.
I'm stuck at this point for quite some time now, how can I achieve the same?
Add this in your application.properties file
spring.jpa.hibernate.ddl-auto=create\update\none
spring.datasource.url=jdbc:postgresql://host:port/db
spring.datasource.username=username
spring.datasource.password=password
Have you tried using ${var} syntax like:
spring.datasource.url=jdbc:postgresql://${server-ip}:5432/mydb
See:
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-application-property-files
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-placeholders-in-properties
I finally implemented it this way.
Specified only database name in my application and created Datasource bean in a separate Spring Boot application (so that it can be reused across other projects as well).

How to create fallback if spring boot fails to create datasource using JNDI look up

I have an application where we connect to database using JNDI which is set as
spring.datasource.jndi-name=java:jboss/jdbc/mydb
in application.properties
We don't want to use profiles as client is not inclined to use.
However in my local environment we want to use H2 database instead of oracle, so is it possible to create a fallback bean if JNDI lookup fails?
Looking if something like below is possible.
#ConditionalOnMissing
#Bean
MyBean mybean(){
//configure datasource using datasource builder
}
As I already replied to you on Gitter, you should move that to a profile specific configuration as it is production specific. I guess your client doesn't want to enable the profile externally or something. You can enable a profile using SpringApplicationBuilder: that way the profile is enabled by the app and won't be in tests.

Configuring Wildfly datasource for a JHipster application

I am to deploy a JHipster application to Wildfly (v. 9.0.2) servers and need to use JNDI data sources.
The basic connection setup is pretty straightforward: I set up new data source in Wildfly's standalone.xml to mimic the out-of-the-box ExampleDS, then included a reference to it into application-dev.yml and slightly modified DatabaseConfiguration.java. Now, that works well.
The problem is that application-xxx.yml files also contain JPA configuration, and I don't know where to stick it into Wildfly (I guess it must be in Wildfly).
To be more precise, I have no idea where to move the following JPA parameters:
jpa:
database-platform
database
openInView
show_sql
generate_ddl
hibernate:
ddl-auto
naming-strategy
properties:
hibernate.cache.use_second_level_cache
hibernate.cache.use_query_cache
hibernate.generate_statistics
hibernate.cache.region.factory_class
I know that Wildfly advises this to be defined in persistence.xml, in which case, can I just drop it into the project?
Thank you in advance.

Configure JDBC oracle specific property v$session.program using Jboss and JPA (hibernate)

I'd like to set the v$session.program Oracle property in order to have information available in the session table. I'm using JPA with a jndi XA datasource created with an oracle-xa-ds.xml deployed in the deploy folder of Jboss, and therefore I haven't access to the constructor of the Connection.
I have access to the Connection object, in JPA 2 using unwrap, in JPA 1 by casting JPA to Hibernate classes, but there are no properties setter (only Client Info properties that are the way to proceed starting JDBC 4.0).
So my question is, using JPA (with Hibernate) using Jboss 4.2 :
Is it possible to configure the v$session.program in the persistence.xml ?
Is it possible to configure the v$session.program in the oracle-ds.xml ?
Is their any other approach to the solution ?
Thank you for any valuable comments and answers !
I had the same Problem today, after much fiddeling and reading documentation finally I had the Eureka moment:
Add following parameter:
<xa-datasource-property name="connectionProperties">v$session.program=YourUniqueName</xa-datasource-property>
Thats all.
I'm pretty sure this must be documented somewhere but here is what we can find in the JBoss wiki:
How To Specify "PROGRAM" Oracle Connection Property
JBoss Version: JBoss 4.0.3 SP1, Oracle DB Version: 10g
To be able to distinguish the JDBC
connections on the Oracle server side,
which are created by different JBoss
instances, Oracle's PROGRAM connection
property might be set within the
Oracle specific JDBC datasource config
file by using the following tags:
<connection-property name="v$session.program">ADistinguishedNameForPROGRAMProperty</connection-property>
i.e.
...
<connection-url>AConnectionURL</connection-url>
<connection-property name="v$session.program">ADistinguishedNameForPROGRAMProperty</connection-property>
<driver-class>oracle.jdbc.OracleDriver</driver-class>
...
That way the DBAs can have proper
granularity in:
AWRs
v$session view
Other tools which are checking/evaluating PROGRAM connection
property

Resources