I'm building a Play! Framework (2.5) Java application and I want to connect it to ElasticSearch using Ebean.
First, I followed the steps here and successfully connected to a dummy jdbc (Added plugin, modified build.sbt, added ebean.default="models.*", default.driver = org.h2.Driver, and default.url = "jdbc:h2:mem:play" to application.conf). So far so good.
Now, I want to switch my connection to ElasticSearch instead of jdbc, so I'm following the steps here, but I can't figure it out for the life of me. What I've done so far:
Added to build.sbt:
libraryDependencies ++= Seq( "org.avaje.ebeanorm" % "avaje-ebeanorm-elastic" % "1.3.1" )
Added to application.conf:
ebean.docstore.url="http://localhost:9200"
ebean.docstore.active=true
ebean.docstore.generateMapping=true
ebean.docstore.dropCreate=true`
Removed from application.conf
default.driver = org.h2.Driver
default.url = "jdbc:h2:mem:play"
I cannot get this code to run, because I keep getting this error:
RuntimeException: com.typesafe.config.ConfigException$WrongType: application.conf # file:/C:/Users/../conf/application.conf: 357-360: docstore has type OBJECT rather than LIST
I would greatly appreciate any pointers in the right direction. Thanks!
You can either :
Override your ebean configuration programmaticaly
Add your configuration in a ebean property file
1 - Programmatically
package models;
import com.avaje.ebean.config.ServerConfig;
import com.avaje.ebean.event.ServerConfigStartup;
public class ConfigOverride implements ServerConfigStartup {
public void onStart(ServerConfig serverConfig) {
serverConfig.getDocStoreConfig().setUrl("http://localhost:9200");
serverConfig.getDocStoreConfig().setActive(true);
serverConfig.getDocStoreConfig().setGenerateMapping(false);
serverConfig.getDocStoreConfig().setDropCreate(false);
serverConfig.getDocStoreConfig().setCreate(false);
serverConfig.getDocStoreConfig().setPathToResources("conf");
}
}
2 - Configuration
You have to put your ebean.docstore properties in a ebean.properties in conf directory. But unfortunatly, I didn't manage to make the application work in stage mode
Source: https://raw.githubusercontent.com/playframework/play-ebean/master/docs/manual/working/javaGuide/main/sql/JavaEbean.md
FYI : I opened an issue here :
https://github.com/playframework/play-ebean/issues/104
Related
Websphere version 9.0 is installed in our RHEL 8.3 OS .
Now in that i have deployed one web app - .war file which contains multiple modules - webservice, web module etc.
This war is successfully deployed and i am able to start it also going to Websphere Enterprise Applications - AppName - START.
The app gets started with a success message.
Now the problem lies ahead. Our application requires a certain file bootstrap.properties.
This file has several configurations like jdbc params, jmx ports, jms configurations, jvm arguments, logging paths etc.
Once the web module of this app is run on <SERVER_IP>:9080/Context url, it throws error on GUI saying Unable to locate bootstrap.properties.
Analysing at the code level , found out that below code is throwing this error:
private static Properties config;
private static final String CONFIG_ROOT = System.getProperty("bootstrap.system.propertiespath");
private static final String configFile = "bootstrap.properties";
private JMXConfig() {
}
public static String getConfigRoot() {
if (CONFIG_ROOT == null) {
System.err.println("Not able to locate bootstrap.properties. Please configure bootstrap.system.propertiespath property.");
throw new ConfigException("Unable to locate bootstrap.properties.");
} else {
return CONFIG_ROOT + File.separator;
}
}
I wanted to know where can we specify the absolute paths in the websphere console where our property file can be read as a system argument once the application is loaded.
Since you're using System.getProperty() to read the property, it needs to be specified as a Java system property passed into the JVM. You can do that from the JVM config panel, adding it as either a custom property on the JVM or as a -D option in the server's generic JVM arguments.
Custom property: https://www.ibm.com/docs/en/was/9.0.5?topic=jvm-java-virtual-machine-custom-properties
Generic JVM argument: https://www.ibm.com/docs/en/was/9.0.5?topic=jvm-java-virtual-machine-settings (search for "Generic JVM arguments")
Note that if you use a custom property, you would simply set the "name" field to "bootstrap.system.propertiespath" and the "value" to the path you need; if you use a generic JVM argument, you'd add an argument with the structure "-Dbootstrap.system.propertiespath=/path/to/file".
There are several ways present to define "active profiles" for a Spring Boot application.
The default one is to pass it through a command line, like this:
java -Dspring.profiles.active=dev,local -jar myapp.jar
it works just fine (as expected): All three sets of profile-related properties will be loaded in proper order:
application.yaml
application-dev.yaml will override the previous one
application-local.yaml will override the previous one as well (these properties will have the most priority)
Based on the idea, that my "local" profile should always "use and overrides" properties from the "dev", let's "hardcode" such behavior.
Let's use the 'spring.profiles.include' feature for this. So, the following lines are added to the 'application-local.yaml':
spring.profiles:
include:
- dev
I expect, now I can pass the "local" profile only in the command line, and the "dev" profile will be applied automatically (with his properties, of course):
java -Dspring.profiles.active=local -jar myapp.jar
But ooop!*: properties from the 'application-dev.yaml' are ignored.
Why? Is it a bug? Is it a feature that forces me to list all profiles in a command line directly?
I'm sure that the behavior around profiles activation should be the same without any difference in how the active-profiles list was passed to Spring Boot framework.
The application:
#SpringBootApplication #EnableConfigurationProperties( MyProps::class )
class SpringApp4
#ConfigurationProperties("my.db") #ConstructorBinding
data class MyProps(val name: String, val url: String, val user: String)
#Component
class MyRunner(val myProps: MyProps, val env: Environment) : CommandLineRunner {
override fun run(vararg args: String) {
println("myProps = $myProps")
println("activeProfiles = ${env.activeProfiles.joinToString()}")
exitProcess(0)
}
}
fun main() { runApplication<SpringApp4>() }
application.yaml:
my.db:
name: "default-name"
url: "default-url"
user: "default-user"
application-dev.yaml:
my.db:
url: "dev-url"
user: "dev-user"
application-local.yaml:
spring.profiles.include:
- dev
my.db:
user: "local-user"
Run1: java -Dspring.profiles.active=dev,local -jar myapp.jar
Correct output:
myProps = MyProps(name=default-name, url=dev-url, user=local-user)
activeProfiles = dev, local
it's correct because the url=dev-url
Run2: java -Dspring.profiles.active=local -jar myapp.jar
Incorrect output:
myProps = MyProps(name=default-name, url=default-url, user=local-user)
activeProfiles = local
It's not correct because the url=default-url and the activeProfiles doesn't contain the "dev" at all.
Help me please to figure out how to use the spring.profiles.include feature in yaml to build a kind of top level profiles that will activate other automatically.
In Run2 You are giving profile as local
i.e
-Dspring.profiles.active=local
So spring will first load application.yml and then application-local.yml
I can see the output is expected.
Since some properties like name and url are not present in application-local.yml, so the values of these fields will be same as present in application.yml
FYI : application.yml is always called irrespective of profile, and then it gets overridden by the profile mentioned in -Dspring.profiles.active property
spring.profiles.include deprecated in Spring Boot 2.4 and no longer works: https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4
It caused recursive resource loading; that broke Kubernates ConfigMap so they removed recursion.
Use spring.profiles.active or spring.profiles.group.
Trying to get the combo jython-zxJDBC-UCanAccess working on my Windows machine; been driving me nuts the entire day as I keep getting
zxJDBC.DatabaseError ( driver [net.ucanaccess.jdbc.UcanaccessDriver] not found ), and
zxJDBC.DatabaseError ( driver [ucanaccess] not found )
I had been using this sample code from Gord Thompson with my particulars - keeps failing with driver-not-found errors whatever I try:
from com.ziclix.python.sql import zxJDBC
#Or also import ucanaccess
jdbc_url = "jdbc:ucanaccess://Z:/Companies.accdb"
username = ""
password = ""
driver_class = "net.ucanaccess.jdbc.UcanaccessDriver"
#Or driver_class = "net.ucanaccess.jdbc.UcanloadDriver"
cnxn = zxJDBC.connect(jdbc_url, username, password, driver_class)
crsr = cnxn.cursor()
crsr.execute("SELECT * FROM tblSICs")
for row in crsr.fetchall():
print row[0]
crsr.close()
cnxn.close()
Jython alone works well, I use it with the SikuliX IDE
'from ... import zxJDBC' passes without error
UCanAccess works fine from the command line (console.bat) with my MS
Access database; all 5 JARs are in and as in the CLASSPATH variable
What I don't get is what is "driver_class"?. I thought that the database-specific driver gets located by means of Jython/SikuliX looking in CLASSPATH, where the '.../ucanaccess-5.0.0.jar' is seen.
Here the CLASSPATH:
"c:\Users\User\AppData\Roaming\UCanAccess\lib\hsqldb-2.5.0.jar;c:\Users\User\AppData\Roaming\UCanAccess\lib\jackcess-3.0.1.jar;c:\Users\User\AppData\Roaming\UCanAccess\lib\commons-lang3-3.8.1.jar;c:\Users\User\AppData\Roaming\UCanAccess\lib\commons-logging
-1.2.jar;c:\Users\User\AppData\Roaming\UCanAccess\ucanaccess-5.0.0.jar"
And here is where the SikuliX IDE looks for resources:
D:\Drawer 3\Sikuli\Training\UCanAccess.sikuli Z:\test.sikuli
C:\Users\User\AppData\Roaming\Sikulix\Lib\site-packages
C:\Users\User\AppData\Roaming\Sikulix\Lib
C:\Users\User\AppData\Roaming\Sikulix\Extensions\Lib
C:\Users\User\AppData\Roaming\Sikulix\Extensions\jython-standalone-2.7.2.jar\Lib
classpath
pyclasspath/
Does anyone spot the issue?
Issue was found to be a special twist of SikuliX 2.0.4 which cannot evaluate the CLASSPATH variable. Here the workarounds / SiluliX' way to get to know the location of JARs:
Two Solutions:
Solution A
SikuliX IDE looks into the folder
<C:\Users\User\AppData\Roaming\Sikulix\Extensions>; all five relevant
JAR files need to be places in that folder (no 'lib' subfolder as in
the deployment pack):
ucanaccess-5.0.0.jar
commons-lang3-3.8.1.jar
commons-logging-1.2.jar
hsqldb-2.5.0.jar
jackcess-3.0.1.jar
Check the extension JARs that get found by seeing
SikuliX->Tools->Extensions...
Solution B
Leave the needed JARs in their original deployment folder, and add
their path to the 'Special File'
'C:\Users\User\AppData\Roaming\Sikulix\Extensions\extensions.txt' as:
ucanaccess = C:/Users/User/AppData/Roaming/UCanAccess/ucanaccess-5.0.0.jar
C:/Users/User/AppData/Roaming/UCanAccess/lib/commons-lang3-3.8.1.jar
C:/Users/User/AppData/Roaming/UCanAccess/lib/commons-logging-1.2.jar
C:/Users/User/AppData/Roaming/UCanAccess/lib/hsqldb-2.5.0.jar
C:/Users/User/AppData/Roaming/UCanAccess/lib/jackcess-3.0.1.jar
This will be evaluated, as a workaround, instead of the CLASSPATH
variable
Thanks for your help!
i'm trying to pass command line arguments in my Spring Boot Yarn application and am having difficulties. i understand that i can set these in the yml document spring.yarn.appmaster.launchcontext.arguments but how can it from the command line? like java -jar MyYarnApp.jar {arg0} {arg1} and get access to it from my #YarnContainer?
i've discovered that #YarnProperties maps to spring.yarn.appmaster.launchcontext.arguments but i want to set them from the command line, not in the yml
You are pretty close on this when you found spring.yarn.client.launchcontext.arguments and spring.yarn.appmaster.launchcontext.arguments. We don't have settings which would automatically pass all command-line arguments from a client into an appmaster which would then pass them into a container launch context. Not sure if we even want to do that because you surely want to be on control what happens with YARN container launch context. User using a client could then potentially pass a rogue arguments along a food chain.
Having said that, lets see what we can do with our Simple Single Project YARN Application Guide.
We still need to use those launch context arguments to define our command line parameters to basically map how things are passed from a client into an appmaster into a container.
What I added in application.yml:
spring:
yarn:
client:
launchcontext:
arguments:
--my.appmaster.arg1: ${my.client.arg1:notset1}
appmaster:
launchcontext:
arguments:
--my.container.arg1: ${my.appmaster.arg1:notset2}
Modified HelloPojo in Application class:
#YarnComponent
#Profile("container")
public static class HelloPojo {
private static final Log log = LogFactory.getLog(HelloPojo.class);
#Autowired
private Configuration configuration;
#Value("${my.container.arg1}")
private String arg1;
#OnContainerStart
public void onStart() throws Exception {
log.info("Hello from HelloPojo");
log.info("Container arg1 value is " + arg1);
log.info("About to list from hdfs root content");
FsShell shell = new FsShell(configuration);
for (FileStatus s : shell.ls(false, "/")) {
log.info(s);
}
shell.close();
}
}
Notice how I added arg1 and used #Value to map it with my.container.arg1. We can either use #ConfigurationProperties or #Value which are normal Spring and Spring Boot functionalities and there's more in Boot's reference docs how to use those.
You could then modify AppIT unit test:
ApplicationInfo info = submitApplicationAndWait(Application.class, new String[]{"--my.client.arg1=arg1value"});
and run build with tests
./gradlew clean build
or just build it without running test:
./gradlew clean build -x test
and then submit into a real hadoop cluster with your my.client.arg1.
java -jar build/libs/gs-yarn-basic-single-0.1.0.jar --my.client.arg1=arg1value
Either way you see arg1value logged in container logs:
[2014-07-18 08:49:09.802] boot - 2003 INFO [main] --- ContainerLauncherRunner: Running YarnContainer with parameters [--spring.profiles.active=container,--my.container.arg1=arg1value]
[2014-07-18 08:49:09.806] boot - 2003 INFO [main] --- Application$HelloPojo: Container arg1 value is arg1value
Using format ${my.client.arg1:notset1} also allows you to automatically define a default value notset1 if my.client.arg1 is omitted by user. We're working on Spring Application Context here orchestrated by Spring Boot so all the goodies from there are in your disposal
If you need more precise control of those user facing arguments(using args4j, jopt, etc) then you'd need to have a separate code/jar for client/appmaster/container order to create a custom client main method. All the other Spring YARN getting started guides are pretty much using multi-project builds so look at those. For example if you just want to have first and second argument value without having a need to use full --my.client.arg1=arg1value on a command line.
Let us know if this works for you and if you have any other ideas to make things simpler.
I am trying to write a program to connect to HBase. However when I execute following command
HBaseConfiguration.create(); I get following error: .
"hbase-default.xml file seems to be for and old version of HBase (null), this version is 0.92.1-cdh4.1.2.
When I dig deep and debug inside observe following:
class HBaseConfiguration
private static void checkDefaultsVersion(Configuration conf) {
if (conf.getBoolean("hbase.defaults.for.version.skip", Boolean.FALSE))return;
String defaultsVersion = conf.get("hbase.defaults.for.version");
String thisVersion = VersionInfo.getVersion();
if (!thisVersion.equals(defaultsVersion)) {
throw new RuntimeException(
"hbase-default.xml file seems to be for and old version of HBase (" +
defaultsVersion + "), this version is " + thisVersion);
}
}
In my case HBase returns default version as null, I am not sure why its returning as null as I checked the corresponding entry in hbase-default.xml packaged with the HBase.jar it has correct entry.
When I try the same thing from a standalone program it works as expected.
Guyz, Please let me know if you have any questions.
Thanks in advance,
Rohit
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property>
<name>hbase.defaults.for.version.skip</name>
<value>true</value>
</property>
</configuration>
Add this to a hbase-default.xml and put the file in the classpath or resource foldr. I got it when i ran from within spring hadoop environment. By adding above file to reosurce folder of the job jar i was able to solve tis-
finally found the workaround to this problem...
The problem is hbase-default.xml is not included in your classpath.
I added hbase-default.xml in target/test-classes ( it will vary in your case ), you can just add hbase-default.xml in various folder and see what works for you.
NOTE : This is just workaround, not the solution
Solution will be load the proper jars ( which I haven't figured out yet )
I've been getting this error using HBase1.1.1.
I created a simple HBase client and it worked fine. Then I built a simple RMI service, and that worked fine. But when I tried putting my simple HBase query code into RMI service I started getting this error on the HBaseConfiguration.create() call. After playing a bit, I found that the HBaseConfiguration.create() call works OK if placed before the security manager stuff that is in my main(). I get the error if the call is placed after block of code containing security manager calls...
Configuration conf = HBaseConfiguration.create(); // This works
if(System.getSecurityManager() == null)
{
System.setSecurityManager(new SecurityManager());
} // End if
// Configuration conf = HBaseConfiguration.create(); // This fails
I get the error if the create() call happens in main() after that security manager block, or in code within the class that is instantiated by main(). I don't get the error if create() is called within a static{ } block in my RMI service class (which I believe gets called before main()), or in main() before the security manager block, as shown.
BTW, the jar files that I include in my class path in order to get a minimal client to run are the following:
commons-codec-1.9.jar,
commons-collections-3.2.1.jar,
commons-configuration-1.6.jar,
commons-lang-2.6.jar,
commons-logging-1.2.jar,
guava-12.0.1.jar,
hadoop-auth-2.5.1.jar,
hadoop-common-2.5.1.jar,
hbase-client-1.1.1.jar,
hbase-common-1.1.1.jar,
hbase-hadoop2-compat-1.1.1.jar,
hbase-it-1.1.1-tests.jar,
hbase-protocol-1.1.1.jar,
htrace-core-3.1.0-incubating.jar,
log4j-1.2.17.jar,
netty-all-4.0.23.Final.jar,
protobuf-java-2.5.0.jar,
slf4j-api-1.7.7.jar,
slf4j-log4j12-1.7.5.jar
Had a similar problem where the error was
java.lang.RuntimeException: hbase-default.xml file seems to be for and old version of HBase (0.98.3-hadoop2), this version is Unknown
at org.apache.hadoop.hbase.HBaseConfiguration.checkDefaultsVersion(HBaseConfiguration.java:70)
at org.apache.hadoop.hbase.HBaseConfiguration.addHbaseResources(HBaseConfiguration.java:102)
at org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:113)
In my case I had same set of jar files at two different levels of classpath, removed from one level and it worked fine.
In my case the issue was caused by old java version (1.5), which was default on the server. But it works fine with 1.7.
In my code, I used this to solve my error.
val config = HBaseConfiguration.create() //error
val config = new Configuration() //works