I try to use Hazelcast (3.9.2, 3.11 no difference) in the following way:
I got Hazelcast servers (members). I run them dedicated, not embedded.
I do not want to teach the Hazelcast members the classes I want to store within them. I used the bundled hazelcast.xml file and did the following addon (3.9.2)
<replicatedmap name="default">
<in-memory-format>BINARY</in-memory-format>
<statistics-enabled>true</statistics-enabled>
</replicatedmap>
I also activated TCP, not Multicast (true/false)
That is all changes I did. I started with one Member listening to 127.0.0.1:5701
Then I try to attach Hazelcast clients to the member for storing and retrieving Maps (Primarily ReplicatedMaps, but Maps also do not work in my scenario)
My Client Code looks like this (Cache is just a Serializable Class with no attributes):
public class Main {
public static final String HAZELCAST_INSTANCE_NAME = "HAZI";
public static final String REPLICATEDMAP_NAME = "REP_MAP";
public static final String MAP_NAME = "NORMAL_MAP";
public static void main(String[] args) {
init();
HazelcastInstance instance = HazelcastClient.getHazelcastClientByName(HAZELCAST_INSTANCE_NAME);
Map<String, Object> repMap = instance.getReplicatedMap(REPLICATEDMAP_NAME);
repMap.put("MyKey", new Cache());
System.err.println("Retrieve " + repMap.get("MyKey"));
Map<String, Object> normalMap = instance.getReplicatedMap(MAP_NAME);
normalMap.put("MyKey", new Cache());
System.err.println("Retrieve " + normalMap.get("MyKey"));
System.exit(1);
}
private static void init() {
ClientConfig cfg = new ClientConfig();
cfg.setInstanceName(HAZELCAST_INSTANCE_NAME);
cfg.addNearCacheConfig(defineNearCache(REPLICATEDMAP_NAME));
cfg.addNearCacheConfig(defineNearCache(MAP_NAME));
// for analysis in the hazelcast management console
cfg.getProperties().put("hazelcast.client.statistics.enabled", "true");
cfg.getProperties().put("hazelcast.client.statistics.period.seconds", "60");
cfg.getNetworkConfig().addAddress("127.0.0.1:5701");
if (HazelcastClient.newHazelcastClient(cfg) == null) {
System.err.println(" !!! ERROR in Cache Config !!!");
}
}
private static NearCacheConfig defineNearCache(String mapName) {
EvictionConfig evictionConfig = new EvictionConfig()
.setMaximumSizePolicy(EvictionConfig.MaxSizePolicy.ENTRY_COUNT)
.setSize(200);
return new NearCacheConfig()
.setName(mapName)
.setInMemoryFormat(InMemoryFormat.BINARY)
.setInvalidateOnChange(true)
.setEvictionConfig(evictionConfig);
}
}
My problem now is:
Using this code I get a ClassNotFoundError trying put put things to the replicated map or regular map, but in the dedicated Hazelcast server (member), not on the client side.
SCHWERWIEGEND: [127.0.0.1]:5701 [dev] [3.9.2] hz._hzInstance_1_dev.event-3 caught an exception while processing task:com.hazelcast.spi.impl.eventservice.impl.LocalEventDispatcher#eeed098
com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: de.empic.hazelwar.model.Cache
at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:224)
at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:185)
at com.hazelcast.map.impl.DataAwareEntryEvent.getValue(DataAwareEntryEvent.java:90)
at com.hazelcast.client.impl.protocol.task.replicatedmap.AbstractReplicatedMapAddEntryListenerMessageTask.handleEvent(AbstractReplicatedMapAddEntryListenerMessageTask.java:92)
at com.hazelcast.client.impl.protocol.task.replicatedmap.AbstractReplicatedMapAddEntryListenerMessageTask.entryAdded(AbstractReplicatedMapAddEntryListenerMessageTask.java:132)
at com.hazelcast.replicatedmap.impl.ReplicatedMapEventPublishingService.dispatchEvent(ReplicatedMapEventPublishingService.java:82)
at com.hazelcast.replicatedmap.impl.ReplicatedMapService.dispatchEvent(ReplicatedMapService.java:247)
at com.hazelcast.spi.impl.eventservice.impl.LocalEventDispatcher.run(LocalEventDispatcher.java:64)
at com.hazelcast.util.executor.StripedExecutor$Worker.process(StripedExecutor.java:225)
at com.hazelcast.util.executor.StripedExecutor$Worker.run(StripedExecutor.java:208)
Caused by: java.lang.ClassNotFoundException: de.empic.hazelwar.model.Cache
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:173)
at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:147)
at com.hazelcast.nio.IOUtil$ClassLoaderAwareObjectInputStream.resolveClass(IOUtil.java:591)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:219)
... 10 more
Whenever I remove the near cache config from the client config, all works pretty perfect, except I do not have a near cache of course.
What do I miss here ?
#magicroomy, I run the same on both 3.9.2 & 3.11. I can confirm that:
If you change Replicated Map to Map, it works with or without Near Cache.
When using Replicated Map, if Near Cache defined, the exception thrown on the server side.
Without Near Cache, ReplicatedMap also works.
I created a github issue as well: https://github.com/hazelcast/hazelcast/issues/14210
My problem is solved using the 3.11.1 version of hazelcast.
Related
I have a Java JDBC application, after 100 queries of Select Jetty server crashed and return below error:
ERROR com.zaxxer.hikari.pool.HikariPool - dev-server - Exception during pool initialization.
java.sql.SQLNonTransientConnectionException: Too many connections
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:526)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:513)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:115)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:1606)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:633)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:347)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:219)
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:358)
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201)
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:443)
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:514)
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:111)
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:72)
at com.jrg.platform.commons.hikari.HikariDataSourceSupport.getDataSource(HikariDataSourceSupport.java:70)
at com.jrg.platform.commons.hikari.HikariDataSourceSupport.getConnection(HikariDataSourceSupport.java:82)
at com.jrg.platform.services.internal.pcms.data.campaign.CampaignAlertDaoImpl.getCampaignAlerts(CampaignAlertDaoImpl.java:40)
at com.jrg.platform.services.internal.pcms.service.CampaignAlertServiceImpl.campaignAlerts(CampaignAlertServiceImpl.java:43)
at com.jrg.platform.services.internal.pcms.resource.CampaignAlertResource.getCampaignAlerts(CampaignAlertResource.java:52)
at com.jrg.platform.services.internal.pcms.resource.CampaignAlertResource_$$_jvstf5a_6._d5getCampaignAlerts(CampaignAlertResource_$$_jvstf5a_6.java)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Unable to find the solution. I am just getting the connection from HikariDataSourceSupport and performing the query.
the sample code of getting connection is given below:
default HikariDataSource getDataSource(E e, boolean readOnly) throws IOException {
String key = this.getKey(e, readOnly);
Map<String, HikariDataSource> sources = this.getDataSources();
if(!sources.containsKey(key)) {
synchronized(this.getMonitor()) {
if(!sources.containsKey(key)) {
if(logger.isDebugEnabled()) {
logger.debug("creating new DataSource for {}", key);
}
Config config = this.getConfig(e, readOnly);
if(!config.hasPathOrNull("jdbcUrl")) {
throw new EnvironmentNotConfigured(e, readOnly);
}
sources.put(key, new HikariDataSource(new HikariConfig(ConfigUtils.asProperties(config))));
}
}
}
return (HikariDataSource)sources.get(key);
}
default Connection getConnection(E env, boolean readOnly) throws SQLException, IOException {
return this.getDataSource(env, readOnly).getConnection();
}
There was problem in configuration HikariCP. When request generated and try to get connection, the create connection method was creating the new datasource because the last state was destroy due to view scope of the bean.
now i have converted that class into to singalton and created the implementation of that interface to inject in the code. it is saving the state of previous data source connection. Now it is working perfect with minimum idle connections.
the code can be seen here:
APP.java
bind(HikariLucktasticDataSourceSupportImpl.class)
.to(HikariDataSourceSupport.class)
.named("hdsSupport")
.in(Singleton.class);
in DAO Layer:
#Inject
#Named("hdsSupport")
private HikariDataSourceSupport hdsSupport;
I have a CXF client configured in my Spring Boot app like so:
#Bean
public ConsumerSupportService consumerSupportService() {
JaxWsProxyFactoryBean jaxWsProxyFactoryBean = new JaxWsProxyFactoryBean();
jaxWsProxyFactoryBean.setServiceClass(ConsumerSupportService.class);
jaxWsProxyFactoryBean.setAddress("https://www.someservice.com/service?wsdl");
jaxWsProxyFactoryBean.setBindingId(SOAPBinding.SOAP12HTTP_BINDING);
WSAddressingFeature wsAddressingFeature = new WSAddressingFeature();
wsAddressingFeature.setAddressingRequired(true);
jaxWsProxyFactoryBean.getFeatures().add(wsAddressingFeature);
ConsumerSupportService service = (ConsumerSupportService) jaxWsProxyFactoryBean.create();
Client client = ClientProxy.getClient(service);
AddressingProperties addressingProperties = new AddressingProperties();
AttributedURIType to = new AttributedURIType();
to.setValue(applicationProperties.getWex().getServices().getConsumersupport().getTo());
addressingProperties.setTo(to);
AttributedURIType action = new AttributedURIType();
action.setValue("http://serviceaction/SearchConsumer");
addressingProperties.setAction(action);
client.getRequestContext().put("javax.xml.ws.addressing.context", addressingProperties);
setClientTimeout(client);
return service;
}
private void setClientTimeout(Client client) {
HTTPConduit conduit = (HTTPConduit) client.getConduit();
HTTPClientPolicy policy = new HTTPClientPolicy();
policy.setConnectionTimeout(applicationProperties.getWex().getServices().getClient().getConnectionTimeout());
policy.setReceiveTimeout(applicationProperties.getWex().getServices().getClient().getReceiveTimeout());
conduit.setClient(policy);
}
This same service bean is accessed by two different threads in the same application sequence. If I execute this particular sequence 10 times in a row, I will get a connection timeout from the service call at least 3 times. What I'm seeing is:
Caused by: java.io.IOException: Timed out waiting for response to operation {http://theservice.com}SearchConsumer.
at org.apache.cxf.endpoint.ClientImpl.waitResponse(ClientImpl.java:685) ~[cxf-core-3.2.0.jar:3.2.0]
at org.apache.cxf.endpoint.ClientImpl.processResult(ClientImpl.java:608) ~[cxf-core-3.2.0.jar:3.2.0]
If I change the sequence such that one of the threads does not call this service, then the error goes away. So, it seems like there's some sort of a race condition happening here. If I look at the logs in our proxy manager for this service, I can see that both of the service calls do return a response very quickly, but the second service call seems to get stuck somewhere in the code and never actually lets go of the connection until the timeout value is reached. I've been trying to track down the cause of this for quite a while, but have been unsuccessful.
I've read some mixed opinions as to whether or not CXF client proxies are thread-safe, but I was under the impression that they were. If this actually not the case, and I should be creating a new client proxy for each invocation, or use a pool of proxies?
Turns out that it is an issue with the proxy not being thread-safe. What I wound up doing was leveraging a solution kind of like one posted at the bottom of this post: Is this JAX-WS client call thread safe? - I created a pool for the proxies and I use that to access proxies from multiple threads in a thread-safe manner. This seems to work out pretty well.
public class JaxWSServiceProxyPool<T> extends GenericObjectPool<T> {
JaxWSServiceProxyPool(Supplier<T> factory, GenericObjectPoolConfig poolConfig) {
super(new BasePooledObjectFactory<T>() {
#Override
public T create() throws Exception {
return factory.get();
}
#Override
public PooledObject<T> wrap(T t) {
return new DefaultPooledObject<>(t);
}
}, poolConfig != null ? poolConfig : new GenericObjectPoolConfig());
}
}
I then created a simple "registry" class to keep references to various pools.
#Component
public class JaxWSServiceProxyPoolRegistry {
private static final Map<Class, JaxWSServiceProxyPool> registry = new HashMap<>();
public synchronized <T> void register(Class<T> serviceTypeClass, Supplier<T> factory, GenericObjectPoolConfig poolConfig) {
Assert.notNull(serviceTypeClass);
Assert.notNull(factory);
if (!registry.containsKey(serviceTypeClass)) {
registry.put(serviceTypeClass, new JaxWSServiceProxyPool<>(factory, poolConfig));
}
}
public <T> void register(Class<T> serviceTypeClass, Supplier<T> factory) {
register(serviceTypeClass, factory, null);
}
#SuppressWarnings("unchecked")
public <T> JaxWSServiceProxyPool<T> getServiceProxyPool(Class<T> serviceTypeClass) {
Assert.notNull(serviceTypeClass);
return registry.get(serviceTypeClass);
}
}
To use it, I did:
JaxWSServiceProxyPoolRegistry jaxWSServiceProxyPoolRegistry = new JaxWSServiceProxyPoolRegistry();
jaxWSServiceProxyPoolRegistry.register(ConsumerSupportService.class,
this::buildConsumerSupportServiceClient,
getConsumerSupportServicePoolConfig());
Where buildConsumerSupportServiceClient uses a JaxWsProxyFactoryBean to build up the client.
To retrieve an instance from the pool I inject my registry class and then do:
JaxWSServiceProxyPool<ConsumerSupportService> consumerSupportServiceJaxWSServiceProxyPool = jaxWSServiceProxyPoolRegistry.getServiceProxyPool(ConsumerSupportService.class);
And then borrow/return the object from/to the pool as necessary.
This seems to work well so far. I've executed some fairly heavy load tests against it and it's held up.
I tried to use org.springframework.boot:spring-boot-devtools to speed up development.
My project uses Firebase to authenticate some requests. Firebase initialized via:
#PostConstruct
public void instantiateFirebase() throws IOException {
FirebaseOptions options = new FirebaseOptions.Builder()
.setDatabaseUrl(String.format("https://%s.firebaseio.com", configuration.getFirebaseDatabase()))
.setServiceAccount(serviceJson.getInputStream())
.build();
FirebaseApp.initializeApp(options);
}
After context reloading on changing .class file Spring reports error:
Caused by: java.lang.IllegalStateException: FirebaseApp name [DEFAULT] already exists!
at com.google.firebase.internal.Preconditions.checkState(Preconditions.java:173)
at com.google.firebase.FirebaseApp.initializeApp(FirebaseApp.java:180)
at com.google.firebase.FirebaseApp.initializeApp(FirebaseApp.java:160)
What Firebase API allow deregister/destroy FirebaseApp that I should use in #PreDestroy?
Looks like it is not possible to disable/shutdown/reinitialize Firebase app.
In my case it is fine to keep that instance in memory without changes.
Depending on your requirements you may use as simple as:
#PostConstruct
public void instantiateFirebase() throws IOException {
// We use only FirebaseApp.DEFAULT_APP_NAME, so check is simple.
if ( ! FirebaseApp.getApps().isEmpty())
return;
Resource serviceJson = applicationContext.getResource(String.format("classpath:firebase/%s", configuration.getFirebaseServiceJson()));
FirebaseOptions options = new FirebaseOptions.Builder()
.setDatabaseUrl(String.format("https://%s.firebaseio.com", configuration.getFirebaseDatabase()))
.setServiceAccount(serviceJson.getInputStream())
.build();
FirebaseApp.initializeApp(options);
}
or filter data like:
for (FirebaseApp app : FirebaseApp.getApps()) {
if (app.getName().equals(FirebaseApp.DEFAULT_APP_NAME))
return;
}
I started Hbase few days back and going through all the material of online.
I have installed and configured HBase and shell commands are working fine.
I got an example of Java client to get data from HBase Table and it executed successfully but I could not understand how it is working? In the code nowhere we have mentioned the port, host of Hbase server? How it able to fetch the data from table?
This is my code:
public class RetriveData {
public static void main(String[] args) throws IOException {
// Instantiating Configuration class
Configuration config = HBaseConfiguration.create();
// Instantiating HTable class
#SuppressWarnings({ "deprecation", "resource" })
HTable table = new HTable(config, "emp");
// Instantiating Get class
Get g = new Get(Bytes.toBytes("1"));
// Reading the data
Result result = table.get(g);
// Reading values from Result class object
byte [] value = result.getValue(Bytes.toBytes("personal data"),Bytes.toBytes("name"));
byte [] value1 = result.getValue(Bytes.toBytes("personal data"),Bytes.toBytes("city"));
// Printing the values
String name = Bytes.toString(value);
String city = Bytes.toString(value1);
System.out.println("name: " + name + " city: " + city);
}
}
The output looks like:
Output:
name: raju city: hyderabad
I agree with Binary Nerds answer
adding some more interesting information for better understanding.
Your Question :
I could not understand how it is working? In the code nowhere we have
mentioned the port, host of Hbase server? How it able to fetch the
data from table?
Since you are executing this program in cluster
// Instantiating Configuration class
Configuration config = HBaseConfiguration.create()
all the cluster properties will be taken care from inside the cluster.. since you are in cluster and you are executing hbase java client program..
Now try like below (execute same program in different way from remote machine eclipse on windows to find out difference of what you have done earlier and now).
public static Configuration configuration; // this is class variable
static { //fill clusternode1,clusternode2,clusternode3 from your cluster
configuration = HBaseConfiguration.create();
configuration.set("hbase.zookeeper.property.clientPort", "2181");
configuration.set("hbase.zookeeper.quorum",
"clusternode1,clusternode2,clusternode3");
configuration.set("hbase.master", "clusternode1:600000");
}
Hope this heps you to understand.
If you look at the source code for HBaseConfiguration on github you can see what it does when it calls create().
public static Configuration create() {
Configuration conf = new Configuration();
// In case HBaseConfiguration is loaded from a different classloader than
// Configuration, conf needs to be set with appropriate class loader to resolve
// HBase resources.
conf.setClassLoader(HBaseConfiguration.class.getClassLoader());
return addHbaseResources(conf);
}
Followed by:
public static Configuration addHbaseResources(Configuration conf) {
conf.addResource("hbase-default.xml");
conf.addResource("hbase-site.xml");
checkDefaultsVersion(conf);
HeapMemorySizeUtil.checkForClusterFreeMemoryLimit(conf);
return conf;
}
So its loading the configuration from your HBase configuration files hbase-default.xml and hbase-site.xml.
I followed the gwt 2.4 validation sample and implemented the whole stuff into my own App. The client side works great.
private void verifyRegistrationData(final RegistrationTO registration) throws ConstraintViolationException {
final Validator validator = Validation.buildDefaultValidatorFactory().getValidator();
final Set<ConstraintViolation<RegistrationTO>> violations = validator.validate(registration);
if (violations.size() > 0) {
final Set<ConstraintViolation<?>> temp = new HashSet<ConstraintViolation<?>>(violations);
throw new ConstraintViolationException(temp);
...
but if I do the same on the server side:
public void update(final RegistrationTO registration) throws IllegalArgumentException, ConstraintViolationException, TestException {
final Set<ConstraintViolation<RegistrationTO>> violations = validator.validate(registration);
if (!violations.isEmpty()) {
final Set<ConstraintViolation<?>> temp = new HashSet<ConstraintViolation<?>>(violations);
throw new ConstraintViolationException(temp);
}
...
the whole thing crashes with the following exception:
javax.servlet.ServletContext log: Exception while dispatching incoming RPC call
com.google.gwt.user.client.rpc.SerializationException: Type 'org.hibernate.validator.engine.PathImpl' was not included in the set of types which can be serialized by this SerializationPolicy or its Class object could not be loaded. For security purposes, this type will not be serialized.
That's how PathImpl looks like hibernate-validator-4.1.0.Final-sources.jar
public class PathImpl implements Path, Serializable {
private static final long serialVersionUID = 7564511574909882392L;
...
looks OK (at least to me)
I am using GWT 2.4, validation-api-1.0.0.GA, hibernate-validator-4.1.0.Final, gwt-servlet-deps ...
Thanks in advance!
Is there an explicitly defined a default constructor? i.e.,
public PathImpl() { } ? This is required by GWT's serialization mechanism; if it isn't in the source, serializing an RPC response will fail.
A custom serializer does exist for PathImpl, it's just that unless that class is explicitly referenced in your service API, it's not going to be added to the serialization policy.
The current work around is to add a dummy PathImpl field somewhere in your service API. The ValidationSupport class exists to group this and other such classes together to make this a bit easier.
I change the whole thing to RequestFactory as Thomas Broyer recommended. It was by far not so easy as GWT-RPC. This was the reason for me to collect all kind of informations and to build a sample program. For those who are interested - here you can find a sample with documentation and source. (Single line client logger is also implemented) (Documentation is in German but logging-output aso. is in English...)