How can I configurate the late acceptance algorithm? - algorithm

I am encountering some problems when it comes to modifying the applied algorithms:
First, it seems I cannot change the late acceptance algorithm configuration. This is my local search configuration, which works nicely when I am not specifying neither acceptor nor forager:
<localSearch>
<localSearchType>LATE_ACCEPTANCE</localSearchType>
<acceptor>
<lateAcceptanceSize>5000</lateAcceptanceSize>
<entityTabuRatio>0.1</entityTabuRatio>
<!--valueTabuSize>10000</valueTabuSize-->
</acceptor>
<forager>
<acceptedCountLimit>1</acceptedCountLimit>
</forager>
<termination>
<minutesSpentLimit>10</minutesSpentLimit>
<bestScoreLimit>[0/11496/11496]hard/[0]soft</bestScoreLimit>
</termination>
</localSearch>
I get this error message as part of the output
java.lang.IllegalArgumentException: The localSearchType (...
java.lang.IllegalArgumentException: The localSearchType (LATE_ACCEPTANCE) must not be configured if the acceptorConfig (AcceptorConfig()) is explicitly configured.
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildAcceptor(LocalSearchPhaseConfig.java:182)
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildDecider(LocalSearchPhaseConfig.java:132)
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildPhase(LocalSearchPhaseConfig.java:117)
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildPhase(LocalSearchPhaseConfig.java:54)
at org.optaplanner.core.config.solver.SolverConfig.buildPhaseList(SolverConfig.java:364)
at org.optaplanner.core.config.solver.SolverConfig.buildSolver(SolverConfig.java:270)
at org.optaplanner.core.impl.solver.AbstractSolverFactory.buildSolver(AbstractSolverFactory.java:61)
at com.gmv.g2gmps.CBand.App.main(App.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
at java.lang.Thread.run(Thread.java:748)

Finally, I was told that the configuration of LATE_ACCEPTANCE and lateAcceptanceSize at the same time is incompatible. Therefore I set late acceptance size as the only parameter (therefore Late acceptance as an acceptor) instead of writing it down as the local search type.
<localSearch>
<acceptor>
<lateAcceptanceSize>5000</lateAcceptanceSize>
<entityTabuRatio>0.1</entityTabuRatio>
<!--valueTabuSize>10000</valueTabuSize-->
</acceptor>
<forager>
<acceptedCountLimit>1</acceptedCountLimit>
</forager>
<termination>
<minutesSpentLimit>10</minutesSpentLimit>
<bestScoreLimit>[0/11496/11496]hard/[0]soft</bestScoreLimit>
</termination>
</localSearch>

Related

ConcurrentModificationException using Collectors.toSet()

I have a Set<Objects> that I want to filter by class to obtain a Set<Foo> (i.e., the subset of Objects that are instanceof Foo). To do this with Java-8 I wrote
Set<Foo> filtered = initialSet.parallelStream().filter(x -> (x instanceof Foo)).map(x -> (Foo) x).collect(Collectors.toSet());
This is throwing a ConcurrentModificationException:
java.util.ConcurrentModificationException
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1388)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
The problem is apparently the collector but I have no clue as to why.
Found (and fixed) the problem. The issue is the initialSet. This is being obtained by a call to a 3rd party library (in this case a JGraphT DirectedPseudoGraph instance). The underlying graph was being modified on another thread. Since the initialSet is returned by reference the result is the ConcurrentModificationException.
The real problem is therefore not the stream processing but using a graph that returned a Set that wasn't thread-safe. The solution was to use AsSynchronizedGraph.

Unable to load driverless ai models with pb and toml formats in the same JVM

I'm trying to use results of two models built in h2o driverless ai. One of them was built in 1.6.0 version and the other one was built in latest 1.7.0 version.
When I try to load the pipeline.mojo of these two models(One of them will be in pb format and the other one in toml format) in my java application, the first model file is read fine where as it throws IllegalArgumentException when reading the second model file in the same JVM instance.
The mojo2-runtime jar that is being used is from 1.7.0 version.
String toml_fileName = "/usr/toml/pipeline.mojo";
MojoPipeline toml_model = MojoPipeline.loadFrom(toml_fileName);
String pb_fileName = "/usr/pb/pipeline.mojo";
MojoPipeline pb_model = MojoPipeline.loadFrom(pb_fileName);
I get the following exception while trying to load the second model file which is pb_model in my case.
Can someone help me in finding out what's going wrong?
java.lang.IllegalArgumentException: wrong number of arguments
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at ai.h2o.mojos.runtime.b.aj.a(SourceFile:62)
at ai.h2o.mojos.runtime.b.J.a(SourceFile:35)
at ai.h2o.mojos.runtime.readers.MojoReader.read(SourceFile:133)
at ai.h2o.mojos.runtime.readers.MojoTransformerReader.readExecPipeTransformer(SourceFile:311)
at ai.h2o.mojos.runtime.readers.MojoTransformerReader.read(SourceFile:41)
at ai.h2o.mojos.runtime.readers.MojoReader.read(SourceFile:121)
at ai.h2o.mojos.runtime.MojoPipelineFactoryImpl.loadFrom(SourceFile:59)
at ai.h2o.mojos.runtime.MojoPipelineFactoryImpl.loadFrom(SourceFile:22)
at ai.h2o.mojos.runtime.MojoPipeline.loadFrom(SourceFile:41)

GemFire PDX Serialization not working with OQL in Pulse or with REST APIs?

We are using Spring as client to work with GemFire. Our data is getting saved to Regions of our running GemFire instances.
We are able to get data via Spring when we make get request.
Have done all configuration for Auto PDX serialization, However when we try to query data via OQL in Pulse or with GemFire REST Apis we get ClassNotFoundException:
{"message" : "Server has encountered while executing Adhoc query!", "stackTrace" : "com.gemstone.gemfire.rest.internal.web.exception.GemfireRestException: Server has encountered while executing Adhoc query!
at com.gemstone.gemfire.rest.internal.web.controllers.QueryAccessController.runAdhocQuery(QueryAccessController.java:200)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:919)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:851)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:844)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.gemstone.gemfire.SerializationException: A ClassNotFoundException was thrown while trying to deserialize cached value.
at com.gemstone.gemfire.internal.cache.EntryEventImpl.deserialize(EntryEventImpl.java:1561)
at com.gemstone.gemfire.internal.cache.EntryEventImpl.deserialize(EntryEventImpl.java:1546)
at com.gemstone.gemfire.internal.cache.VMCachedDeserializable.getDeserializedValue(VMCachedDeserializable.java:124)
at com.gemstone.gemfire.internal.cache.LocalRegion.getDeserialized(LocalRegion.java:1275)
at com.gemstone.gemfire.internal.cache.LocalRegion$NonTXEntry.getValue(LocalRegion.java:9140)
at com.gemstone.gemfire.internal.cache.EntriesSet$EntriesIterator.moveNext(EntriesSet.java:183)
at com.gemstone.gemfire.internal.cache.EntriesSet$EntriesIterator.<init>(EntriesSet.java:121)
at com.gemstone.gemfire.internal.cache.EntriesSet.iterator(EntriesSet.java:85)
at com.gemstone.gemfire.cache.query.internal.ResultsCollectionWrapper.iterator(ResultsCollectionWrapper.java:181)
at com.gemstone.gemfire.cache.query.internal.QRegion.iterator(QRegion.java:225)
at com.gemstone.gemfire.cache.query.internal.CompiledSelect.doNestedIterations(CompiledSelect.java:712)
at com.gemstone.gemfire.cache.query.internal.CompiledSelect.doIterationEvaluate(CompiledSelect.java:577)
at com.gemstone.gemfire.cache.query.internal.CompiledSelect.evaluate(CompiledSelect.java:284)
at com.gemstone.gemfire.cache.query.internal.DefaultQuery.executeUsingContext(DefaultQuery.java:529)
at com.gemstone.gemfire.cache.query.internal.DefaultQuery.execute(DefaultQuery.java:365)
at com.gemstone.gemfire.cache.query.internal.DefaultQuery.execute(DefaultQuery.java:303)
at com.gemstone.gemfire.rest.internal.web.controllers.QueryAccessController.runAdhocQuery(QueryAccessController.java:180)
... 40 more
Caused by: java.lang.ClassNotFoundException: in.cycorax.User
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
It looks like the query is trying to deserialize in.cycorax.User object, but that class is not present in the server's classpath.
What is your query? With PDX you should be able to query fields of your values without the objects needing to be deserialized on the server.
But if your query invokes a method on your object or tries to read a field that isn't part of the PDX serialized data (maybe even a misspelled field!), then gemfire will try to deserialize your object and see if it can invoke that method or find that field on the object.
So, you can either change your query so that it doesn't require deserialization, or you can add your classes to the server's classpath.
+1 to what Dan Smith said above. You must also be careful with "SELECT * FROM /Region" queries as well. This query will attempt to deserialize the entire object. As Dan mentioned, limit your queries to the fields of your object that will get serialized to PDX. If your object gets deserialized in anyway, then you must have your application domain object types (classes) on the GemFire server's classpath, no exceptions.

SQLException when accessing H2 metamodel through JOOQ-meta

I am trying to obtain a list of all tables in my H2 in-memory DB using JOOQ's DSLContext.meta():
DSL_CONTEXT_PROVIDER.db().meta().getTables();
results in:
java.lang.RuntimeException: org.jooq.exception.DataAccessException: Error while accessing DatabaseMetaData
at MyTest.deleteEntities(MyTest.java:222)
at MyTest.cleanupDatabase(MyTest.java:201)
at MyTest.afterTestCase(MyTest.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
[... omitted for brevity ...]
Caused by: org.jooq.exception.DataAccessException: Error while accessing DatabaseMetaData
at org.jooq.impl.MetaImpl.getCatalogs(MetaImpl.java:160)
at org.jooq.impl.MetaImpl.getSchemas(MetaImpl.java:168)
at org.jooq.impl.MetaImpl.getTables(MetaImpl.java:179)
at MyTest.deleteEntities(MyTest.java:210)
... 29 more
Caused by: org.h2.jdbc.JdbcSQLException: The object is already closed [90007-174]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:332)
at org.h2.message.DbException.get(DbException.java:172)
at org.h2.message.DbException.get(DbException.java:149)
at org.h2.message.DbException.get(DbException.java:138)
at org.h2.jdbc.JdbcConnection.checkClosed(JdbcConnection.java:1410)
at org.h2.jdbc.JdbcConnection.checkClosed(JdbcConnection.java:1388)
at org.h2.jdbc.JdbcDatabaseMetaData.checkClosed(JdbcDatabaseMetaData.java:2963)
at org.h2.jdbc.JdbcDatabaseMetaData.getCatalogs(JdbcDatabaseMetaData.java:756)
at org.jooq.impl.MetaImpl.getCatalogs(MetaImpl.java:143)
... 32 more
DSL_CONTEXT_PROVIDER.db() looks like this:
JdbcDataSource h2ds = new JdbcDataSource();
h2ds.setURL("jdbc:h2:mem:testDB;create=true");
h2ds.setUser("");
h2ds.setPassword("");
return DSL.using(new DefaultConfiguration().set(new DataSourceConnectionProvider(h2ds)));
Ordinary queries work fine with the above configuration, but not the meta().getTables(). If I replace DataSourceConnectionProvider with an anonymous implementation that doesn't close the connection, no exception is thrown anymore.
It seems H2 does not approve of calling methods like getCatalogs() on the object returned by connection.getMetaData() after the underlying connection has been closed. Is this this a bug in jooq-meta (I use 3.7.0) or is my configuration flawed?
jOOQ 3.7.0 / 3.7.1 and earlier are caching the DatabaseMetaData in org.jooq.Meta. This is a bug (4762) and will be fixed soon.
The reason why you're running into this issue is because you're using the DataSourceConnectionProvider, which isn't really intended to work with standalone connections, or "simple" DataSources. It closes the connection after every query (which normally translates to returning it to the pool). After closing the connection, the cached DatabaseMetaData reference is stale.
You've already documented the workaround: Don't use a "simple" DataSource with jOOQ's DSLContext.meta() API.

Unknown error in XPath

One of our web application, which has been running in our production environment for a long time, most recently it is running into an weird error when there is a high volume of transactions. We couldn't figure out what is exactly the root cause of the problem, but we found some similar issues in the previous version, WebSphere 6, related to a bug in Xalan version used by the app server. Our application server actually is WebSphere 7, which is supposed to have it fixed, besides it's not using Xalan under the hood anymore. Our application doesn't have Xalan jar embedded too.
To have it fixed we just restart the application itself.
One important note is that the Document is being cached (docs.get(tableName)) and reused to execute the XPath evaluation. We did it to avoid the cost of parsing the Document every time.
The app code is
Document doc = null;
try {
doc = docs.get(tableName);
if (doc == null)
return null;
XPathFactory xFactory = XPathFactory.newInstance();
XPath xpath = xFactory.newXPath();
XPathExpression expr = xpath.compile(toUpper(xPathQuery));
Object result = expr.evaluate(doc, XPathConstants.NODESET);
return (NodeList) result;
} catch (XPathExpressionException e) {
logger.error("Error executing XPath", e);
}
And the error stack is here
javax.xml.transform.TransformerException: Unknown error in XPath.
at java.lang.Throwable.<init>(Throwable.java:67)
at javax.xml.transform.TransformerException.<init>(Unknown Source)
at org.apache.xpath.XPath.execute(Unknown Source)
at org.apache.xpath.jaxp.XPathExpressionImpl.evaluate(Unknown Source)
Caused by: java.lang.NullPointerException
at org.apache.xerces.dom.ElementNSImpl.getPrefix(Unknown Source)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.processNamespacesAndAttributes(Unknown Source)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.nextNode(Unknown Source)
at org.apache.xml.dtm.ref.DTMDefaultBase._nextsib(Unknown Source)
at org.apache.xml.dtm.ref.DTMDefaultBase.getNextSibling(Unknown Source)
at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$ChildTraverser.next(Unknown Source)
at org.apache.xpath.axes.AxesWalker.getNextNode(Unknown Source)
at org.apache.xpath.axes.AxesWalker.nextNode(Unknown Source)
at org.apache.xpath.axes.WalkingIterator.nextNode(Unknown Source)
at org.apache.xpath.axes.NodeSequence.nextNode(Unknown Source)
at org.apache.xpath.axes.NodeSequence.runTo(Unknown Source)
at org.apache.xpath.axes.NodeSequence.setRoot(Unknown Source)
at org.apache.xpath.axes.LocPathIterator.execute(Unknown Source)
... 16 more
This is the similar issue what I mentioned.
http://www-01.ibm.com/support/docview.wss?uid=swg1PK42574
Thakns.
Many people don't realise that the DOM is not thread-safe (even if you are only doing reads). If you are caching a DOM object, make sure you synchronize all access to it.
Frankly, this makes DOM unsuited to this kind of application. I know I'm biased, but I would suggest switching to Saxon, whose native tree implementation is not only far faster than DOM, but also thread-safe. One user recently tweeted about getting a 100x performance improvement when they made this switch.

Resources