I tried to run COPAC ELKI implementation on the example dataset provided on the official site (mouse.csv) but I get a NullPointerException which leads me to think that there is some detail that I omit (shame on me).
The exception is the following:
Task failed java.lang.NullPointerException at
de.lmu.ifi.dbs.elki.database.ids.integer.DoubleDistanceIntegerDBIDPair.compareByDistance(Unknown
Source) at
de.lmu.ifi.dbs.elki.distance.distanceresultlist.AbstractKNNHeap$Comp.compare(Unknown
Source) at
de.lmu.ifi.dbs.elki.distance.distanceresultlist.AbstractKNNHeap$Comp.compare(Unknown
Source) at
de.lmu.ifi.dbs.elki.utilities.datastructures.heap.TopBoundedHeap.add(Unknown
Source) at
de.lmu.ifi.dbs.elki.database.query.knn.LinearScanRawDoubleDistanceKNNQuery.getKNNForObjectBenchmarked(Unknown
Source) at
de.lmu.ifi.dbs.elki.database.query.knn.LinearScanRawDoubleDistanceKNNQuery.getKNNForObject(Unknown
Source) at
de.lmu.ifi.dbs.elki.database.query.knn.LinearScanRawDoubleDistanceKNNQuery.getKNNForDBID(Unknown
Source) at
de.lmu.ifi.dbs.elki.index.preprocessed.localpca.KNNQueryFilteredPCAIndex.objectsForPCA(Unknown
Source) at
de.lmu.ifi.dbs.elki.index.preprocessed.localpca.KNNQueryFilteredPCAIndex.objectsForPCA(Unknown
Source) at
de.lmu.ifi.dbs.elki.index.preprocessed.localpca.AbstractFilteredPCAIndex.preprocess(Unknown
Source) at
de.lmu.ifi.dbs.elki.index.preprocessed.localpca.AbstractFilteredPCAIndex.getLocalProjection(Unknown
Source) at
de.lmu.ifi.dbs.elki.index.preprocessed.localpca.AbstractFilteredPCAIndex.getLocalProjection(Unknown
Source) at
de.lmu.ifi.dbs.elki.algorithm.clustering.correlation.COPAC.run(Unknown
Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616) at
de.lmu.ifi.dbs.elki.algorithm.AbstractAlgorithm.run(Unknown Source)
at de.lmu.ifi.dbs.elki.workflow.AlgorithmStep.runAlgorithms(Unknown
Source) at [...]
This is the setting of the parameters that I use:
-dbc.in
[PATH OMITTED]/mouse.csv
-algorithm
clustering.correlation.COPAC
-copac.preprocessor
localpca.KNNQueryFilteredPCAIndex
-copac.partitionAlgorithm
gdbscan.GeneralizedDBSCAN
-dbscan.epsilon
0.00400000
-dbscan.minpts
3
Can anyone give me some info?
You must specify the k parameter for kNN Local PCA:
-localpca.k 10
there is a slight bug in the last ELKI release, where the parameter is incorrectly marked as optional. It is not optional. Note that k should be significantly larger than the data dimensionality to give stable PCA results.
Note that using COPAC on a low dimensional data set such as the mouse example does not make sense.
Related
I have the following situation with jmeter:
I have two thread groups and I want to use a variable extracted from a response from the first one into the second thread.
What I am doing:
the variable is extracted with JSON/YAML Path Extractor and then it is set as a property with BeanShell Assertion ${__setProperty(id, ${id})};
Then in the second thread group I have a BeanShell PreProcessor where I am trying to modify the value with the following script
String ids2 = props.get("id");
String ids3 = vars.put(${__intSum(2,-4)});
String ids = vars.put(ids,"${__intSum(${ids2},${ids3})}");
As result I am getting an exception in jmeter
ava.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(Unknown Source) ~[?:1.8.0_221]
at java.lang.Integer.parseInt(Unknown Source) ~[?:1.8.0_221]
at java.lang.Integer.parseInt(Unknown Source) ~[?:1.8.0_221]
at org.apache.jmeter.functions.IntSum.execute(IntSum.java:66) ~[ApacheJMeter_functions.jar:5.4.1]
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:138) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.util.CompoundVariable.execute(CompoundVariable.java:113) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.testelement.property.FunctionProperty.getStringValue(FunctionProperty.java:91) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.testbeans.TestBeanHelper.unwrapProperty(TestBeanHelper.java:129) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.testbeans.TestBeanHelper.prepare(TestBeanHelper.java:84) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:202) ~[ApacheJMeter_core.jar:5.4.1]
at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:382) ~[ApacheJMeter_core.jar:5.4.1]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_221]
The goal is to reduce the variable with 2 and then to transfer it to another variable which I will use only in the second thread group
Any help will be highly appreciated
Since JMeter 3.1 it's recommended to use JSR223 Test Elements and Groovy language for scripting
Your approach might work for 1 thread but if you will have > 1 thread the property will be overwritten, if you want each thread (virtual user) to have its own value you should add the current thread number as the prefix or postfix for the property. The relevant function is __threadNum() or if you prefer scripting you can call ctx.getThreadNum() function
In the vast majority of cases Inter-Thread Communication Plugin is easier to use
I am encountering some problems when it comes to modifying the applied algorithms:
First, it seems I cannot change the late acceptance algorithm configuration. This is my local search configuration, which works nicely when I am not specifying neither acceptor nor forager:
<localSearch>
<localSearchType>LATE_ACCEPTANCE</localSearchType>
<acceptor>
<lateAcceptanceSize>5000</lateAcceptanceSize>
<entityTabuRatio>0.1</entityTabuRatio>
<!--valueTabuSize>10000</valueTabuSize-->
</acceptor>
<forager>
<acceptedCountLimit>1</acceptedCountLimit>
</forager>
<termination>
<minutesSpentLimit>10</minutesSpentLimit>
<bestScoreLimit>[0/11496/11496]hard/[0]soft</bestScoreLimit>
</termination>
</localSearch>
I get this error message as part of the output
java.lang.IllegalArgumentException: The localSearchType (...
java.lang.IllegalArgumentException: The localSearchType (LATE_ACCEPTANCE) must not be configured if the acceptorConfig (AcceptorConfig()) is explicitly configured.
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildAcceptor(LocalSearchPhaseConfig.java:182)
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildDecider(LocalSearchPhaseConfig.java:132)
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildPhase(LocalSearchPhaseConfig.java:117)
at org.optaplanner.core.config.localsearch.LocalSearchPhaseConfig.buildPhase(LocalSearchPhaseConfig.java:54)
at org.optaplanner.core.config.solver.SolverConfig.buildPhaseList(SolverConfig.java:364)
at org.optaplanner.core.config.solver.SolverConfig.buildSolver(SolverConfig.java:270)
at org.optaplanner.core.impl.solver.AbstractSolverFactory.buildSolver(AbstractSolverFactory.java:61)
at com.gmv.g2gmps.CBand.App.main(App.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:282)
at java.lang.Thread.run(Thread.java:748)
Finally, I was told that the configuration of LATE_ACCEPTANCE and lateAcceptanceSize at the same time is incompatible. Therefore I set late acceptance size as the only parameter (therefore Late acceptance as an acceptor) instead of writing it down as the local search type.
<localSearch>
<acceptor>
<lateAcceptanceSize>5000</lateAcceptanceSize>
<entityTabuRatio>0.1</entityTabuRatio>
<!--valueTabuSize>10000</valueTabuSize-->
</acceptor>
<forager>
<acceptedCountLimit>1</acceptedCountLimit>
</forager>
<termination>
<minutesSpentLimit>10</minutesSpentLimit>
<bestScoreLimit>[0/11496/11496]hard/[0]soft</bestScoreLimit>
</termination>
</localSearch>
We are using Spring as client to work with GemFire. Our data is getting saved to Regions of our running GemFire instances.
We are able to get data via Spring when we make get request.
Have done all configuration for Auto PDX serialization, However when we try to query data via OQL in Pulse or with GemFire REST Apis we get ClassNotFoundException:
{"message" : "Server has encountered while executing Adhoc query!", "stackTrace" : "com.gemstone.gemfire.rest.internal.web.exception.GemfireRestException: Server has encountered while executing Adhoc query!
at com.gemstone.gemfire.rest.internal.web.controllers.QueryAccessController.runAdhocQuery(QueryAccessController.java:200)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:919)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:851)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:844)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.gemstone.gemfire.SerializationException: A ClassNotFoundException was thrown while trying to deserialize cached value.
at com.gemstone.gemfire.internal.cache.EntryEventImpl.deserialize(EntryEventImpl.java:1561)
at com.gemstone.gemfire.internal.cache.EntryEventImpl.deserialize(EntryEventImpl.java:1546)
at com.gemstone.gemfire.internal.cache.VMCachedDeserializable.getDeserializedValue(VMCachedDeserializable.java:124)
at com.gemstone.gemfire.internal.cache.LocalRegion.getDeserialized(LocalRegion.java:1275)
at com.gemstone.gemfire.internal.cache.LocalRegion$NonTXEntry.getValue(LocalRegion.java:9140)
at com.gemstone.gemfire.internal.cache.EntriesSet$EntriesIterator.moveNext(EntriesSet.java:183)
at com.gemstone.gemfire.internal.cache.EntriesSet$EntriesIterator.<init>(EntriesSet.java:121)
at com.gemstone.gemfire.internal.cache.EntriesSet.iterator(EntriesSet.java:85)
at com.gemstone.gemfire.cache.query.internal.ResultsCollectionWrapper.iterator(ResultsCollectionWrapper.java:181)
at com.gemstone.gemfire.cache.query.internal.QRegion.iterator(QRegion.java:225)
at com.gemstone.gemfire.cache.query.internal.CompiledSelect.doNestedIterations(CompiledSelect.java:712)
at com.gemstone.gemfire.cache.query.internal.CompiledSelect.doIterationEvaluate(CompiledSelect.java:577)
at com.gemstone.gemfire.cache.query.internal.CompiledSelect.evaluate(CompiledSelect.java:284)
at com.gemstone.gemfire.cache.query.internal.DefaultQuery.executeUsingContext(DefaultQuery.java:529)
at com.gemstone.gemfire.cache.query.internal.DefaultQuery.execute(DefaultQuery.java:365)
at com.gemstone.gemfire.cache.query.internal.DefaultQuery.execute(DefaultQuery.java:303)
at com.gemstone.gemfire.rest.internal.web.controllers.QueryAccessController.runAdhocQuery(QueryAccessController.java:180)
... 40 more
Caused by: java.lang.ClassNotFoundException: in.cycorax.User
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
It looks like the query is trying to deserialize in.cycorax.User object, but that class is not present in the server's classpath.
What is your query? With PDX you should be able to query fields of your values without the objects needing to be deserialized on the server.
But if your query invokes a method on your object or tries to read a field that isn't part of the PDX serialized data (maybe even a misspelled field!), then gemfire will try to deserialize your object and see if it can invoke that method or find that field on the object.
So, you can either change your query so that it doesn't require deserialization, or you can add your classes to the server's classpath.
+1 to what Dan Smith said above. You must also be careful with "SELECT * FROM /Region" queries as well. This query will attempt to deserialize the entire object. As Dan mentioned, limit your queries to the fields of your object that will get serialized to PDX. If your object gets deserialized in anyway, then you must have your application domain object types (classes) on the GemFire server's classpath, no exceptions.
I am trying to obtain a list of all tables in my H2 in-memory DB using JOOQ's DSLContext.meta():
DSL_CONTEXT_PROVIDER.db().meta().getTables();
results in:
java.lang.RuntimeException: org.jooq.exception.DataAccessException: Error while accessing DatabaseMetaData
at MyTest.deleteEntities(MyTest.java:222)
at MyTest.cleanupDatabase(MyTest.java:201)
at MyTest.afterTestCase(MyTest.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
[... omitted for brevity ...]
Caused by: org.jooq.exception.DataAccessException: Error while accessing DatabaseMetaData
at org.jooq.impl.MetaImpl.getCatalogs(MetaImpl.java:160)
at org.jooq.impl.MetaImpl.getSchemas(MetaImpl.java:168)
at org.jooq.impl.MetaImpl.getTables(MetaImpl.java:179)
at MyTest.deleteEntities(MyTest.java:210)
... 29 more
Caused by: org.h2.jdbc.JdbcSQLException: The object is already closed [90007-174]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:332)
at org.h2.message.DbException.get(DbException.java:172)
at org.h2.message.DbException.get(DbException.java:149)
at org.h2.message.DbException.get(DbException.java:138)
at org.h2.jdbc.JdbcConnection.checkClosed(JdbcConnection.java:1410)
at org.h2.jdbc.JdbcConnection.checkClosed(JdbcConnection.java:1388)
at org.h2.jdbc.JdbcDatabaseMetaData.checkClosed(JdbcDatabaseMetaData.java:2963)
at org.h2.jdbc.JdbcDatabaseMetaData.getCatalogs(JdbcDatabaseMetaData.java:756)
at org.jooq.impl.MetaImpl.getCatalogs(MetaImpl.java:143)
... 32 more
DSL_CONTEXT_PROVIDER.db() looks like this:
JdbcDataSource h2ds = new JdbcDataSource();
h2ds.setURL("jdbc:h2:mem:testDB;create=true");
h2ds.setUser("");
h2ds.setPassword("");
return DSL.using(new DefaultConfiguration().set(new DataSourceConnectionProvider(h2ds)));
Ordinary queries work fine with the above configuration, but not the meta().getTables(). If I replace DataSourceConnectionProvider with an anonymous implementation that doesn't close the connection, no exception is thrown anymore.
It seems H2 does not approve of calling methods like getCatalogs() on the object returned by connection.getMetaData() after the underlying connection has been closed. Is this this a bug in jooq-meta (I use 3.7.0) or is my configuration flawed?
jOOQ 3.7.0 / 3.7.1 and earlier are caching the DatabaseMetaData in org.jooq.Meta. This is a bug (4762) and will be fixed soon.
The reason why you're running into this issue is because you're using the DataSourceConnectionProvider, which isn't really intended to work with standalone connections, or "simple" DataSources. It closes the connection after every query (which normally translates to returning it to the pool). After closing the connection, the cached DatabaseMetaData reference is stale.
You've already documented the workaround: Don't use a "simple" DataSource with jOOQ's DSLContext.meta() API.
One of our web application, which has been running in our production environment for a long time, most recently it is running into an weird error when there is a high volume of transactions. We couldn't figure out what is exactly the root cause of the problem, but we found some similar issues in the previous version, WebSphere 6, related to a bug in Xalan version used by the app server. Our application server actually is WebSphere 7, which is supposed to have it fixed, besides it's not using Xalan under the hood anymore. Our application doesn't have Xalan jar embedded too.
To have it fixed we just restart the application itself.
One important note is that the Document is being cached (docs.get(tableName)) and reused to execute the XPath evaluation. We did it to avoid the cost of parsing the Document every time.
The app code is
Document doc = null;
try {
doc = docs.get(tableName);
if (doc == null)
return null;
XPathFactory xFactory = XPathFactory.newInstance();
XPath xpath = xFactory.newXPath();
XPathExpression expr = xpath.compile(toUpper(xPathQuery));
Object result = expr.evaluate(doc, XPathConstants.NODESET);
return (NodeList) result;
} catch (XPathExpressionException e) {
logger.error("Error executing XPath", e);
}
And the error stack is here
javax.xml.transform.TransformerException: Unknown error in XPath.
at java.lang.Throwable.<init>(Throwable.java:67)
at javax.xml.transform.TransformerException.<init>(Unknown Source)
at org.apache.xpath.XPath.execute(Unknown Source)
at org.apache.xpath.jaxp.XPathExpressionImpl.evaluate(Unknown Source)
Caused by: java.lang.NullPointerException
at org.apache.xerces.dom.ElementNSImpl.getPrefix(Unknown Source)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.processNamespacesAndAttributes(Unknown Source)
at org.apache.xml.dtm.ref.dom2dtm.DOM2DTM.nextNode(Unknown Source)
at org.apache.xml.dtm.ref.DTMDefaultBase._nextsib(Unknown Source)
at org.apache.xml.dtm.ref.DTMDefaultBase.getNextSibling(Unknown Source)
at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$ChildTraverser.next(Unknown Source)
at org.apache.xpath.axes.AxesWalker.getNextNode(Unknown Source)
at org.apache.xpath.axes.AxesWalker.nextNode(Unknown Source)
at org.apache.xpath.axes.WalkingIterator.nextNode(Unknown Source)
at org.apache.xpath.axes.NodeSequence.nextNode(Unknown Source)
at org.apache.xpath.axes.NodeSequence.runTo(Unknown Source)
at org.apache.xpath.axes.NodeSequence.setRoot(Unknown Source)
at org.apache.xpath.axes.LocPathIterator.execute(Unknown Source)
... 16 more
This is the similar issue what I mentioned.
http://www-01.ibm.com/support/docview.wss?uid=swg1PK42574
Thakns.
Many people don't realise that the DOM is not thread-safe (even if you are only doing reads). If you are caching a DOM object, make sure you synchronize all access to it.
Frankly, this makes DOM unsuited to this kind of application. I know I'm biased, but I would suggest switching to Saxon, whose native tree implementation is not only far faster than DOM, but also thread-safe. One user recently tweeted about getting a 100x performance improvement when they made this switch.