I am new to stanford-nlp API. I am trying to just sentimental analysis with stanford API but it's throwing exception. please see the below logs.
Adding annotator tokenize
Adding annotator ssplit
Adding annotator pos
Reading POS tagger model from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [1.4 sec].
Adding annotator lemma
Adding annotator ner
Loading classifier from edu/stanford/nlp/models/ner/english.all.3class.distsim.crf.ser.gz ... done [5.3 sec].
Loading classifier from edu/stanford/nlp/models/ner/english.muc.7class.distsim.crf.ser.gz ... done [2.3 sec].
Loading classifier from edu/stanford/nlp/models/ner/english.conll.4class.distsim.crf.ser.gz ... done [4.7 sec].
Adding annotator parse
Loading parser from serialized file edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz ... done [1.1 sec].
Adding annotator dcoref
Adding annotator sentiment
Exception in thread "main" java.lang.NoClassDefFoundError: org/ejml/simple/SimpleBase
at edu.stanford.nlp.pipeline.SentimentAnnotator.<init> (SentimentAnnotator.java:48)
at edu.stanford.nlp.pipeline.StanfordCoreNLP$14.create(StanfordCoreNLP.java:850)
at edu.stanford.nlp.pipeline.AnnotatorPool.get(AnnotatorPool.java:81)
at edu.stanford.nlp.pipeline.StanfordCoreNLP.construct(StanfordCoreNLP.java:262)
at edu.stanford.nlp.pipeline.StanfordCoreNLP.<init>(StanfordCoreNLP.java:129)
at edu.stanford.nlp.pipeline.StanfordCoreNLP.<init>(StanfordCoreNLP.java:125)
at io.stanford.NLP.findSentiment(NLP.java:30)
at io.stanford.TestStanford.main(TestStanford.java:8)
Caused by: java.lang.ClassNotFoundException: org.ejml.simple.SimpleBase
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 8 more
With the Stanford CoreNLP 3.5.2 distribution there should be a .jar file called ejml-0.23.jar which contains the missing class ; make sure to put this jar in your classpath, in fact you probably want all of the jars that come with Stanford CoreNLP 3.5.2 in your classpath!
What is the code that produces this output? My strong suspicion is that you have not included the "sentiment" annotator in your annotators list, either in the properties file you are using to run the code, or the properties object you have passed into the annotation pipeline. Without running the sentiment annotator, the document will not have the sentiment annotations attached, and will therefore null pointer when trying to retrieve them.
Related
I have two data sources. The moment I add ANY repository to the second datasource, this error comes up for whatever entity the repository used.
Using spring config instead of persistance, and EclipseLink JPA. The strange thing is that is has nearly identical to a working project that was used as template. Different data sources and obviously different tree scanned, but otherwise the config seems setup exactly the same. What is the equivalent spring data config equivalent to exclude-unlisted-classes. I will happily put up code but can anyone give me hint of where I should start looking?
stackdump looks like:
Caused by: java.lang.IllegalArgumentException: No [ManagedType] was found for the key class [nz.cri.gns.mapservice.userdomain.DataUser] in the Metamodel - please verify that the [Managed] class was referenced in persistence.xml using a specific nz.cri.gns.mapservice.userdomain.DataUser property or a global false element.
at org.eclipse.persistence.internal.jpa.metamodel.MetamodelImpl.entityEmbeddableManagedTypeNotFound(MetamodelImpl.java:177)
at org.eclipse.persistence.internal.jpa.metamodel.MetamodelImpl.managedType(MetamodelImpl.java:519)
at org.springframework.data.jpa.repository.support.JpaMetamodelEntityInformation.(JpaMetamodelEntityInformation.java:68)
at org.springframework.data.jpa.repository.support.JpaEntityInformationSupport.getEntityInformation(JpaEntityInformationSupport.java:67)
at org.springframework.data.jpa.repository.support.JpaRepositoryFactory.getEntityInformation(JpaRepositoryFactory.java:152)
at org.springframework.data.jpa.repository.support.JpaRepositoryFactory.getTargetRepository(JpaRepositoryFactory.java:99)
at org.springframework.data.jpa.repository.support.JpaRepositoryFactory.getTargetRepository(JpaRepositoryFactory.java:81)
at org.springframework.data.repository.core.support.RepositoryFactorySupport.getRepository(RepositoryFactorySupport.java:185)
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.initAndReturn(RepositoryFactoryBeanSupport.java:251)
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.afterPropertiesSet(RepositoryFactoryBeanSupport.java:237)
at org.springframework.data.jpa.repository.support.JpaRepositoryFactoryBean.afterPropertiesSet(JpaRepositoryFactoryBean.java:92)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)
Duh! Make sure everything in the SetPackagesToScan is spelled correctly! No errors result from a typo, but classes dont go into the metamodel either.
I'm using Spring Boot 1.4.2 and both Hibernate and JPA in my project. I use IntelliJ Community 2016.2.4, same as my colleague. The difference is that when he types something in HQL (i.e. #Query("SELECT id FROM usesrs)) it automatically checks the table names against the real DB structure - there is no "usesrs" table, but there is a "user" table. So his project doesn't compile unless he corrects it - what a wonderful feature! Sometimes it takes me a lot of time to spot such bugs...
Can you tell me how can I enable this in IntelliJ? What I tried:
1) When I go to Preferences / Editor / Language injections / "+" sign / Java Parameter I don't see SQL on the list, not speaking about HQL (but JQL is there which is crazy...). Anyway looks like I want to enable something deeper than just plain syntax check, it has to check against the database structure.
2) When I go to File / Project structure / Facets / "+" sign I don't see JPA there, just some Android options and Java-Gradle.
Any suggestions here?
Unfortunate JPA/Hibernate features are supported in the Ultimate version.
IDEA has the Language Injections which can inject a language (such as SQL, RegExp, XML, HTML, CSS, etc.) into a string literal in your code and, as a result, get comprehensive coding assistance when editing that literal.
You can get a help here and a tip here to Using language injections in SQL
In the Database view is your datasource setup?
If it's not already visible go to view -> Tool Windows -> Database
There you should see a data source for your connection.
If it is not there you'll need to set it up using + -> "Data Source" then select the DB that your using and set up your connection.
Once this is done OR if the DB Connection is already present you'll want to synchronize the datasource using the "Refresh" looking icon in the Database view.
Then click on more schemas and pick the schema your using and hit Synchronize on that as well. Intellij should now know about the DB and your table structures.
Hopefully it can then check your queries automatically unless there is something else amiss.
Three conclusions on this topic:
1) As #Liping Huang pointed out, the live checks (and I imagine mistakes highligting) is only in Ultimate edition
2) I investigated the issue more deeply and looks like I actually have some basic support for this, but I just omitted the error message hidden lower in the stack. In the Community I have to try to run the app, so no live highlighting. Then, if there is an error in the table name, the message will be in the stack trace.
Here is the example, notice the "Usesrs is not mapped" message:
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'protectedController': Unsatisfied dependency expressed through field 'userRepository'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'userRepositoryImpl': Unsatisfied dependency expressed through field 'userRepository'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'userRepository': Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: Validation failed for query for method public abstract void com.ekinnolab.database.repository.user.userRepository.setAsDeletedById(java.lang.Long)!
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:592) ~[spring-beans-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88) ~[spring-beans-4.3.4.RELEASE.jar:4.3.4.RELEASE]
(…)
// cutting out 150 about lines
(...)
at org.hibernate.internal.AbstractSessionImpl.createQuery(AbstractSessionImpl.java:240) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.internal.SessionImpl.createQuery(SessionImpl.java:1894) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.createQuery(AbstractEntityManagerImpl.java:291) ~[hibernate-entitymanager-5.0.11.Final.jar:5.0.11.Final]
... 69 common frames omitted
HERE ==> Caused by: org.hibernate.hql.internal.ast.QuerySyntaxException: Usesrs is not mapped
at org.hibernate.hql.internal.ast.util.SessionFactoryHelper.requireClassPersister(SessionFactoryHelper.java:171) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.ast.tree.FromElementFactory.addFromElement(FromElementFactory.java:91) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.ast.tree.FromClause.addFromElement(FromClause.java:76) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.ast.HqlSqlWalker.createFromElement(HqlSqlWalker.java:321) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.fromElement(HqlSqlBaseWalker.java:3687) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.fromElementList(HqlSqlBaseWalker.java:3576) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.fromClause(HqlSqlBaseWalker.java:716) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.updateStatement(HqlSqlBaseWalker.java:373) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.antlr.HqlSqlBaseWalker.statement(HqlSqlBaseWalker.java:265) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.analyze(QueryTranslatorImpl.java:262) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
at org.hibernate.hql.internal.ast.QueryTranslatorImpl.doCompile(QueryTranslatorImpl.java:190) ~[hibernate-core-5.0.11.Final.jar:5.0.11.Final]
... 77 common frames omitted
3) BTW, for pure SQL there are some interesting plugins out there. Just go to IntelliJ IDEA / Preferences / Plugins / Browse repositories. "SQL Query Plugin" is one of those looking promising.
Why does spark throw NotSerializableException org.apache.hadoop.io.NullWritable with sequence files? My code (very simple):
import org.apache.hadoop.io.{BytesWritable, NullWritable}
sc.sequenceFile[NullWritable, BytesWritable](in).repartition(1000).saveAsSequenceFile(out, None)
The exception
org.apache.spark.SparkException: Job aborted: Task 1.0:66 had a not serializable result: java.io.NotSerializableException: org.apache.hadoop.io.NullWritable
at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$abortStage$1.apply(DAGScheduler.scala:1028)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$org$apache$spark$scheduler$DAGScheduler$$abortStage$1.apply(DAGScheduler.scala:1026)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$abortStage(DAGScheduler.scala:1026)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$processEvent$10.apply(DAGScheduler.scala:619)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$processEvent$10.apply(DAGScheduler.scala:619)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.scheduler.DAGScheduler.processEvent(DAGScheduler.scala:619)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$start$1$$anon$2$$anonfun$receive$1.applyOrElse(DAGScheduler.scala:207)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
at akka.actor.ActorCell.invoke(ActorCell.scala:456)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
So it is possible to read non-serializable types into an RDD - i.e. have an RDD of something that is not serializable (which seems counter intuitive). But once you wish to perform an operation on that RDD that requires the objects to be serializable, like repartition it needs to be serializable. Moreover it turns out that those weird classes SomethingWritable, although invented for the sole perpose of serializing things are not actually serializable :(. So you must map these things to byte arrays and back again:
sc.sequenceFile[NullWritable, BytesWritable](in)
.map(_._2.copyBytes()).repartition(1000)
.map(a => (NullWritable.get(), new BytesWritable(a)))
.saveAsSequenceFile(out, None)
Also see: https://stackoverflow.com/a/22594142/1586965
In spark if you try to use a third party class which is not serializable it throws NotSerializable exception.It's because of the closure property of spark i.e whatever instance variable (which are defined outside the transformation operation) you try to access inside a transformation operation spark tries to serialize it as well as all the dependent classes of that object.
I got this exception!!!
Caused by: java.lang.IllegalStateException: Cannot convert value of type [org.hibernate.impl.SessionFactoryImpl] to required type [com.liferay.portal.kernel.dao.orm.SessionFactory] for property 'sessionFactory': no matching editors or conversion strategy found
at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:289)
at org.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:154)
at org.springframework.beans.BeanWrapperImpl.convertForProperty(BeanWrapperImpl.java:452)
... 41 more
Do you have any solution?
One of the beans that you pass sessionFactory to them require the Liferay one instead of the Hibernate one.
There are 2 possible mistakes:
You made a mistake in wiring - fix that by passing Lifray's SessionFactory (create one if you don't have one already)
The mistake is in Java class imports - fix the imports
I am trying to use my own TableInputFormat for a MapReduceJob in the following way
TableMapReduceUtil.initTableMapperJob("mytable",
MyScan,
MyMapper.class,
MyKey.class,
MyValue.class,
myJob,true, MyTableInputFormat.class);
When I run the job, I get a ClassNotFoundException: org.apache.hadoop.hbase.mapreduce.TableMapper - any idea why ?
if I do not use the last two parameters of initTableMapperJob, then there is no ClassNotFoundException (but obviously that defeats the purpose)
I am struggling on this for few days now.
This is somewhere someone did the same thing Extending Hadoop's TableInputFormat to scan with a prefix used for distribution of timestamp keys, but I am not being able to ask the question on that thread.
I am working on a Cloudera Cluster 4.3.0 with Hadoop 2
Adding the stacktrace error:
java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapreduce.TableMapper at
java.net.URLClassLoader$1.run(URLClassLoader.java:202) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:190) at
java.lang.ClassLoader.loadClass(ClassLoader.java:306) at
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at
java.lang.ClassLoader.loadClass(ClassLoader.java:247) at
java.lang.ClassLoader.defineClass1(Native Method) ....
Thanks a lot for helping
Regards
Please see Overriding TableMapper splits. So I overrode TableMapReduceUtil and added TableMapper.class to the addDependencyJars method. Then I proceeded in the same way
MyTableMapReduceUtil.initTableMapperJob("MyTable", // input table
myScan,
MyMapper.class,
MyKey.class,
MyValue.class,
myJob,
true,CustomSplitTableInputFormat.class);
Where CustomSplitTableInputFormat extends TableInputFormat