I'm just learning asciidoc to create my own Documentation. So far I'm using the following markup to highlight source code:
[source,xml,subs="verbatim,attributes"]
----
<dependency>
<groupId>javax.jms</groupId>
<artifactId>javax.jms-api</artifactId>
<version>2.0.1</version>
</dependency>
----
I still haven't figured out how to highlight parts of the code.Any help ?
Thanks
I guess the asciidoctor way is to use callouts:
[source,xml,subs="verbatim,attributes"]
----
<dependency>
<groupId>javax.jms</groupId>
<artifactId>javax.jms-api</artifactId>
<version>2.0.1</version> <!--1-->
</dependency>
----
<1> : The version is part of the coordinates
This is rendered like this:
Related
I have a unit test to databricks code, and I want to run it locally on windows. Unluckily when I run pytest with PyCharm, it throws the following exception:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism(Ljava/lang/String;)V
at org.apache.hadoop.security.HadoopKerberosName.setConfiguration(HadoopKerberosName.java:84)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:315)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:300)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:575)
at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2747)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2747)
at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:79)
at org.apache.spark.deploy.SparkSubmit.secMgr$lzycompute$1(SparkSubmit.scala:368)
at org.apache.spark.deploy.SparkSubmit.secMgr$1(SparkSubmit.scala:368)
at org.apache.spark.deploy.SparkSubmit.$anonfun$prepareSubmitEnvironment$8(SparkSubmit.scala:376)
at scala.Option.map(Option.scala:230)
at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:376)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
And from source code it is from the initialization:
spark = SparkSession.builder \
.master("local[2]") \
.appName("Helper Functions Unit Testing") \
.getOrCreate()
I do search the above error and most of them are related to maven configure to add dependency of hadoop auth. However, for pyspark, I don't know how to deal with it. Does anyone have experience or insight for this error?
Here my workaround is to have python version to 3.7 and change pyspark version to 3.0, and then it seems ok. So it is related to the environment and dependency inconsistent.
This is just limit to my case, and from my search on web most is related to maven to add hadoop-auth.jar dependency for hadoop configuration.
Encountered this error for a Maven project written in Scala, not Python. What did it for me was adding not only the hadoop-auth dependency like OP specified but also the hadoop-common dependency in my pom file like so,
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>3.1.2</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-auth</artifactId>
<version>3.1.2</version>
</dependency>
Replace 3.1.2 with whatever version you're using. However, I also found that I had to find other dependencies that conflicted with hadoop-common and hadoop-auth and add exclusions to them like so,
<exclusions>
<exclusion>
<artifactId>hadoop-common</artifactId>
<groupId>org.apache.hadoop</groupId>
</exclusion>
<exclusion>
<artifactId>hadoop-auth</artifactId>
<groupId>org.apache.hadoop</groupId>
</exclusion>
</exclusions>
I am working on a bug using log4j2 and slf4j. I have been reviewing the commit history in Github, but cannot find where the bug was introduced. In the log file rawlogs.log the newline character %n has been replaced by %replace%n. Please see the example below:
applog.line=16,applog.msg=ConfigChangeListener Initilized!%replace%nXXXXXXXX03172 123456789 2019-06-11 15:06:35.080 devHostname
I am using the following dependencies:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.10.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.10.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-to-slf4j</artifactId>
<version>2.10.0</version>
</dependency>
In the files with logs the LoggerFactory is created with the example below:
private static final Logger LOGGER = LoggerFactory.getLogger(ClassName.class);
I have tried using both a log4j2.properties, and log4j2.xml files from log4j2 examples, but always get this error on startup:
2019-06-11 15:19:16,063 warmup - 7 ERROR No log4j2 configuration file found. Using default configuration: logging only errors to the console.
The rawlogs.log file is created and populated on startup.
Has anyone experienced this before? Any insight is appreciated.
EDIT: To clarify, project did not use a .xml or .properties file for configurations. I introduced them in an attempt to expose the root cause of the bug, but have not had success.
I am using this piece of code to calculate spark recommendations:
SparkSession spark = SparkSession
.builder()
.appName("SomeAppName")
.config("spark.master", "local[" + args[2] + "]")
.config("spark.local.dir",args[4])
.getOrCreate();
JavaRDD<Rating> ratingsRDD = spark
.read().textFile(args[0]).javaRDD()
.map(Rating::parseRating);
Dataset<Row> ratings = spark.createDataFrame(ratingsRDD, Rating.class);
ALS als = new ALS()
.setMaxIter(Integer.parseInt(args[3]))
.setRegParam(0.01)
.setUserCol("userId")
.setItemCol("movieId")
.setRatingCol("rating").setImplicitPrefs(true);
ALSModel model = als.fit(ratings);
model.setColdStartStrategy("drop");
Dataset<Row> rowDataset = model.recommendForAllUsers(50);
These are maven dependencies to make this piece of code work:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.8</version>
</dependency>
Calculating recommendations with this code takes ~70sec for my data file. This code produces following warning:
WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
WARN LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK
WARN LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeRefLAPACK
Now I try to enable netlib-java by adding this dependency in maven:
<dependency>
<groupId>com.github.fommil.netlib</groupId>
<artifactId>all</artifactId>
<version>1.1.2</version>
<type>pom</type>
</dependency>
to avoid crashing of this new environment I had to do this extra trick:
LD_PRELOAD=/usr/lib64/libopenblas.so
Now it also works, it gives no warnings, but it works slower and it takes ~170sec on average to perform the same calculation. I am running this on CentOS.
Shouldn't it be faster with native libraries? Is it possible to make it faster?
first you can check you centos version ,for centos 6 may not using the native libraries, check this
As far as I konw , the ALS algorithm has been improved since 2.0 version , you can check
Highlights in 2.2
And the source code from 2.2 as blow :
so the native libraries has not help!
I want to make a GWT app with the Errai framework but I run in some problems with the Data Binding and Validation.
My pom.xml
<dependency>
<groupId>org.jboss.errai</groupId>
<artifactId>errai-validation</artifactId>
<version>${errai.version}</version>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<classifier>sources</classifier>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>4.2.0.Final</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>4.2.0.Final</version>
<scope>provided</scope>
<classifier>sources</classifier>
</dependency>
My app.gwt.xml includes the Errai-Validation and HibernateValidator modules:
<inherits name="org.jboss.errai.validation.Validation" />
<inherits name="org.hibernate.validator.HibernateValidator" />
There are no unresolved dependencies I have already double checked this.
When I try to run the application with mvn gwt:run I'm getting the following error:
java.util.concurrent.ExecutionException: org.jboss.errai.ioc.rebind.ioc.exception.UnsatisfiedDependenciesException: #> org.jboss.errai.ui.nav.client.local.Navigation
- field org.jboss.errai.codegen.meta.MetaField:org.jboss.errai.ui.nav.client.local.Navigation.stateChangeEvent could not be satisfied for type: org.jboss.errai.ioc.client.lifecycle.api.StateChange
Message: can't resolve bean: org.jboss.errai.ioc.client.lifecycle.api.StateChange<java.lang.Object> ( #Default )
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at org.jboss.errai.config.rebind.AsyncGenerators$FutureWrapper.get(AsyncGenerators.java:112)
at org.jboss.errai.config.rebind.AsyncGenerators$FutureWrapper.get(AsyncGenerators.java:86)
at org.jboss.errai.config.rebind.AbstractAsyncGenerator.startAsyncGeneratorsAndWaitFor(AbstractAsyncGenerator.java:100)
at org.jboss.errai.ioc.rebind.ioc.bootstrapper.IOCGenerator.generate(IOCGenerator.java:58)
at com.google.gwt.core.ext.IncrementalGenerator.generateNonIncrementally(IncrementalGenerator.java:40)
at com.google.gwt.dev.javac.StandardGeneratorContext.runGeneratorIncrementally(StandardGeneratorContext.java:657)
at com.google.gwt.dev.cfg.RuleGenerateWith.realize(RuleGenerateWith.java:41)
at com.google.gwt.dev.shell.StandardRebindOracle$Rebinder.rebind(StandardRebindOracle.java:79)
at com.google.gwt.dev.shell.StandardRebindOracle.rebind(StandardRebindOracle.java:276)
at com.google.gwt.dev.shell.ShellModuleSpaceHost.rebind(ShellModuleSpaceHost.java:141)
at com.google.gwt.dev.shell.ModuleSpace.rebind(ModuleSpace.java:595)
at com.google.gwt.dev.shell.ModuleSpace.rebindAndCreate(ModuleSpace.java:465)
at com.google.gwt.dev.shell.GWTBridgeImpl.create(GWTBridgeImpl.java:49)
at com.google.gwt.core.shared.GWT.create(GWT.java:57)
at com.google.gwt.core.client.GWT.create(GWT.java:85)
at org.jboss.errai.ioc.client.Container.bootstrapContainer(Container.java:64)
at org.jboss.errai.ioc.client.Container.onModuleLoad(Container.java:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.google.gwt.dev.shell.ModuleSpace.onLoad(ModuleSpace.java:406)
at com.google.gwt.dev.shell.OophmSessionHandler.loadModule(OophmSessionHandler.java:200)
at com.google.gwt.dev.shell.BrowserChannelServer.processConnection(BrowserChannelServer.java:526)
at com.google.gwt.dev.shell.BrowserChannelServer.run(BrowserChannelServer.java:364)
at java.lang.Thread.run(Thread.java:744)
Thats why the Bootstrap is failing and the application is throwing a onModuleLoad Exception and is not starting.
If I remove the 2 validation modules I'm able to start the application without any errors.
Im using the Errai Tutorial with version 3.0.1 FINAL.
Thanks for your help :)
EDIT:
I resolved the error by adding
<inherits name="org.jboss.errai.ui.nav.Navigation" />
to my app.gwt.xml but now I'm running into the next problem with this exception:
java.lang.RuntimeException: Deferred binding failed for 'org.jboss.errai.validation.client.ValidatorFactoryImpl$GwtValidator' (did you forget to inherit a required module?)
at com.google.gwt.dev.shell.GWTBridgeImpl.create(GWTBridgeImpl.java:53)
at com.google.gwt.core.shared.GWT.create(GWT.java:57)
at com.google.gwt.core.client.GWT.create(GWT.java:85)
at org.jboss.errai.validation.client.ValidatorFactoryImpl.createValidator(ValidatorFactoryImpl.java:11)
at com.google.gwt.validation.client.AbstractGwtValidatorFactory.getValidator(AbstractGwtValidatorFactory.java:90)
at org.jboss.errai.validation.client.ValidatorProvider.get(ValidatorProvider.java:37)
at org.jboss.errai.ioc.client.BootstrapperImpl$28.getInstance(BootstrapperImpl.java:432)
at org.jboss.errai.ioc.client.BootstrapperImpl$28.getInstance(BootstrapperImpl.java:1)
at org.jboss.errai.ioc.client.container.IOCDependentBean.getInstance(IOCDependentBean.java:96)
at org.jboss.errai.ioc.client.container.IOCDependentBean.getInstance(IOCDependentBean.java:87)
at org.jboss.errai.ioc.client.container.SyncToAsyncBeanManagerAdapter$1.getInstance(SyncToAsyncBeanManagerAdapter.java:148)
at org.jboss.errai.ui.nav.client.local.spi.GeneratedNavigationGraph$2.produceContent(GeneratedNavigationGraph.java:69)
at org.jboss.errai.ui.nav.client.local.Navigation.maybeShowPage(Navigation.java:304)
at org.jboss.errai.ui.nav.client.local.Navigation.navigate(Navigation.java:249)
at org.jboss.errai.ui.nav.client.local.Navigation.navigate(Navigation.java:230)
at org.jboss.errai.ui.nav.client.local.Navigation.navigate(Navigation.java:225)
at org.jboss.errai.ui.nav.client.local.Navigation.goTo(Navigation.java:191)
at org.jboss.errai.ui.nav.client.local.DefaultNavigationErrorHandler.handleError(DefaultNavigationErrorHandler.java:27)
at org.jboss.errai.ui.nav.client.local.Navigation.goTo(Navigation.java:193)
Is there another module that is missing?
I'm correct that Errai is creating the ValidationFactory and injecting the correct instance? So I don't have to create my own ValidationFactory like here:
GWT Validation Tutorial
Yes, that's correct. You don't have to create your own ValidationFactory. Errai will do that for you. You can simply #Inject a Validator.
I have prepared a version of the Errai tutorial using 3.0.1.Final that shows exactly that (following the instructions from the reference guide). I've put the project on GitHub.
The last error you pasted doesn't contain enough information to investigate why this is failing for you. However, you should see more error information in the devmode console.
When building the sample application from the Stanford CoreNLP website, I ran into a curious exception:
Exception in thread "main" java.lang.RuntimeException: edu.stanford.nlp.io.RuntimeIOException: Unrecoverable error while loading a tagger model
at edu.stanford.nlp.pipeline.StanfordCoreNLP$4.create(StanfordCoreNLP.java:493)
…
Caused by: java.io.IOException: Unable to resolve "edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger" as either class path, filename or URL
…
This only happened when the property pos and the ones after it were included in the properties.
Properties props = new Properties();
props.put("annotators", "tokenize, ssplit, pos, lemma, ner, parse, dcoref");
StanfordCoreNLP pipeline = new StanfordCoreNLP(props);
Here is the dependency from my pom.xml:
<dependencies>
<dependency>
<groupId>edu.stanford.nlp</groupId>
<artifactId>stanford-corenlp</artifactId>
<version>3.2.0</version>
<scope>compile</scope>
</dependency>
</dependencies>
I actually found the answer to that in the problem description of another question on Stackoverflow.
Quoting W.P. McNeill:
Maven does not download the model files automatically, but only if you
add models line to the .pom. Here is a .pom
snippet that fetches both the code and the models.
Here's what my dependencies look like now:
<dependencies>
<dependency>
<groupId>edu.stanford.nlp</groupId>
<artifactId>stanford-corenlp</artifactId>
<version>3.2.0</version>
</dependency>
<dependency>
<groupId>edu.stanford.nlp</groupId>
<artifactId>stanford-corenlp</artifactId>
<version>3.2.0</version>
<classifier>models</classifier>
</dependency>
</dependencies>
The important part to note is the entry <classifier>models</classifier> at the bottom. In order for Eclipse to maintain both references, you'll need to configure a dependency for each stanford-corenlp-3.2.0 and stanford-corenlp-3.2.0-models.
In case you need to use the models for other languages (like Chinese, Spanish, or Arabic) you can add the following piece to your pom.xml file (replace models-chinese with models-spanish or models-arabic for these two languages, respectively):
<dependency>
<groupId>edu.stanford.nlp</groupId>
<artifactId>stanford-corenlp</artifactId>
<version>3.8.0</version>
<classifier>models-chinese</classifier>
</dependency>
With Gradle apparently you can use:
implementation 'edu.stanford.nlp:stanford-corenlp:3.9.2'
implementation 'edu.stanford.nlp:stanford-corenlp:3.9.2:models'
or if you use compile (depricated):
compile group: 'edu.stanford.nlp', name: 'stanford-corenlp', version: '3.9.2'
compile group: 'edu.stanford.nlp', name: 'stanford-corenlp', version: '3.9.2' classifier: 'models'