I have a spark job I'm developing in IntelliJ. It's builds via maven, the tests pass, and I can run the job locally. However, If I try to run the tests via IntelliJ, I get
Error:scalac: bad symbolic reference. A signature in
SparkContext.class refers to term akka in package root which is not
available. It may be completely missing from the current classpath, or
the version on the classpath might be incompatible with the version
used when compiling SparkContext.class.
I ended up just busting and rebuilding the IMLs in IntelliJ and this fixed the issue.
Related
We have a Bamboo build that includes a multi-module Maven build task, for a mixed scala/java project.
The build has worked fine for many months, but started failing today.
Maven successfully builds several modules, then consistently fails on a particular module, with the error:
[ERROR] No compiler is provided in this environment.
Perhaps you are running on a JRE rather than a JDK?
However, a JDK is installed, and this module compiles and builds fine if I log into the bamboo server and run maven manually just for this module.
JAVA_HOME appears to be set correctly on the bamboo server and points to a valid JDK, and I can run javac -version without issue.
No updates have been made to this server recently.
How can we fix this build?
I've found the problem (though not the root cause).
The Maven module that fails is the first one that includes java source files (rather than pure scala), so it is the first module for which Maven tries to invoke javac.
The Maven task in Bamboo specifies a particular named JDK profile.
Checking the Bamboo settings for this JDK, I found that it is set to a path that actually points to a JRE (as indicated by the original error message).
(A JRE is no good because it doesn't include the javac compiler and other build tools).
It is not clear how this path got changed/corrupted (nobody else had access to the server, so it suggests an issue within Bamboo itself) but editing this path to point to the installed JDK again fixes the build.
I'm currently trying to migrate a project from Gradle 3.5 to the latest Gradle version 4.9. and I'm running into the problem, that Gradle doesn't find the resources that it needs to execute the junit tests. These resources are generated upfront and contain some configuration files and referential data.
The project uses a combination of npm and Gradle to compile. Usually we run a npm script which executes a couple of tasks sequentially using run-s, e.g. webpack builds to generate some static resources. At the end we compile everything together using Gradle. The last Gradle task also executes the junit tests. This step fails and makes the whole build fail. The Gradle runner complains that some resources are not found, although the get properly copied to the right places.
When I execute ./gradlew test --rerun-tasks (Yes, we use the wrapper, which has been properly upgraded, too) right after the last Gradle build step failed, it executed the junit tests successfully.
I know, that the directory structure changed in Gradle 4.x, but this doesn't seem to be the problem.
My suspicion is, that it has something to do with how we normally run the whole build process. Something in the combination with npm.
I just converted my Maven project to a gradle project. It was a multi project structure:
master-project
pom.xml
---->project1
-------->pom.xml
---->project2
-------->pom.xml
---->project3
-------->pom.xml
I ran a gradle init on it and have this structure now:
master-project
build.gradle
---->project1
-------->build.gradle
---->project2
-------->build.gradle
---->project3
-------->build.gradle
Everything builds fine, and I have been able to get some things done with that I couldn't figure out how to do with Maven, so that's great. Next step was to integrate that into the IDE since the Maven Dependencies are gone since I have removed the pom.xml files.
However the project isn't recognized as a gradle project - and I am not sure how to change that?
In Eclipse Mars it's still recognized as a Maven build, not gradle....
Thanks in advance.
EDIT: I reimported the projects which enabled the plugin for Eclipse. Now I am having weird behavior.
The build works from the command line, however when attempting the same execution from within Eclipse, it fails trying to copy the file dependencies.
For example:
Couldn't copy dependency jakarta-regexp-1.4.jar
java.nio.file.NoSuchFileException: C:\Users\user.m2\repository\jakarta-regexp\jakarta-regexp\1.4\jakarta-regexp-1.4.jar -> build\jfx\app\lib\jakarta-regexp-1.4.jar
I haven't changed the repo from maven yet - just changed the build scripts. This is running from the master project. So I am confused as to why the script would work from the commandline but not from within eclipse.
EDIT 2: Turns out this behavior is also present when running from the command line when the --daemon flag is set. Is there anyway to run the tasks without the daemon in Buildship? Or perhaps a way to fix this issue when the --daemon flag is enabled?
Thanks.
The issue with the build was that there is a leak in the JDK when bundling the JRE with the native app. This only happens when running with the --daemon flag (which all IDEs user). Therefore until this is fixed you will need to run gradle --stop and then run the clean.
The plug in I am using is no longer running the native task when running with --daemon.
IntelliJ doesn't seem to put the provided dependency on the classpath when I run it, however I can do this successfully in Eclipse.
As it would be a lot more convenient for me, how can I do this in IntelliJ?
I'm having the same problem. Intellij does not include provided dependencies in classpath. See this source. The best solution I found is to run it as maven app, using the exec:java goal. For example:
exec:java -Dexec.classpathScope=compile -Dexec.mainClass=com.splout.db.integration.NShardEnsemble -Dexec.args=4
Better solutions are welcome.
Does it work in Maven via command line? The behaviour seems correct. Eclipse used to handle classpath badly, so I guess it still does.
There is a difference if you run something in Test source root or Source root, since the scope provided is only available on the compilation and test classpath.
If you run a test, or a main method in Test source root, then it can use provided dependencies, but if you try to execute something (via IntelliJ, or exec-maven-plugin) in Source root, then it will fail on ClassNotFoundException.
IntelliJ now has an option to Include dependencies with provided scope in the Run Configuration:
Any library marked as scope - provided means that the library (as the name suggests) is supposed to be provided by the JDK or the container (e.g. tomcat) at runtime.
this answer is based on #Meo's answer.
ALT + Enter to create a unit test:
then run it :
I'm having some difficulty understanding the purpose of this plugin. Is it to modify the settings in Tomcat during the build?
I am deploying to tomcat using maven, without the plugin and it seems to work fine. Not sure if I am missing something
Cheers
Maven Tomcat plugin basically just bootstraps an embedded Tomcat container for you. Saves you the trouble of configuring an external Tomcat instance for development purposes. It can also auto-start this Tomcat instance during the build and run integration tests on it, stopping it afterwards.
If you already have a functioning workflow that you're comfortable with, no need to introduce the plugin, but it's pretty easy to configure, can run multiple web apps, can run unassembled applications etc so it's convenient to have for local development.
An even more light-weight alternative would be the Jetty plugin which starts an embedded Jetty server instead.
Maven is actually a plugin execution framework where every task is actually done by plugins.
Maven Plugins are generally used to :
create jar file
create war file
compile code files
unit testing of code
create project documentation
create project reports
A plugin generally provides a set of goals and which can be executed using following syntax:
mvn [plugin-name]:[goal-name]
For example, a Java project can be compiled with the maven-compiler-plugin's compile-goal by running following command
mvn compiler:compile
for more information go to http://www.tutorialspoint.com/maven/maven_plugins.htm
so pulgins is used to execute goals.
suppose if you don't include plugin that is required in your execution environment then it will throw an error like
A required class is missing: Lorg/apache/maven/plugin/BuildPluginManager;
so by adding appropriate plugin in pom.xml it will resolve the dependencies and execute the goal succesfully.
to remove above error just add the following plugins :
<plugin>
<groupId>com.atlassian.maven.plugins</groupId>
<artifactId>maven-amps-plugin</artifactId>
<version>${amps.version}</version>
<extensions>true</extensions>
</plugin>
Maven is a framework used to build JAVA applications, it provides the following
Directory Structure
Configuration Files
Build Settings
It helps in easy and structured development, primarily used for REST API Calls.
Applications built on Maven Framework, would require the same to be deployed
Its better that you get the plugin installed, since on a long run you never know what dependency may go missing
-If this helps, Mark as Answer