Compiling Apache Hadoop Source in Eclipse - maven

After about 4 tries I've managed to use git to checkout apache's Hadoop source code, issue a
mvn eclipse:eclipse
command and then import all of the projects into eclipse. So far this has been the most successful I have been. I am ALMOST there. I attempted to build and clean all projects and everything was going well. Now I only have 3 errors to get rid of. I would be EXTREMELY appreciate if anyone could help me with this. I have been trying to get this to work for about 2 days or so. Anyway, in Eclipse I have about 20 Hadoop projects. There are only two that have errors. One is called "hadoop-streaming" and the other is "hadoop-tools-dist."
The error says the following:
Project 'hadoop-streaming' is missing required source folder: 'C:/Users/User/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf'
The other errors are mainly related to this one. That's what Eclipse says anyway. I tried to copy the folder to where it was needed but that didn't clear up the error. Does anyone know how to fix this? If you need anything from me just ask.

I don't know how the Hadoop project looks like, but I suppose that it is using maven. So instead of using the
mvn eclipse:eclipse
I would suggest using the import functionality of eclipse -> import maven projects (even though sometimes works harder than one might expect). Related to your issue, this kind of error occurred to me whenever there was some kind of folder that is not added as source folder in eclipse (generated folders for source etc.). So, I would advise checking that the folder you mentioned is already there and if it is added as a source folder. If not, just mark it accordingly (source folder). Clean the project (Project > Clean...) and if need update the maven project(Right click on the parent project > Maven > Update project ...)
Good luck!

In eclipse:
right click on the hadoop-streaming program
select Properties
On the left side of the bar,click Java Build Path button,then select source lable page
delete the path that went wrong
click Link Source
select the linked directory as (your hadoop-src root path)/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf
click Next button, inclusion patterns add capacity-scheduler.xml,exclusion patters add **/*.java

Related

Could not load compiled classes for initialization script from cache

So i ran a java program in IntelliJ using Gradle for 2 hours and then my computer (which has Windows as its OS) didn't work correctly so I had to restart it. But then this problem occurred when building my Gradle Project, where it told me that it "Could not load compiled classes for initialization script from cache".
Solution moved from question
The solution is really simple but it took me a long long time to figure it out so I wanted to share it with anyone who might also stumble across this problem (I also haven't seen a simple solution elsewhere).
Step 1: Open the Explorer and go to "C:/Users/username/.gradle/caches" (if you don't see the .gradle folder in your file explorer you have either moved it to a different directory or you have to enable the ability to view hidden folders in your file explorer: https://support.microsoft.com/en-us/windows/view-hidden-files-and-folders-in-windows-97fbc472-c603-9d90-91d0-1166d1d9f4b5 , I don't mean the .gradle folder inside your IntelliJ project but the Gradle user home directory which you can find in IntelliJ by clicking "File > Settings > Build, Execution, Deployment > Build Tools > Gradle")
Step 2: delete everything inside the caches folder (in other words, clear the Gradle cache)
Step 3: start IntelliJ and rebuild your Project with gradle
I did that and then my gradle Build worked again perfectly and I could run my code again. I hope it helps :)!

Unable to find Groovy method when building Rhino with Intellij IDEA

While trying to build https://github.com/mozilla/rhino, I'm getting Unable to find method on org.codehaus.groovy.runtime.StringGroovyMethods.
so the build fails, and I'm unable to test Rhino built from source code, which is the goal.
Steps Taken
From the opening screen in IntelliJ, I check out the project:
I get this error Unable to find method ''java.lang.String org.codehaus.groovy.runtime.StringGroovyMethods.capitalize(java.lang.String)'' right away, which I have not been able to fix:
There are two suggestions in the above Re-download dependencies and sync project and Stop Gradle build processes. I have tried both with no improvement.
When I look at the build.gradle file, I see another suggestion ...configure Gradle wrapper to use distribution with sources.... This also does not get past the missing Groovy method problem.
I have looked on S.O. for similar issues and tried various things under File > Settings > Build, Execution, Deployment, but I realized I'm over my head since, conceptually, I'm not sure what I need and where that would go in the settings.
The command line items from the readme work as expected, but going back to File > Build still fails (added after tim_yates comment).
What changes are required to the IDE or to the build definitions to allow Mozilla Rhino to build properly?
I installed the latest (2022.1) version of IntelliJ IDEA Community Edition. During the install, it asked if you wanted this IDE to be associated with Groovy file types, and I answered in the affirmative.
On this new install, the Unable to find method on org.codehaus.groovy.runtime.StringGroovyMethodserror did not appear, so the problem was solved by upgrading the IDE (and also that required updating GIT).

STS on Linux - Could Not Calculate Build Plan

After installing the Spring Tool Suite, I tried to create a maven project and this is what I get in an instant:
So waddap Maven/STS creators? How do I make your product work?
To any noobs like me who got overwhelmed by this error, here's what I did:
Go to your .m2/repository, where I think Maven stores its configurations files, then wipe it clean, delete all files, delete the 'repository' folder even.
I then closed the STS IDE and restarted it. You can see in 'Progress tab' that Maven is downloading new files.
Saw the same 5 errors, right clicked and clicked 'Quick Fix'. That's it.
This post was helpful too.

Sonar refuses to analize multimodule project

I have upgraded from Sonar 5.3 to 6.2 and a project X is giving me some weird errors at the sonar stage of the jenkins job (altough build finishes as STABLE).
The project is a standard multimodule maven java project.
Entering the project dashboard a single message appears on screen:
"No analysis has been performed since creation. The only available
section is the configuration."
But in the upper-right corner the red "FAILED" tag shows up. That leads to the Background tasks of the project and there is the failed task with this error log:
The project "com.foo:bar-submodule-1" is already defined in SonarQube
but not as a module of project "com.foo:bar". If you really want to
stop directly analysing project "com.foo:bar-submodule-1", please
first delete it from SonarQube and then relaunch the analysis of
project "com.foo:bar"
I do not want to delete the project and lose the historical data.
So the question is: How can i add a project Y (that is a submodule of project X but Sonar does not recognize it) as a submodule of project X?
EDIT 1
The parent project is the only one that has been analyzed through jenkins jobs.
Your problem is not that Sonar can't figure out your submodules, but the root issue is that there is another project/module with the same name conflicting with the project you are analyzing now. It happens, for example, when you extract a submodule as a standalone library and moves it as a root project and try to analyse the new project. Since it is a new project, but sometime in the past you had another project/module with the exact same name, it won't be able to finish the process. I have found three ways to fix this:
Delete the old project, if it is possible
This is a option that I use when I have analysed mistakenly new projects (with wrong parameters for example) or when I am migrating servers and still have some wrong configuration in place. You can do this in the project's Configuration/Administration menu, 'Delete project'.
Rename the old projects keys
This is my default option, since usually what I want is just keep the old analyses data for old submodules that we are moving to other projects or promoting to root modules. You can do this in the project's Configuration/Administration menu, 'Update key' option.
Updating your current project's keys
This option I use when the submodules conflict with other valid submodules (It can happen when you have different teams creating small libraries that happen to have the same keys). You can do this in two steps. First you need to update your project's keys following the option 2. The next step is to go to your project and update the modules and submodules names using the key sonar.moduleKey. The default value for this is : similar to sonar.projectKey, as especified here. Following these steps you won't lose all your records and old analyses.
I hope it helps.

copying local source code to jenkins Workspace

I want to set up a small cucumber (maven) project in Jenkins in my company. The source code is local on my machine and not on any subversion system. I have already created a new job in jenkins. While configuring I am asked to give path to my pom.xml. But I dont know how do I give path.
Also I triggered Jenkins build without even configuring it properly, and therefore it has generated Workspace (Arbeitsbereich in german) for me. Now I think I just need to copy my source code to the created Workspace, but I dont know how do I do that exactly.
May be I am doing something really dumb here :( could someone please help ?
Jenkins default directory would be '~/.jenkins' where your jobs should be located. you can follow this link to use a custom directory as your work space.
After that, you can specify the path of your application pom.xml file relatively to your custom workspace location.

Resources