To render my own spring boot starter guide, I followed these 2 pages:
https://github.com/spring-guides/getting-started-guides/wiki/Testing-edits-to-your-guide-and-macros
https://github.com/spring-guides/getting-started-guides/wiki/Run-sagan-to-test-your-own-content
I can run Sagan, but not with my own guide. Particularly, first wiki page says:
Example 1. sagan-site/src/main/resources/application.yml
...
guides:
owner:
name: ${GITHUB_GUIDES_OWNER_NAME:/*your githubusername*/}
type: ${GITHUB_GUIDES_OWNER_TYPE:users}
...
But nowhere in sagan's code it uses GITHUB_GUIDES_OWNER_NAME.
I did find sagan/sagan-renderer/src/main/resources/application.yml, which defines the organization property, but editing like this didn't get sagan to render my guide:
sagan:
renderer:
guides:
organization: ge0ffrey
# github:
...
# token: ${GITHUB_ACCESS_TOKEN:}
I got sagan to render my guide by hacking the sagan code (see below). Basically, get the guides list from spring-guides, but get your guide from your repository (ge0ffrey in my case).
For what it's worth (not much), here's the patch diff of my local hack:
Index: sagan-renderer/src/main/java/sagan/renderer/guides/GuideRenderer.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- sagan-renderer/src/main/java/sagan/renderer/guides/GuideRenderer.java (revision 68d9ced83d8ee97c7972a0d603a9077299c0435f)
+++ sagan-renderer/src/main/java/sagan/renderer/guides/GuideRenderer.java (date 1578485243882)
## -41,7 +41,7 ##
GuideContentResource guideContent = new GuideContentResource();
guideContent.setName(guideName);
String repositoryName = type.getPrefix() + guideName;
- String org = this.properties.getGuides().getOrganization();
+ String org = "ge0ffrey";
String tempFilePrefix = org + "-" + repositoryName;
File unzippedRoot = null;
Index: sagan-renderer/src/main/java/sagan/renderer/guides/GuidesController.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- sagan-renderer/src/main/java/sagan/renderer/guides/GuidesController.java (revision 68d9ced83d8ee97c7972a0d603a9077299c0435f)
+++ sagan-renderer/src/main/java/sagan/renderer/guides/GuidesController.java (date 1578484725643)
## -53,6 +53,13 ##
.toResources(this.githubClient.fetchOrgRepositories(properties.getGuides().getOrganization()))
.stream().filter(guide -> !guide.getType().equals(GuideType.UNKNOWN))
.collect(Collectors.toList());
+ Repository optaplannerGuide = new Repository(15L, "gs-constraint-solving-ai-optaplanner",
+ "spring-guides/gs-constraint-solving-ai-optaplanner", "OptaPlanner Guide description",
+ "https://github.com/ge0ffrey/gs-constraint-solving-ai-optaplanner/",
+ "git#github.com:ge0ffrey/gs-constraint-solving-ai-optaplanner.git",
+ "https://github.com/ge0ffrey/gs-constraint-solving-ai-optaplanner.git",
+ "git://github.com/ge0ffrey/gs-constraint-solving-ai-optaplanner.git", null);
+ guideResources.add(new GuideResource(optaplannerGuide));
Resources<GuideResource> resources = new Resources<>(guideResources);
for (GuideType type : GuideType.values()) {
## -70,7 +77,7 ##
if (GuideType.UNKNOWN.equals(guideType)) {
return ResponseEntity.notFound().build();
}
- Repository repository = this.githubClient.fetchOrgRepository(properties.getGuides().getOrganization(),
+ Repository repository = this.githubClient.fetchOrgRepository("ge0ffrey",
guideType.getPrefix() + guide);
GuideResource guideResource = this.guideAssembler.toResource(repository);
if (guideResource.getType().equals(GuideType.UNKNOWN)) {
Note that just overriding the organization in application.properties doesn't work, because it needs to get a full listing of all guides first. Maybe it works if you clone other repositories too (no idea which ones).
Related
We have a large project that has multiple separate declarative pipeline file definitions. This is used to build different apps and installers from the single code base.
Right now, all of these files contain a large block of "code" used to generate the email body and JIRA update messages. examples:
// Get a JIRA's to add Comments to
// Return map of JIRA id to comment text from all commits for that JIRA
#NonCPS
def getJiraMap() {
a bunch of stuff
return jiraset
}
// Get the body text for the emails
def getMailBody1() {
return "See: ${BUILD_URL}\n\nChanges:\n" + getChangeString() + "\n" + testStatuses()
}
etc...
What I would like to do is have all these common methods in a separate file that all the other pipeline files can include. This seems like it SHOULD be easy, but all examples I've found appear to be rather complex involving a separate SCM - which is NOT what I want.
Updates:
Going through the various suggestions given in that link, I make the following file - BuildTools.groovy: Note that this file is in the same directory as the jenkins pipeline file that uses it.
import hudson.tasks.test.AbstractTestResultAction
import hudson.model.Actionable
Class BuildTools {
// Get a JIRA's to add Comments to
// Return map of JIRA id to comment text from all commits for that JIRA
#NonCPS
def getJiraMap() {
def jiraset = [:]
.. whole bunch of stuff ..
Here are the various things I've tried, and the results.
File sourceFile = new File("./AutomatedBuild/BuildTools.groovy");
Class gcl = new GroovyClassLoader(getClass().getClassLoader()).parseClass(sourceFile);
GroovyObject bt = (GroovyObject) gcl.newInstance();
Fails with:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method java.lang.Class getClassLoader
evaluate(new File("./AutomatedBuild/BuildTools.groovy"))
def bt = new BuildTools()
Fails with:
15:29:07 WorkflowScript: 8: unable to resolve class BuildTools
15:29:07 # line 8, column 10.
15:29:07 def bt = new BuildTools()
15:29:07 ^
import BuildTools
def bt = new BuildTools()
Fails with:
15:35:58 WorkflowScript: 16: unable to resolve class BuildTools (note that BuildTools.groovy is in the same folder as this script)
15:35:58 # line 16, column 1.
15:35:58 import BuildTools
15:35:58 ^
GroovyShell shell = new GroovyShell()
def bt = shell.parse(new File("./AutomatedBuild/BuildTools.groovy"))
Fails with:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new groovy.lang.GroovyShell
I have checked following questions but none of them helped -
Gradle + TestNG Only Running Specified Group
Gradle command syntax for executing TESTNG tests as a group
The project I am using is available at - https://github.com/tarun3kumar/gradle-demo
It is standard maven project and I am not using testng.xml file.
Test method - com.org.corpsite.LandingPageTest is grouped as - smoke
I am running test as - gradle clean test and test is executed. Test fails due to genuine reason and let's ignore it.
Then I passed test group from command line as -
gradle clean test -P testGroups='doesnotexist'
Notice that 'doesnotexist' is not a valid group but it still executes test.
Following this I added includeGroups in build.gradle as -
test {
useTestNG() {
includeGroups 'smoke'
}
}
and now gradle clean test -P testGroups='doesnotexist' fails with NPE on one of the java class - java.lang.NullPointerException
at com.org.pageobjects.BasePage.findElements(BasePage.java:24)
Questions -
What is right flag to specify test group from command line? Seems -P is wrong else gradle clean test -P testGroups='doesnotexist' would not execute test.
What is wrong with specifying includeGroups 'smoke'?
I am using Gradle 5.1 on macbook pro
Here are the set of things that need to be done to get this to work.
You need to add the attribute alwaysRun=true to your #BeforeMethod and #AfterMethod annotations from your base class com.org.core.SelTestCase. This is to ensure that TestNG executes these configuration methods all the time irrespective of what group is chosen.
Alter the test task in your build.gradle to look like below:
test {
def groups = System.getProperty('groups', 'smoke')
useTestNG() {
includeGroups groups
}
}
This ensures that we try to extract the JVM argument groups value. If its not specified we default to smoke.
We now execute the tests by specifying the groups needed using the below command:
./gradlew clean test --info -Dgroups=smoke
Now if we execute the below command, you would notice that no tests are executed.
./gradlew clean test --info -Dgroups=smoke1
Here's a patch that you can apply to your project
From 25133a5d2a0f96d4a305f34e1f5a17e70be2bb54 Mon Sep 17 00:00:00 2001
From: Krishnan Mahadevan <krishnan.mahadevan#stackoverflow.com>
Date: Mon, 14 Jan 2019 22:38:27 +0530
Subject: [PATCH] Fixing the bug
---
build.gradle | 2 ++
src/main/java/com/org/core/SelTestCase.java | 5 +++--
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/build.gradle b/build.gradle
index 10ba91d..2d08991 100644
--- a/build.gradle
+++ b/build.gradle
## -38,7 +38,9 ## task smokeTests(type: Test) {
}*/
test {
+ def groups = System.getProperty('groups', 'smoke')
useTestNG() {
+ includeGroups groups
}
}
diff --git a/src/main/java/com/org/core/SelTestCase.java b/src/main/java/com/org/core/SelTestCase.java
index 80cad09..651529a 100644
--- a/src/main/java/com/org/core/SelTestCase.java
+++ b/src/main/java/com/org/core/SelTestCase.java
## -22,7 +22,7 ## public class SelTestCase {
private WebDriver webDriver;
- #BeforeMethod
+ #BeforeMethod(alwaysRun = true)
#Parameters({"browser", "url"})
public void setUp(#Optional("firefox") String browser, #Optional("https://www.google.com/") String URL) {
switch (browser) {
## -40,8 +40,9 ## public class SelTestCase {
webDriver.get(URL);
}
- #AfterMethod
+ #AfterMethod(alwaysRun = true)
public void tearDown() {
webDriver.quit();
}
+
}
--
2.20.1
You can save the above contents to a file say mypatch.patch and then apply the patch using the instructions detailed in this StackOverFlow post.
You should be able to run a specific test with the 'testInstrumentationRunnerArguments' flag:
-Pandroid.testInstrumentationRunnerArguments.class=com.abc.NameOfMyTestClass
The following program is based on the example in the v8 Getting Started page. I have made three changes to demonstrate a problem I am encountering:
I create an empty array put it into the global context.
The script being run references the zeroth element in the array, which should return undefined.
I run the compiled script twice.
The first run works fine. The second fails: v8 calls V8_Fatal() in Deoptimizer::DoComputeCompiledStubFrame() because descriptor->register_param_count_ == -1.
Am I doing something wrong here? How can I fix it?
Isolate* isolate = Isolate::New();
Isolate::Scope isolate_scope(isolate);
HandleScope handle_scope(isolate);
Local<Context> context = Context::New(isolate);
Context::Scope context_scope(context);
Local<Array> a = Array::New(isolate);
context->Global()->Set(String::NewFromUtf8(isolate, "a"), a);
Local<String> source = String::NewFromUtf8(isolate, "a[0];");
Local<Script> script = Script::Compile(source);
Local<Value> result = script->Run();
Local<Value> result2 = script->Run();
return 0;
NOTES:
This is the entire body of main().
Other fragments of JavaScript code run twice without a problem. Somehow this relates to the out-of-bound array reference, which is perhaps triggering deoptimization.
I do not want to recompile the script from scratch each time because I am typically running these scripts thousands of times, and sometimes millions of times.
I have tried compiling the script as an UnboundScript and then binding it for each execution, but the result is the same.
I have reported this as a v8 issue, but nobody has responded so I'm hoping that the StackOverflow community can help.
I am seeing this on VS2012 Update 4, but I also see it on VS2008, and in both x64 and x86 and in both Debug and Release builds.
OK, found it. The problem is an uninitialized code stub for dictionary loads - your use case triggers this as a failure as the stub isn't initialized through other means, eg compilation.
Below is a patch against v8 trunk revision 22629 that fixes the problem for me, tested on Windows with VS 2010 and Linux with g++ 4.9. Please let me know how you go with this:
Index: src/code-stubs.cc
===================================================================
--- src/code-stubs.cc (revision 22629)
+++ src/code-stubs.cc (working copy)
## -236,6 +236,8 ##
CODE_STUB_LIST(DEF_CASE)
#undef DEF_CASE
case UninitializedMajorKey: return "<UninitializedMajorKey>Stub";
+ case NoCache:
+ return "<NoCache>Stub";
default:
if (!allow_unknown_keys) {
UNREACHABLE();
## -939,6 +941,13 ##
// static
+void KeyedLoadDictionaryElementStub::InstallDescriptors(Isolate* isolate) {
+ KeyedLoadDictionaryElementStub stub(isolate);
+ InstallDescriptor(isolate, &stub);
+}
+
+
+// static
void KeyedLoadGenericElementStub::InstallDescriptors(Isolate* isolate) {
KeyedLoadGenericElementStub stub(isolate);
InstallDescriptor(isolate, &stub);
Index: src/code-stubs.h
===================================================================
--- src/code-stubs.h (revision 22629)
+++ src/code-stubs.h (working copy)
## -1862,6 +1862,8 ##
virtual void InitializeInterfaceDescriptor(
CodeStubInterfaceDescriptor* descriptor) V8_OVERRIDE;
+ static void InstallDescriptors(Isolate* isolate);
+
private:
Major MajorKey() const { return KeyedLoadElement; }
int NotMissMinorKey() const { return DICTIONARY_ELEMENTS; }
Index: src/isolate.cc
===================================================================
--- src/isolate.cc (revision 22629)
+++ src/isolate.cc (working copy)
## -2000,6 +2000,7 ##
NumberToStringStub::InstallDescriptors(this);
StringAddStub::InstallDescriptors(this);
RegExpConstructResultStub::InstallDescriptors(this);
+ KeyedLoadDictionaryElementStub::InstallDescriptors(this);
KeyedLoadGenericElementStub::InstallDescriptors(this);
}
As a workaround if you don't want to compile your own V8 for now, you could execute some code on each Isolate that uses the KeyedLoadDictionaryElementStub directly, prior to running your code --- this should initialize the stub. Something like the following works for me:
Isolate* isolate = Isolate::New();
Isolate::Scope isolate_scope(isolate);
HandleScope handle_scope(isolate);
Local<Context> context = Context::New(isolate);
Context::Scope context_scope(context);
Local<Array> a = Array::New(isolate);
context->Global()->Set(String::NewFromUtf8(isolate, "a"), a);
// Workaround code for initializing KeyedLoadDictionaryElementStub
Local<String> workaround_source = String::NewFromUtf8(isolate, "Math.random()");
Local<Script> workaround_script = Script::Compile(workaround_source);
Local<Value> workaround_value = workaround_script->Run();
// End workaround
Local<String> source = String::NewFromUtf8(isolate, "a[0]");
Local<Script> script = Script::Compile(source);
// ...and so on
Hadoop itself (i.e hdfs and map/reduce) is working - and brew-installed hive as well (which validates the first two). Just the GUI is weird: see the attached screenshot: the webserver is up but the JobTracker is not providing any useful info
jobtracker.jsp not found:
Anyone home at all on port 50030 (yes!)
What about namenode ui?
I was getting the same error on hadoop-2.0.0-cdh4.5.0 installation. After spending some time looking into the code, I managed to get this working by applying following patch to hadoop-common-2.0.0-cdh4.5.0.jar:
diff --git a/org/apache/hadoop/http/HttpServer.java b/org/apache/hadoop/http/HttpServer.java
index 25b4c75..1896623 100644
--- a/org/apache/hadoop/http/HttpServer.java
+++ b/org/apache/hadoop/http/HttpServer.java
## -237,6 +237,10 ## public class HttpServer implements FilterContainer {
webAppContext.setDisplayName(name);
webAppContext.setContextPath("/");
webAppContext.setWar(appDir + "/" + name);
+
+ webAppContext.setDescriptor(appDir + "/" + name + "/WEB-INF/web.xml");
+ webAppContext.setResourceBase(appDir + "/" + name);
+
webAppContext.getServletContext().setAttribute(CONF_CONTEXT_ATTRIBUTE, conf);
webAppContext.getServletContext().setAttribute(ADMINS_ACL, adminsAcl);
addNoCacheFilter(webAppContext);
basically the jobtracker/tasktracker webapp initialization using Jetty WebAppContext was missing the web.xml location, adding it there using setDescriptor() and setting resourceBase correctly using setResourceBase() solves the problem.
H2 database with custom function alias defined as:
create alias to_date as $$
java.util.Date toDate(java.lang.String dateString, java.lang.String pattern) {
try {
return new java.text.SimpleDateFormat(javaPattern).parse(dateString);
} catch(java.text.ParseException e) {
throw new java.lang.RuntimeException(e);
}
}
$$;
H2 initialized as:
jdbc:h2:mem:testdb;INIT=runscript from 'classpath:create_alias.sql
This is used in tests, executed for multiple projects concurrently on a Jenkins instance. Sometimes such tests would fail with following error:
Could not get JDBC Connection; nested exception is org.h2.jdbc.JdbcSQLException: Syntax error in SQL statement "javac: file not found: org/h2/dynamic/TO_DATE.java
Usage: javac <options> <source files>
use -help for a list of possible options
"; SQL statement:
create alias to_date as $$
java.util.Date toDate(java.lang.String dateString, java.lang.String pattern) {
....
My guess is that org.h2.util.SourceCompiler is assuming that there is only one instance of h2 running at the time and writes the generated Java source to 'java.io.tmpdir', which is shared among all processes running under same account. I propose following fix:
Index: SourceCompiler.java
===================================================================
--- SourceCompiler.java (revision 5086)
+++ SourceCompiler.java (working copy)
## -40,7 +40,15 ##
*/
final HashMap<String, Class<?>> compiled = New.hashMap();
- private final String compileDir = Utils.getProperty("java.io.tmpdir", ".");
+ private final String compileDir;
+
+ {
+ // use random folder under java.io.tmpdir so multiple h2 could compile at the same time
+ // without overwriting each other files
+ File tmp = File.createTempFile("h2tmp", ".tmp");
+ tmp.mkdir();
+ compileDir = tmp.getAbsolutePath();
+ }
static {
Class<?> clazz;
Should I open the support ticket or there are workarounds for this issue?
You can use javax.tools.JavaCompiler API and provide your own implementation for in-memory JavaFileManager to completely avoid creating those temp files.
BTW, Janino also support javax.tools.JavaCompiler API.
I had the same problem running multiple Jenkins executors and having Arquillian/Wildfly/H2 integration tests configuration. I found a workaround by setting java.io.tmpdir property to the build directory in the test standalone.xml.