I have created a domain class named Curr_Stud_Info.groovy
package ads2
import gorm.logical.delete.LogicalDelete
class Curr_Stud_Info implements LogicalDelete{
static constraints = {
current_std nullable: false
division nullable: false
roll_no nullable: false
}
static belongsTo = [student : Student]
static mapping = {student column: id} //this is needed only in belongs to methods for giving relationship to different column
Integer current_std
Character division
Integer roll_no
}
during building this project i am getting error in it's specs file Curr_Stud_InfoSpec.groovy which is
package ads2
import grails.testing.gorm.DomainUnitTest
import spock.lang.Specification
class Curr_Stud_InfoSpec extends Specification implements DomainUnitTest<Curr_Stud_Info> {
def setup() {
}
def cleanup() {
}
void "test something"() {
expect:"fix me"
true == false
}
}
which is grails generated.
error is
Error:Groovyc: While compiling tests of ads2.ads2.test: BUG! exception
in phase 'instruction selection' in source unit
'/home/pathpanchal/PROJECT/ads2/ads2/src/test/groovy/ads2/Curr_Stud_InfoSpec.groovy'
Type is null. Most probably you let a transform reuse existing
ClassNodes with generics information, that is now used in a wrong
context. at
org.codehaus.groovy.vmplugin.v5.Java5.configureType(Java5.java:125)
at
org.codehaus.groovy.vmplugin.v5.Java5.configureTypeArguments(Java5.java:186)
at
org.codehaus.groovy.vmplugin.v5.Java5.configureParameterizedType(Java5.java:166)
at
org.codehaus.groovy.vmplugin.v5.Java5.configureType(Java5.java:117)
at
org.codehaus.groovy.vmplugin.v5.Java5.makeClassNode(Java5.java:439)
at
org.codehaus.groovy.vmplugin.v5.Java5.configureClassNode(Java5.java:370)
at
org.codehaus.groovy.ast.ClassNode.lazyClassInit(ClassNode.java:280)
at
org.codehaus.groovy.ast.ClassNode.getInterfaces(ClassNode.java:375)
at
org.codehaus.groovy.ast.ClassNode.declaresInterface(ClassNode.java:981)
at
org.codehaus.groovy.ast.ClassNode.implementsInterface(ClassNode.java:961)
at
org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.getResultType(StaticTypeCheckingVisitor.java:3717)
at
org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.visitBinaryExpression(StaticTypeCheckingVisitor.java:716)
at
org.codehaus.groovy.ast.expr.BinaryExpression.visit(BinaryExpression.java:51)
at
org.codehaus.groovy.ast.CodeVisitorSupport.visitExpressionStatement(CodeVisitorSupport.java:122)
at
org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitExpressionStatement(ClassCodeVisitorSupport.java:196)
at
org.codehaus.groovy.ast.stmt.ExpressionStatement.visit(ExpressionStatement.java:42)
at
org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitClassCodeContainer(ClassCodeVisitorSupport.java:104)
at
org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitConstructorOrMethod(ClassCodeVisitorSupport.java:115)
at
org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.visitConstructorOrMethod(StaticTypeCheckingVisitor.java:1963)
at
org.codehaus.groovy.ast.ClassCodeVisitorSupport.visitMethod(ClassCodeVisitorSupport.java:126)
at
org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.startMethodInference(StaticTypeCheckingVisitor.java:2318)
at
org.codehaus.groovy.transform.stc.StaticTypeCheckingVisitor.visitMethod(StaticTypeCheckingVisitor.java:2277)
at
org.codehaus.groovy.transform.sc.StaticCompilationVisitor.visitMethod(StaticCompilationVisitor.java:224)
at
org.codehaus.groovy.transform.sc.StaticCompileTransformation.visit(StaticCompileTransformation.java:80)
at
org.codehaus.groovy.transform.ASTTransformationVisitor.visitClass(ASTTransformationVisitor.java:152)
at
org.codehaus.groovy.transform.ASTTransformationVisitor$2.call(ASTTransformationVisitor.java:196)
at
org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1087)
at
org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
at
org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
at
org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
at
org.jetbrains.groovy.compiler.rt.GroovyCompilerWrapper.compile(GroovyCompilerWrapper.java:62)
at
org.jetbrains.groovy.compiler.rt.DependentGroovycRunner.runGroovyc(DependentGroovycRunner.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.jetbrains.groovy.compiler.rt.GroovycRunner.intMain2(GroovycRunner.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.jetbrains.jps.incremental.groovy.InProcessGroovyc.runGroovycInThisProcess(InProcessGroovyc.java:160)
at
org.jetbrains.jps.incremental.groovy.InProcessGroovyc.lambda$runGroovyc$0(InProcessGroovyc.java:89)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
i dont know why this error is occuring and how to solve it ??pls help
Related
I have one zip that contains 2 JSON files. When I process those files I got the following error. The first file is processed properly when it comes to the second file which throws ZipException.
How can I fix this?
My code,
private Path archivePath = Paths.get("src/test/resources/data/files.zip";
public void uploadData() throws IOException {
FileSystem fileFS = FileSystems.newFileSystem(archivePath,(ClassLoader) null);
String[] pathNames = {"file_1.json","file_2.json"};
for(String pathName : pathNames) {
//unzip files
Path path = fileFS.getPath(pathName);
Request request = new Request("POST","/_bulk");
RequestOptions.Builder options = RequestOptions.DEFAULT.toBuilder();
options.addHeader("Content-Type","application/x-ndjson");
request.setOptions(options);
request.setEntity(new InputStreamEntity(Files.newInputStream(path)));
Response response = client.performRequest(request);
assertResponseSuccessful(response,false);
}
Error:
invalid stored block lengths
java.io.IOException: invalid stored block lengths
at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:828)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:248)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:134)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:597)
at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:173)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:816)
at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:146)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.testng.TestRunner.privateRun(TestRunner.java:766)
at org.testng.TestRunner.run(TestRunner.java:587)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:384)
at org.testng.SuiteRunner.access$000(SuiteRunner.java:28)
at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:425)
at org.testng.internal.thread.ThreadUtil.lambda$execute$0(ThreadUtil.java:68)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.zip.ZipException: invalid stored block lengths
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:165)
at java.base/java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:388)
at org.apache.http.nio.entity.EntityAsyncContentProducer.produceContent(EntityAsyncContentProducer.java:67)
at org.apache.http.nio.protocol.BasicAsyncRequestProducer.produceContent(BasicAsyncRequestProducer.java:125)
at org.apache.http.impl.nio.client.MainClientExec.produceContent(MainClientExec.java:262)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.produceContent(DefaultClientExchangeHandlerImpl.java:140)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.outputReady(HttpAsyncRequestExecutor.java:249)
at org.apache.http.impl.nio.client.InternalRequestExecutor.outputReady(InternalRequestExecutor.java:96)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.produceOutput(DefaultNHttpClientConnection.java:290)
at org.apache.http.impl.nio.client.InternalIODispatch.onOutputReady(InternalIODispatch.java:86)
at org.apache.http.impl.nio.client.InternalIODispatch.onOutputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.outputReady(AbstractIODispatch.java:145)
at org.apache.http.impl.nio.reactor.BaseIOReactor.writable(BaseIOReactor.java:187)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:341)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
... 1 more
Any inputs here are really be appreciated.
I am trying to setup a multi branch pipeline between Github and Jenkins. I am working on a spring boot project and I have setup my local jdk and maven within Jenkins.
I am running the pipeline using jenkinsfile, the pipeline code is as below:
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
Jenkins is able to scan my repo successfully, however on running the pipeline I am getting below error which I am clueless about. Could you please help me in breaking this barrier I am stuck at this.
16:35:38 Connecting to https://api.github.com using shred22/****** (Git Hub Credentials)
Obtained Jenkinsfile from ab4673fa1cc076e4ce773fbc2be9c3756aac0bc8
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] End of Pipeline
GitHub has been notified of this commit’s build result
hudson.remoting.ProxyException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
jar:file:/var/lib/jenkins/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy: 43: unable to resolve class javax.annotation.Nonnull
# line 43, column 1.
import javax.annotation.Nonnull
^
jar:file:/var/lib/jenkins/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy: 42: unable to resolve class javax.annotation.CheckForNull
# line 42, column 1.
import javax.annotation.CheckForNull
^
jar:file:/var/lib/jenkins/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy: 455: unable to resolve class javax.annotation.CheckForNull , unable to find class for annotation
# line 455, column 30.
def withCredentialsBlock(#CheckForNull Environment environment, Closure body) {
^
jar:file:/var/lib/jenkins/plugins/pipeline-model-definition/WEB-INF/lib/pipeline-model-definition.jar!/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy: 493: unable to resolve class javax.annotation.Nonnull , unable to find class for annotation
# line 493, column 13.
#Nonnull Map<String, CredentialWrapper> credentials) {
^
4 errors
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:958)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:605)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:554)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:250)
at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:766)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:575)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$TimingLoader.loadClass(CpsGroovyShell.java:170)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:575)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:775)
at org.jenkinsci.plugins.pipeline.modeldefinition.ModelStepLoader.getValue(ModelStepLoader.java:60)
at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:113)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)
at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:142)
at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
at WorkflowScript.run(WorkflowScript:1)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.ClosureBlock.eval(ClosureBlock.java:46)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Finished: FAILURE
Please check the upstream issue.
I worked around by downgrading Pipeline: Groovy package to v2.80 from v2.81. (However, the PR is ready in upstream so I believe v2.82 will be released soon.)
https://issues.jenkins-ci.org/browse/JENKINS-62988
https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/397
I have worked around the problem by removing the two imports and the annotations from the coded in ModelInterpreter.groovy in the jar file and replacing the jar file. That makes it work, since the annotations are optional it will probably not cause problems.
Our declarative pipeline(for isolating the problem) is using a custom step of our shared library. This custom step shall represent a long running operation by just calling Thread.sleep (360000).
Code:
pipeline
{
agent
{
label'master'
}
options
{
timestamps()
buildDiscarder(logRotator(numToKeepStr: '10'))
skipDefaultCheckout()
}
stages
{
stage('Thread.sleep')
{
steps{
sleepService(360000)
}
}
}
}
Custom step from shared library:
import java.lang.*;
def String call(long millis) {
timeout(600)
{
echo "Current thread name: ${Thread.currentThread().getName()}"
echo "Current thread classname: ${Thread.currentThread().getClass().getName()}"
Thread.sleep (millis)
}
}
Executing this, is causing the following exception:
java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at java_lang_Thread$sleep.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)
at soiServiceSleep.call(soiServiceSleep.groovy:9)
at ___cps.transform___(Native Method)
at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
at sun.reflect.GeneratedMethodAccessor577.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39)
at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)
at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28)
at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)
at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)
at com.cloudbees.groovy.cps.Next.step(Next.java:83)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:405)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:317)
at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:281)
at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
It seems like this is caused by timeout of 5 minutes of the CPSThread.
How can we overcome this problem?
Best regards,
Thomas
** fisrt. My project is about the list of articles recommended,every article have there own rule,so i use AsyncTaskExecutor this tool class to concurrent query different articles, and now,some rules is special,So I split them into different rules into two parts. next is my code:
i use springboot + mybatis to do**
#Bean
public AsyncTaskExecutor dataTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(16);
executor.setThreadNamePrefix("data_task_executor-");
return executor;
}
here i initialization AsyncTaskExecutor class for ready
Next is the partial code of the concurrent query.
// here i get different rule list
List<Rule> ruleList = JSON.parseArray(scene.getRules(), Rule.class);
Iterator<Rule> ruleIterator = ruleList.iterator();
CountDownLatch latch1 = new CountDownLatch(ruleList.size());
while (ruleIterator.hasNext()) {
Rule ruleNext = ruleIterator.next();
// unAsyncScenes is a array,this rule query in here
if (Arrays.binarySearch(unAsyncScenes, ruleNext.getSource()) >= 0) {
dataTaskExecutor.execute(() -> {
try {
searchIDSByRule(idWithRtsMap, articleReferralList, sceneId, feedSum, userId, isNewUserByHistory, discussHistoryList, discussList, graphHistorys, ruleNext);
//Record browsing history
graphHistorys.addAll(idWithRtsMap.keySet());
} catch (Exception e) {
log.warn("子规则查图失败", e);
} finally {
latch1.countDown();
}
});
//Query deleted
ruleIterator.remove();
} else {
latch1.countDown();
}
}
try {
latch1.await(10, TimeUnit.SECONDS);
} catch (InterruptedException e) {
log.error("多线等待异常:", e);
}
//deal with Duplicate article
Set<Long> articleSet = new HashSet();
articleReferralList.forEach(article -> articleSet.add(article));
if (articleReferralList.size() != articleSet.size()) {
log.warn("出现了重复的文章");
articleReferralList.clear();
articleReferralList.addAll(articleSet);
}
final CountDownLatch latch = new CountDownLatch(ruleList.size());
for (Rule rule : ruleList) {
// second concurrent query(query for other article)
dataTaskExecutor.execute(() -> {
try { *****// here hava error!!!!!!!!!!!!*****
searchIDSByRule(idWithRtsMap, articleReferralList, sceneId, feedSum, userId, isNewUserByHistory, discussHistoryList, discussList, graphHistorys, rule);
} catch (Exception e) {
log.warn("子规则查图失败", e);
} finally {
latch.countDown();
}
});
}
try {
latch.await(10, TimeUnit.SECONDS);
} catch (InterruptedException e) {
log.error("多线等待异常:", e);
}
this is all query code, but went i run this code, it sometimes gives an error like this:
org.apache.ibatis.exceptions.PersistenceException: \
### Error querying database. Cause: java.util.ConcurrentModificationException\
### Cause: java.util.ConcurrentModificationException\
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:77) ~[mybatis-spring-1.3.1.jar!\/:1.3.1]\
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:446) ~[mybatis-spring-1.3.1.jar!\/:1.3.1]\
at com.sun.proxy.$Proxy91.selectList(Unknown Source) ~[?:?]\
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:230) ~[mybatis-spring-1.3.1.jar!\/:1.3.1]\
at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:137) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:75) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:59) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at com.sun.proxy.$Proxy131.searchBySigAndExample(Unknown Source) ~[?:?]\
at com.coffee.ref.service.impl.ReferralServiceImpl.searchIDSByRule(ReferralServiceImpl.java:842) ~[classes!\/:0.0.1]\
at com.coffee.ref.service.impl.ReferralServiceImpl.lambda$findArticleIDSByRule$7(ReferralServiceImpl.java:625) ~[classes!\/:0.0.1]\
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]\
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]\
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]\
Caused by: org.apache.ibatis.exceptions.PersistenceException: \
### Error querying database. Cause: java.util.ConcurrentModificationException\
### Cause: java.util.ConcurrentModificationException\
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:150) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at sun.reflect.GeneratedMethodAccessor143.invoke(Unknown Source) ~[?:?]\
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]\
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]\
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:433) ~[mybatis-spring-1.3.1.jar!\/:1.3.1]\
... 11 more\
Caused by: java.util.ConcurrentModificationException\
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909) ~[?:1.8.0_212]\
at java.util.ArrayList$Itr.next(ArrayList.java:859) ~[?:1.8.0_212]\
at org.apache.ibatis.scripting.xmltags.ForEachSqlNode.apply(ForEachSqlNode.java:62) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.scripting.xmltags.MixedSqlNode.apply(MixedSqlNode.java:33) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.scripting.xmltags.IfSqlNode.apply(IfSqlNode.java:35) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.scripting.xmltags.MixedSqlNode.apply(MixedSqlNode.java:33) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.scripting.xmltags.DynamicSqlSource.getBoundSql(DynamicSqlSource.java:41) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.mapping.MappedStatement.getBoundSql(MappedStatement.java:292) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at com.github.pagehelper.PageInterceptor.intercept(PageInterceptor.java:83) ~[pagehelper-5.1.2.jar!\/:?]\
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:61) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at com.sun.proxy.$Proxy188.query(Unknown Source) ~[?:?]\
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141) ~[mybatis-3.4.5.jar!\/:3.4.5]\
at sun.reflect.GeneratedMethodAccessor143.invoke(Unknown Source) ~[?:?]\
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_212]\
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_212]\
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:433) ~[mybatis-spring-1.3.1.jar!\/:1.3.1]\
... 11 more\"}"]
The place where the error was reported is marked above.I don't understand why, mybatis should be thread safe.
enter image description here
The official documentation says it is thread safe.
SqlSessionTemplate itself is thread safe. The problem is in your code.
The exception shows that error happens in the foreach element. Note this piece of the stacktrace:
Caused by: java.util.ConcurrentModificationException\
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909) ~[?:1.8.0_212]\
at java.util.ArrayList$Itr.next(ArrayList.java:859) ~[?:1.8.0_212]\
at org.apache.ibatis.scripting.xmltags.ForEachSqlNode.apply(ForEachSqlNode.java:62) ~[mybatis-3.4.5.jar!\/:3.4.5]\
So what happens here? In the mapper you build SQL dynamically by iterating over some collection. This collection is modified concurrently by another thread. The iterator over the collection has built-in check that collection is not modified and this check indicates you that there is a problem.
In order to fix this you need to synchronize access to the collection that is used in multiple threads so that when you use the collection to query some data based on it this happens atomically and no modification can happen in the middle of the query generation.
One possible reason of this is that the result of this await is not analyzed:
latch1.await(10, TimeUnit.SECONDS);
If the processing takes more than 10 seconds the second part starts executing while the data that query is based on is still being modified. This can happen as the amount of work to do depends on the data.
You need to check the result of this await and do not continue processing until all tasks in the first part of the procedure are finished.
This is my first post! I've been searching for solutions to this to no avail for weeks (on and off)...
I have a java Hive UDF where I want to run SQL against Hive table to map data in memory for later use. I have the connection information correct, but it cannot find the HiveDriver. I have tried adding the Hive jdbc driver jar in hive and exporting it into a runnable jar.
Error below, relevant code after.
hive> add jar /tmp/xhUDFs.jar;
Added [/tmp/xhUDFs.jar] to class path
Added resources: [/tmp/xhUDFs.jar]
hive> CREATE TEMPORARY FUNCTION getRefdata as 'xhUDFs.GetRefdata';
OK
Time taken: 0.042 seconds
hive> select '06001',getRefdata('CBSAname','06001');
java.lang.ClassNotFoundException:
org.apache.hadoop.hive.jdbc.HiveDriver
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at xhUDFs.GetRefdata.getRefmap(GetRefdata.java:53)
at xhUDFs.GetRefdata.evaluate(GetRefdata.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:954)
at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:182)
at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:168)
at org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:233)
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:1093)
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1312)
at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:95)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:79)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:133)
at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:110)
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:209)
at org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:153)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:10499)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:10455)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3822)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3601)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8943)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8898)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9743)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9636)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10109)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:329)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10120)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:211)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:454)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:314)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1164)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1212)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1101)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1091)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136) java.lang.ClassNotFoundException:
org.apache.hadoop.hive.jdbc.HiveDriver
Now the code snippet:
try {
Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver");
Connection wConn = DriverManager.getConnection("jdbc:hive2://172.20.2.40:10000/refdata", "", "");
Statement sql = wConn.createStatement();
ResultSet wResult = sql.executeQuery("select distinct zipcode5,cbsa_name,msa_name,pmsa_name,region,csaname,cbsa_div_name from refdata.zip_code_business_all_16_1_2016_orc where primaryrecord='P'");
while (wResult.next()){
censusMap.put(wResult.getString("zipcode5"), wResult.getString("cbsa_name"));
}
} catch (SQLException | ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}