I have a script in pentaho where it gets values like process id and few variables from the successful execution of previous job and write it as a file name into another location based on the process id and variable. While the shell script is being executed, it is throwing below run time error only few times. Please help.
ERROR 21-07 03:27:26,604 - Shell_Create_Trigger_File - (stderr) java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
at java.io.BufferedInputStream.read(BufferedInputStream.java:325)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:154)
at java.io.BufferedReader.readLine(BufferedReader.java:317)
at java.io.BufferedReader.readLine(BufferedReader.java:382)
at org.pentaho.di.core.util.StreamLogger.run(StreamLogger.java:57)
at java.lang.Thread.run(Thread.java:745)
"(stderr) java.io.IOException: Stream closed" usually happens if there is an abrupt closing of the connection or data connection is getting lost. You can check this question.
Also there could be multiple reasons like slow network, pdi process getting killed in the middle. But answer to your question can be broad. As you said the error is happening only few times, i suggest you look into the server you are running the code.
Hope it helps :)
Related
I am trying to cleanly start Kafka 2.10 - 0.8.2.1 on Windows but I am getting an annoying error everytime I start it.
I have just installed Kafka by following the Quick Start guide (with the exception that I have installed Zookeeper myself). Both Kafka and Zookeeper were installed very basic, on a single machine.
Problem
When I run the starting script:
kafka-server-start.bat C:\kafka_2.10-0.8.2.1\config\server.properties
I get the error:
Error
[2015-07-14 17:00:45,197] WARN Error when freeing index buffer (kafka.log.OffsetIndex)
java.lang.NullPointerException
at kafka.log.OffsetIndex.kafka$log$OffsetIndex$$forceUnmap(OffsetIndex.scala:301)
at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:283)
at kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:276)
at kafka.utils.Utils$.inLock(Utils.scala:535)
at kafka.log.OffsetIndex.resize(OffsetIndex.scala:276)
at kafka.log.Log.loadSegments(Log.scala:179)
at kafka.log.Log.<init>(Log.scala:67)
at kafka.log.LogManager$$anonfun$loadLogs$2$$anonfun$3$$anonfun$apply$7$$anonfun$apply$1.apply$mcV
at kafka.utils.Utils$$anon$1.run(Utils.scala:54)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2015-07-14 17:00:45,219] INFO Completed load of log test-0 with log end offset 0 (kafka.log.Log)
What I observed
When I delete the Kafka log folder, the error does not appear the first time I re-run the starting script. Kafka log folder path:
C:\tmp\kafka-logs
I have tried to stop the service using the provided script, but it does not help. The server stop script:
kafka-server-stop.bat
Although the same error appears when I start it the second time, Kafka start-up continues, and it seems that it works normally.
Help
How to get rid of the above error?
This exception shouldn't be a problem during startup but if you want to get rid of it I guess you have to file a Jira for it.
This seems to be related to KAFKA-1008 and this commit.
/**
* Forcefully free the buffer's mmap. We do this only on windows.
*/
private def forceUnmap(m: MappedByteBuffer) {
try {
if(m.isInstanceOf[sun.nio.ch.DirectBuffer])
(m.asInstanceOf[sun.nio.ch.DirectBuffer]).cleaner().clean()
} catch {
case t: Throwable => warn("Error when freeing index buffer", t)
}
}
when index file size == 0 , cleaner where be null
see
https://github.com/apache/kafka/pull/1718
I am getting following error while running following pig script
REGISTER /opt/cloudera/parcels/CDH/lib/pig/lib/avro.jar
REGISTER /opt/cloudera/parcels/CDH/lib/pig/lib/json-simple-1.1.jar
REGISTER /opt/cloudera/parcels/CDH/lib/pig/lib/jackson-core-asl-1.8.8.jar
REGISTER /opt/cloudera/parcels/CDH/lib/pig/lib/jackson-mapper-asl-1.8.8.jar
REGISTER /opt/cloudera/parcels/CDH/lib/pig/piggybank.jar
list_cookies = LOAD '/user/xyz/testbed/llama-2014-Oct-12d/abc'
USING org.apache.pig.piggybank.storage.avro.AvroStorage();
got following error
2014-10-22 11:51:14,705 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2245: Cannot get schema from loadFunc org.apache.pig.builtin.AvroStorage
Details at logfile: /home/xyz/pig_1413991623605.log
In my case, it was simply the fact that the input folder did not exist. Pig error messages are off the mark and not at all helpful. After changing the input folder to one that existed, this error went away. So, be sure to check that before spending a lot of time more difficult debugging!
After upgrading to OLE 1.5.2.1 when I try to upload a local MARC .mrc-file via batch process I get the following error:
Batch process Failed for profile :: Test_Bib_Import
The same exact file worked fine in OLE 1.5.0-M2
Catalina.out contains the following error:
[INFO] org.kuali.ole.batch.impl.OLEBatchProcessAdhocStep - Executing Batch process type :: Bib Import
[ERROR] org.kuali.ole.batch.ingest.BatchProcessBibImport - java.lang.NullPointerException
[ERROR] org.kuali.ole.batch.helper.OLEBatchProcessDataHelper - Error while performing batch process for profile :: Test_Bib_Import
java.lang.RuntimeException: java.lang.NullPointerException
at org.kuali.ole.batch.ingest.BatchProcessBibImport.processBatch(BatchProcessBibImport.java:90)
at org.kuali.ole.batch.impl.AbstractBatchProcess.process(AbstractBatchProcess.java:87)
at org.kuali.ole.batch.impl.OLEBatchProcessAdhocStep.executeBatch(OLEBatchProcessAdhocStep.java:50)
at org.kuali.ole.batch.impl.OLEBatchProcessAdhocStep.execute(OLEBatchProcessAdhocStep.java:30)
at org.kuali.ole.sys.batch.Job.runStep(Job.java:175)
at org.kuali.ole.sys.batch.Job.execute(Job.java:121)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: java.lang.NullPointerException
at org.kuali.ole.batch.helper.BatchBibImportHelper.processBibMarcRecord(BatchBibImportHelper.java:89)
at org.kuali.ole.batch.helper.BatchBibImportHelper.processBatch(BatchBibImportHelper.java:70)
at org.kuali.ole.batch.ingest.BatchProcessBibImport.processBatch(BatchProcessBibImport.java:152)
at org.kuali.ole.batch.ingest.BatchProcessBibImport.processBatch(BatchProcessBibImport.java:83)
... 7 more
[ERROR] org.kuali.ole.batch.impl.OLEBatchProcessAdhocStep - Error while running Batch Process Step::OLEBatchProcessAdhocStep
java.lang.Exception: Batch process Failed
at org.kuali.ole.batch.impl.AbstractBatchProcess.process(AbstractBatchProcess.java:123)
at org.kuali.ole.batch.impl.OLEBatchProcessAdhocStep.executeBatch(OLEBatchProcessAdhocStep.java:50)
at org.kuali.ole.batch.impl.OLEBatchProcessAdhocStep.execute(OLEBatchProcessAdhocStep.java:30)
at org.kuali.ole.sys.batch.Job.runStep(Job.java:175)
at org.kuali.ole.sys.batch.Job.execute(Job.java:121)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.kuali.ole.batch.ingest.BatchProcessBibImport.processBatch(BatchProcessBibImport.java:90)
at org.kuali.ole.batch.impl.AbstractBatchProcess.process(AbstractBatchProcess.java:87)
... 6 more
Caused by: java.lang.NullPointerException
at org.kuali.ole.batch.helper.BatchBibImportHelper.processBibMarcRecord(BatchBibImportHelper.java:89)
at org.kuali.ole.batch.helper.BatchBibImportHelper.processBatch(BatchBibImportHelper.java:70)
at org.kuali.ole.batch.ingest.BatchProcessBibImport.processBatch(BatchProcessBibImport.java:152)
at org.kuali.ole.batch.ingest.BatchProcessBibImport.processBatch(BatchProcessBibImport.java:83)
... 7 more
The line from which that that error is coming from suggests that the Batch Profile you are using has not been set up correctly in the database (specifically, it can't find the matching profile). One definitive indicator would be a stack trace output in catalina.out involving the getMatchingProfileObj method in the org.kuali.ole.batch.bo.OLEBatchProcessProfileBo class.
If you migrated your application code over top of an existing database without having fully migrated the data in that database correctly, this problem could result. Given that your question illustrates you used 1.5.0-M2 previously, which is a pre-release milestone, you are better off re-initializing your database and reloading your data before running batch processes with a new version of the OLE codebase.
If you already did that, then this may be fodder for a bug report.
Note that there were a lot of changes to the match/overlay part of the profiles in 1.5.2. You might want to look at how it is set up and make sure the choices apply. Match and overlay was added for holdings and items and is being tested, this makes for more complicated possible choices. YOu could try setting it to no match, just add the bib and see if that works. As long as your file is utf8, not MARC8 encoding it shoul dload
Following an install of Eclipse 3.7.2 on my Ubuntu 12.04 development machine, I have been unable to execute any ODI packages/interfaces/procedures. On execution (for both simulated and actual runs), an error is thrown (java trace below). I am not sure if it's anything to do with the Eclipse install, but it seems likely. Does anyone have an idea how to fix this?
Also, when launching ODI from the terminal using 'bash odi', the following error is displayed in the terminal:
2013-08-15 14:43:46.162 ERROR Error during RuntimeClassLoader initialization. ODI will start without RuntimeClassLoader
Error output:
oracle.odi.core.exception.OdiRuntimeException: Error during Code Interpretor creation
at com.sunopsis.dwg.codeinterpretor.SnpCodeInterpretor.getInstance(SnpCodeInterpretor.java:209)
at com.sunopsis.dwg.codeinterpretor.SnpGeneratorSQLCIT.<init>(SnpGeneratorSQLCIT.java:300)
at com.sunopsis.graphical.dialog.SnpsDialogExecution.doPackageExecuter(SnpsDialogExecution.java:907)
at oracle.odi.ui.action.SnpsPopupActionExecuteHandler.actionPerformed(SnpsPopupActionExecuteHandler.java:68)
at oracle.odi.ui.SnpsActionControler.handleEvent(SnpsActionControler.java:75)
at oracle.ide.controller.IdeAction.performAction(IdeAction.java:529)
at oracle.ide.controller.IdeAction.actionPerformedImpl(IdeAction.java:884)
at oracle.ide.controller.IdeAction.actionPerformed(IdeAction.java:501)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1995)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2318)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.AbstractButton.doClick(AbstractButton.java:357)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(BasicMenuItemUI.java:809)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(BasicMenuItemUI.java:850)
at java.awt.Component.processMouseEvent(Component.java:6297)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3275)
at java.awt.Component.processEvent(Component.java:6062)
at java.awt.Container.processEvent(Container.java:2039)
at java.awt.Component.dispatchEventImpl(Component.java:4660)
at java.awt.Container.dispatchEventImpl(Container.java:2097)
at java.awt.Component.dispatchEvent(Component.java:4488)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4575)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4236)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4166)
at java.awt.Container.dispatchEventImpl(Container.java:2083)
at java.awt.Window.dispatchEventImpl(Window.java:2489)
at java.awt.Component.dispatchEvent(Component.java:4488)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:674)
at java.awt.EventQueue.access$400(EventQueue.java:81)
at java.awt.EventQueue$2.run(EventQueue.java:633)
at java.awt.EventQueue$2.run(EventQueue.java:631)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:98)
at java.awt.EventQueue$3.run(EventQueue.java:647)
at java.awt.EventQueue$3.run(EventQueue.java:645)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:644)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Caused by: org.apache.bsf.BSFException: unable to load language: beanshell
at org.apache.bsf.BSFManager.loadScriptingEngine(BSFManager.java:718)
at com.sunopsis.dwg.codeinterpretor.SnpCodeInterpretor.loadEngine(SnpCodeInterpretor.java:85)
at com.sunopsis.dwg.codeinterpretor.SnpCodeInterpretor.<init>(SnpCodeInterpretor.java:75)
at com.sunopsis.dwg.codeinterpretor.SnpCodeInterpretor.getInstance(SnpCodeInterpretor.java:184)
... 45 more
After digging around for about a day on this issue, I brazenly tried running ODI as the root user on the off chance that this was a permissions issue. I started ODI from the command line (using 'bash odi') for greater verbosity, and it loaded without the error mentioned above. Something gave me the impression that this wasn't a permissions issue, but one related to the user settings.
To rectify the issue, I removed my user's odi settings folder (renaming it, for safety):
mv ~/.odi ~/.backup_odi
Then I started ODI from the terminal under my own user (i.e. not root) - there were no errors! None of my connections were available in the new settings folder though. This I fixed by closing ODI and entering the following:
cp ~/.backup_odi/oracledi/snps_login_work.xml ~/.odi/oracledi/
If anybody else encounters this issue, I hope you find this post quicker than it took me to fix it!
org.apache.bsf.BSFException: unable to load language: beanshell
The exception was thrown because bsh-2.Ob4.jar was not in the classpath and it is a dependent jar of bsf.jar
I'm using Teamcity 5 for our CI environment. It's a great tool, but I've been struggling with one thing: the stdout_yyyyMMdd.log file in the \TeamCity\logs folder grows to ridiculous sizes. Is there a way to turn it off?
Places I've looked so far:
Jetbrains: Nothing on stdout;
Google for "tomcat stdout logs": the first few links don't really address the issue.
Edit:
At KIR's suggestion, I actually looked to see what's in stdout. It's the same exception message repeated over and over again:
[2010-12-01 08:57:21,268] WARN - jetbrains.buildServer.SERVER - java.io.FileNotFoundException: <...Path...>\.BuildServer\system\caches\search\_8p.prx (The system cannot find the file specified)
[2010-12-01 08:57:21,315] ERROR - erverSide.search.SearchService - SearchService.enqueueHistory
java.io.FileNotFoundException: <...Path...>\.BuildServer\system\caches\search\_8p.prx (The system cannot find the file specified)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(Unknown Source)
at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.<init>(SimpleFSDirectory.java:78)
at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(SimpleFSDirectory.java:108)
at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:65)
at org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:132)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:638)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:599)
at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:104)
at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27)
at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:74)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:704)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:69)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:476)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:314)
at jetbrains.buildServer.serverSide.search.SearchService.getIndexSearcher(SearchService.java:451)
at jetbrains.buildServer.serverSide.search.SearchService.enqueueHistory(SearchService.java:515)
at jetbrains.buildServer.serverSide.search.BackgroundIndexer.run(BackgroundIndexer.java:32)
at java.lang.Thread.run(Unknown Source)
Any idea what this file is?
If you're running TC on unix you could use logrotate: http://linuxcommand.org/man_pages/logrotate8.html (Obviously, this is a workaround but it should be effective.)
This guy has a windows equivalent that may do the trick too: http://www.datori.org/?p=7
Remove .BuildServer\system\caches\search directory and restart TeamCity. May be this would help.
The problem is caused by someone or something deleting the Lucene Index in Team city.
Everytime you hit a page after this it will log in stdout that it couldnt find the file.
If you clear out the whole folder which should be
%USERPROFILE%.BuildServer\system\caches\search\
See http://confluence.jetbrains.net/display/TCD5/TeamCity+Data+Directory for more information on where to find the folder.
And restart Teamcity it will recreate the index on start up and stop logging the error message.
Oh and search should start working again too.