I am trying to validate that an assignment is correct. I can't say much, however we have internal and external users. I have an SQL Script that looks for anything other than internal on an internal assignment - result should be 0 rows. I then place this in a SQL table. After that, I've got a statement to calculate if there is an assignment error, and then I store that in a variable. Based off this, I try to validate the data with the 'Data Validator' step. Running the code manually, it should pass, however Spoon PDI is giving me the following error:
2015/05/04 13:03:19 - Data Validator.0 - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Unexpected error
2015/05/04 13:03:19 - Data Validator.0 - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : org.pentaho.di.core.exception.KettleException:
2015/05/04 13:03:19 - Data Validator.0 - Correct Group/Dashboard Assignment
2015/05/04 13:03:19 - Data Validator.0 - Correct Group/Dashboard Assignment
2015/05/04 13:03:19 - Data Validator.0 -
2015/05/04 13:03:19 - Data Validator.0 - at org.pentaho.di.trans.steps.validator.Validator.processRow(Validator.java:159)
2015/05/04 13:03:19 - Data Validator.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2015/05/04 13:03:19 - Data Validator.0 - at java.lang.Thread.run(Unknown Source)
2015/05/04 13:03:19 - Data Validator.0 - Caused by: org.pentaho.di.trans.steps.validator.KettleValidatorException: Correct Group/Dashboard Assignment
2015/05/04 13:03:19 - Data Validator.0 - at org.pentaho.di.trans.steps.validator.Validator.validateFields(Validator.java:258)
2015/05/04 13:03:19 - Data Validator.0 - at org.pentaho.di.trans.steps.validator.Validator.processRow(Validator.java:130)
2015/05/04 13:03:19 - Data Validator.0 - ... 2 more
2015/05/04 13:03:19 - Data Validator.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)
2015/05/04 13:03:19 - transformation_group_dashboard_validator - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Errors detected!
2015/05/04 13:03:19 - Spoon - The transformation has finished!!
2015/05/04 13:03:19 - transformation_group_dashboard_validator - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Errors detected!
2015/05/04 13:03:19 - transformation_group_dashboard_validator - ERROR (version 5.2.0.0, build 1 from 2014-09-30_19-48-28 by buildguy) : Errors detected!
2015/05/04 13:03:19 - transformation_group_dashboard_validator - Transformation detected one or more steps with errors.
2015/05/04 13:03:19 - transformation_group_dashboard_validator - Transformation is killing the other steps!
Is there anyway I can try to fix this?
It looks like the validator is rejecting your input(s), and according to the line in the source code, it isn't handling errors so all you get is an exception. Try creating another step linked to that validator, then right-click on the validator and choose "Define error handling..." and set up some error-related fields that the step will fill in. Also you will want to double-click on the Data Validator step and make sure the "Report all errors" and "...concatenate all errors" checkboxes are selected. That will ensure each row gets a full list of any validation errors that may have occurred.
This often happens when the validation conditions are not set the way the user intended them to be, so rows are rejected when they "should be" selected :)
I managed to fix my problem by deleting my Data Validator step and re-adding afresh one. I've noticed this with Spoon PDI a lot - the end outcome can sometimes be unpredictable and an occasional refresh of step fixes the issue.
Related
I have tried running job using pentaho 9.2 it is not picking the job when running by kitchen comment
./kitchen.sh -rep=repo1 -file=/var/lib/jenkins/project/path/etl/Job1.kjb
Job1.kjb contains multiple sub_jobs. Pentaho 9.2 is not picking the sub job and showing following error.
Error Log:
2022/03/02 05:00:28 - Job1 - Start of job execution
2022/03/02 05:00:28 - Job1 - Starting entry [sub_job_1]
2022/03/02 05:00:28 - sub_job_1 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2021-06-02 06.36.08 by buildguy) : Error running job entry 'job' :
2022/03/02 05:00:28 - sub_job_1 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2021-06-02 06.36.08 by buildguy) : org.pentaho.di.core.exception.KettleException:
2022/03/02 05:00:28 - sub_job_1 - Unexpected error during job metadata load
2022/03/02 05:00:28 - sub_job_1 -
2022/03/02 05:00:28 - sub_job_1 - Unable to load the job from XML file [/var/lib/jenkins/project/path/etl/sub_job_1.kjb.kjb]
2022/03/02 05:00:28 - sub_job_1 -
2022/03/02 05:00:28 - sub_job_1 - File [file:///var/lib/jenkins/project/path/etl/sub_job_1.kjb.kjb] does not exists.**
2022/03/02 05:00:28 - sub_job_1 -
2022/03/02 05:00:28 - sub_job_1 -
2022/03/02 05:00:28 - sub_job_1 -
2022/03/02 05:00:28 - sub_job_1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1467)
2022/03/02 05:00:28 - sub_job_1 - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1385)
2022/03/02 05:00:28 - sub_job_1 - at org.pentaho.di.job.entries.job.JobEntryJob.execute(JobEntryJob.java:695)
2022/03/02 05:00:28 - sub_job_1 - at org.pentaho.di.job.Job.execute(Job.java:693)
2022/03/02 05:00:28 - sub_job_1 - at org.pentaho.di.job.Job.execute(Job.java:834)
2022/03/02 05:00:28 - sub_job_1 - at org.pentaho.di.job.Job.execute(Job.java:503)
2022/03/02 05:00:28 - sub_job_1 - at org.pentaho.di.job.Job.run(Job.java:389)
2022/03/02 05:00:28 - sub_job_1 - Caused by: org.pentaho.di.core.exception.KettleXMLException:
2022/03/02 05:00:28 - sub_job_1 - Unable to load the job from XML file [/var/lib/jenkins/project/path/etl/sub_job_1.kjb.kjb]
You are not providing much details, but maybe the problem is that in your Job1.kjb job you have not defined correctly the location of the subjob. In the log it's saying that it cannot find the file:
/var/lib/jenkins/project/path/etl/sub_job_1.kjb.kjb
Probably your subjob is the file: /var/lib/jenkins/project/path/etl/sub_job_1.kjb
I am using Starschema's JDBC driver to connect Pentaho to BigQuery. I am able to successfully fetch data from BigQuery into Pentaho. However I am not able to write data from Pentaho into BigQuery. There is an exception thrown while Inserting Rows into BigQuery and it seems that the operation may not be supported. How do I solve this?
Error message:
2017/10/30 14:27:43 - Table output 2.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Because of an error, this step can't continue:
2017/10/30 14:27:43 - Table output 2.0 - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : org.pentaho.di.core.exception.KettleException:
2017/10/30 14:27:43 - Table output 2.0 - Error inserting row into table [TableID] with values: [A], [I], [G], [1], [2016-02-18], [11], [2016-02-18-12.00.00.123456], [GG], [CB], [132], [null], [null], [null]
2017/10/30 14:27:43 - Table output 2.0 -
2017/10/30 14:27:43 - Table output 2.0 - Error inserting/updating row
2017/10/30 14:27:43 - Table output 2.0 - executeUpdate()
2017/10/30 14:27:43 - Table output 2.0 -
2017/10/30 14:27:43 - Table output 2.0 -
2017/10/30 14:27:43 - Table output 2.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:385)
2017/10/30 14:27:43 - Table output 2.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:125)
2017/10/30 14:27:43 - Table output 2.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2017/10/30 14:27:43 - Table output 2.0 - at java.lang.Thread.run(Unknown Source)
2017/10/30 14:27:43 - Table output 2.0 - Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
2017/10/30 14:27:43 - Table output 2.0 - Error inserting/updating row
2017/10/30 14:27:43 - Table output 2.0 - executeUpdate()
2017/10/30 14:27:43 - Table output 2.0 -
2017/10/30 14:27:43 - Table output 2.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1321)
2017/10/30 14:27:43 - Table output 2.0 - at org.pentaho.di.trans.steps.tableoutput.TableOutput.writeToTable(TableOutput.java:262)
2017/10/30 14:27:43 - Table output 2.0 - ... 3 more
2017/10/30 14:27:43 - Table output 2.0 - Caused by: net.starschema.clouddb.jdbc.BQSQLFeatureNotSupportedException: executeUpdate()
2017/10/30 14:27:43 - Table output 2.0 - at net.starschema.clouddb.jdbc.BQPreparedStatement.executeUpdate(BQPreparedStatement.java:317)
2017/10/30 14:27:43 - Table output 2.0 - at org.pentaho.di.core.database.Database.insertRow(Database.java:1288)
2017/10/30 14:27:43 - Table output 2.0 - ... 4 more
2017/10/30 14:27:43 - BigQuery_rwa-tooling - Statement canceled!
2017/10/30 14:27:43 - Simple Read Write from csv to txt - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : Something went wrong while trying to stop the transformation: org.pentaho.di.core.exception.KettleDatabaseException:
2017/10/30 14:27:43 - Simple Read Write from csv to txt - Error cancelling statement
2017/10/30 14:27:43 - Simple Read Write from csv to txt - cancel()
2017/10/30 14:27:43 - Simple Read Write from csv to txt - ERROR (version 7.1.0.0-12, build 1 from 2017-05-16 17.18.02 by buildguy) : org.pentaho.di.core.exception.KettleDatabaseException:
2017/10/30 14:27:43 - Simple Read Write from csv to txt - Error cancelling statement
2017/10/30 14:27:43 - Simple Read Write from csv to txt - cancel()
2017/10/30 14:27:43 - Simple Read Write from csv to txt -
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.core.database.Database.cancelStatement(Database.java:750)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.core.database.Database.cancelQuery(Database.java:732)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.trans.steps.tableinput.TableInput.stopRunning(TableInput.java:299)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.trans.Trans.stopAll(Trans.java:1889)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.trans.step.BaseStep.stopAll(BaseStep.java:2915)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.trans.steps.tableoutput.TableOutput.processRow(TableOutput.java:139)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at java.lang.Thread.run(Unknown Source)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - Caused by: net.starschema.clouddb.jdbc.BQSQLFeatureNotSupportedException: cancel()
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at net.starschema.clouddb.jdbc.BQStatementRoot.cancel(BQStatementRoot.java:113)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - at org.pentaho.di.core.database.Database.cancelStatement(Database.java:744)
2017/10/30 14:27:43 - Simple Read Write from csv to txt - ... 7 more
2017/10/30 14:27:43 - Table output 2.0 - Signaling 'output done' to 0 output rowsets.
2017/10/30 14:27:43 - BigQuery_prID - No commit possible on database connection [BigQuery_prID]
It looks like you may be trying to do this via legacy SQL, which has no support for DML statements (INSERT/UPDATE/DELETE).
Standard SQL does support DML, but these are largely to support bulk table manipulations as opposed to row-oriented insertions; ingesting data via the use of individual DML INSERTs is not recommended. See the quotas on the DML reference documentation for more details.
You're better off using either BigQuery streaming or bulk ingestion via a load job for ingestion, but as these mechanisms are outside of the query language you may need to move beyond using a JDBC driver.
I am facing an issue when I am trying analyzing iPhone app code with the help of sonarqube.
I am getting below error when trying to execute sonar-runner from app root directory.
11:06:17.040 INFO - Index files<br/>
11:06:18.276 INFO - 214 files indexed<br/>
11:06:18.966 INFO - Quality profile for objc: OCLint<br/>
11:06:19.016 INFO - Sensor QProfileSensor...<br/>
11:06:19.020 INFO - Sensor QProfileSensor done: 4 ms<br/>
11:06:19.020 INFO - Sensor ObjectiveCSquidSensor...<br/>
11:06:19.334 INFO - Sensor ObjectiveCSquidSensor done: 314 ms<br/>
11:06:19.334 INFO - Sensor Objective-C SurefireSensor...<br/>
11:06:19.334 INFO - parsing sonar-reports<br/>
11:06:19.334 INFO - Sensor Objective-C SurefireSensor done: 0 ms<br/>
11:06:19.335 INFO - Sensor org.sonar.plugins.objectivec.coverage.CoberturaSensor#6544899b...<br/>
11:06:19.342 INFO - Sensor
org.sonar.plugins.objectivec.coverage.CoberturaSensor#6544899b done: 7 ms<br/>
11:06:19.342 INFO - Sensor org.sonar.plugins.objectivec.violations.OCLintSensor#6da54910...<br/>
11:06:19.342 INFO - Processing OCLint report /Users/org/Documents/C/ISH/projects/Proj1/mobile2.2.0.5/mobile/ios/Proj1-Mobile/./sonar-reports/oclint.xml<br/>
11:06:19.346 ERROR - Error processing file named /Users/org/Documents/C/ISH/projects/Proj1/mobile2.2.0.5/mobile/ios/Proj1-Mobile/./sonar-reports/oclint.xml<br/>
java.io.FileNotFoundException: /Users/org/Documents/C/ISH/projects/Proj1/mobile2.2.0.5/mobile/ios/Proj1-Mobile/./sonar-reports/oclint.xml (No such file or directory)
at java.io.FileInputStream.open0(Native Method) ~[na:1.8.0_60]
at java.io.FileInputStream.open(FileInputStream.java:195) ~[na:1.8.0_60]
at java.io.FileInputStream.<init>(FileInputStream.java:138) ~[na:1.8.0_60]
at org.sonar.plugins.objectivec.violations.OCLintParser.parseReport(OCLintParser.java:49) ~[sonar-objective-c-plugin-0.4.0.jar:na]
Thanks in advance ,Let me know if someone faced same problem before.
If my understanding is correct, you're using the community SonarQube Objective-C plugin (https://github.com/octo-technology/sonar-objective-c), the best best to get some help is to create an issue here : https://github.com/octo-technology/sonar-objective-c/issues
I met same issue before, I solved it by using
sudo ./run-sonar.sh
The reason could be the oclint.xml is generated by root user before, and you are running as non-root user, so no privilege for you to access oclint.xml
Another possible reason is the dot(.) prefix in sonar-project.properties, Proj1-Mobile/./sonar-reports/oclint.xml, try to remove the ./ prefix in your properties file for the oclint.xml
I have a stored procedure in Sybase that returns some data in a temp table (let's call it #results). If I create this table from my own code and call the stored procedure and check the #results everything is fine (I use Python for this purpose). I can generate the expected result set for my test.
However when I try to create a DBUnit test case so that the code
creates the temp table via the Spring JdbcTemplate
calls this procedure and checks the returned data
I get a NoSuchTableException from DbUnit when checking the result set.
15:55:52.980 [main] DEBUG c.m.e.e.dbtest.helper.SQLTestHelper - *** expected table: #results
15:55:52.980 [main] DEBUG org.dbunit.dataset.FilteredDataSet - getTable(tableName=#results) - start
15:55:52.980 [main] DEBUG o.d.d.filter.SequenceTableFilter - accept(tableName=#results) - start
15:55:52.980 [main] DEBUG o.dbunit.dataset.OrderedTableNameMap - getTableName(tableName=#results) - start
15:55:52.980 [main] DEBUG o.dbunit.dataset.OrderedTableNameMap - getTableName(tableName=#results) - end - result=#RESULTS
15:55:52.980 [main] DEBUG org.dbunit.database.DatabaseDataSet - getTable(tableName=#results) - start
15:55:52.980 [main] DEBUG org.dbunit.database.DatabaseDataSet - initialize() - start
15:55:52.980 [main] DEBUG org.dbunit.database.DatabaseDataSet - Initializing the data set from the database...
15:55:52.980 [main] DEBUG o.d.d.DatabaseDataSourceConnection - getConnection() - start
15:55:52.980 [main] DEBUG m.b.t.TransactionResourceManager - Fetched TransactionResourceResolver [null]
15:55:52.980 [main] DEBUG msjava.pool.BasicObjectPool - Attempting to retrieve object from pool 'ConnectionPoolOf-dbunitDS'
15:55:52.980 [main] DEBUG msjava.pool.BasicObjectPool - Retrieved object 'TransactionInfoCachingConnection on DBCallSavingConnection on com.sybase.jdbc3.jdbc.SybConnection#170984c' from pool 'ConnectionPoolOf-dbunitDS'
15:55:53.402 [main] DEBUG org.dbunit.database.DatabaseDataSet -
database product name=Adaptive Server Enterprise
database major version=12
database minor version=5
jdbc driver name=jConnect (TM) for JDBC (TM)
jdbc driver version=jConnect (TM) for JDBC(TM)/6.05(Build 26564)/P/EBF16903/JDK14/Sun May 31 1:05:35 2009
jdbc driver major version=6
jdbc driver minor version=0
15:55:53.402 [main] DEBUG org.dbunit.database.DatabaseDataSet - metadata resultset=com.sybase.jdbc3.jdbc.SybResultSet#1b595f3
15:55:53.402 [main] DEBUG org.dbunit.database.DatabaseDataSet - getTableMetaData(tableName=#results) - start
15:55:53.402 [main] DEBUG org.dbunit.database.DatabaseDataSet - initialize() - start
15:55:53.402 [main] DEBUG o.dbunit.dataset.OrderedTableNameMap - getTableName(tableName=#results) - start
15:55:53.402 [main] DEBUG o.dbunit.dataset.OrderedTableNameMap - getTableName(tableName=#results) - end - result=#RESULTS
15:55:53.402 [main] ERROR org.dbunit.database.DatabaseDataSet - Table '#results' not found in tableMap=org.dbunit.dataset.OrderedTableNameMap[_tableNames=[], _tableMap={}, _caseSensitiveTableNames=false]
Checking the database in parallel with the test code execution (at a breakpoint) I can see that the temp table is indeed created in tempdb and filled with data.
Has anyone done such a thing successfully (eg. used a Sybase temp table from DbUnit)? If yes, how?
If you create temporary tables by using “#” as the first character of the table name:
Temporary tables:
Exist only for the duration of the user session or for the scope of the procedure that creates them
Cannot be shared between user connections
Are automatically dropped at the end of the session or procedure (or can be dropped manually)
You can create regular user tables in tempdb by specifying the database name in the command that creates the table:
Regular user tables in tempdb:
Can persist across sessions
Can be used by bulk copy operations
Can be shared by granting permissions on them
Must be explicitly dropped by the owner (otherwise, they are removed when Adaptive Server is restarted)
I'm using (the rather heavy weight) xtext to create a basic DSL parser and text highlighting.
I'm following the getting started tutorial at http://wiki.eclipse.org/Xtext/GettingStarted
The output in the console i'm getting is:
0 [main] INFO eclipse.emf.mwe.core.WorkflowRunner - --------------------------------------------------------------------------------------
0 [main] INFO eclipse.emf.mwe.core.WorkflowRunner - EMF Modeling Workflow Engine 0.7.2, Build v200908120417
0 [main] INFO eclipse.emf.mwe.core.WorkflowRunner - (c) 2005-2009 openarchitectureware.org and contributors
0 [main] INFO eclipse.emf.mwe.core.WorkflowRunner - --------------------------------------------------------------------------------------
0 [main] INFO eclipse.emf.mwe.core.WorkflowRunner - running workflow: C:/Users/Richard/workspace/org.xtext.example.mydsl.generator/src/workflow/MyDslGenerator.mwe
0 [main] INFO eclipse.emf.mwe.core.WorkflowRunner -
297 [main] ERROR eclipse.emf.mwe.core.WorkflowRunner - [ERROR]: Class not found: 'org.xtext.example.MyDslStandaloneSetup'(Element: bean register class='org.xtext.example.MyDslStandaloneSetup' in C:/Users/Richard/workspace/org.xtext.example.mydsl.generator/src/workflow/MyDslGenerator.mwe:11; Reported by: -UNKNOWN-)
297 [main] ERROR eclipse.emf.mwe.core.WorkflowRunner - Workflow interrupted because of configuration errors.
I get this output at step 4 "DSL Generation" in the tutorial.
The answer was a simple mestake made by me, while following the tutorial. In short I was running the incorrect .mwe file. The one I was running found in org.xtext.example.mydsl.generator/src/workflow while the .mwe I needed is listed under org.xtext.example.mydsl/src/org.xtext.example.