Our Continuous Inegration server (Hudosn) is having a strange issue when attempting to run a simple create table statement in Teradata.
This statement tests the existence of the max_call table:
unless $teradata_connection.table_exists? :arm_custom_db__max_call_attempt_parameters
$teradata_connection.run('CREATE TABLE all_wkscratchpad_db.max_call_attempt_parameters AS (SELECT * FROM arm_custom_db.max_call_attempt_parameters ) WITH NO DATA')
end
The table_exists? method does the following:
def table_exists?(name)
v ||= false # only retry once
sch, table_name = schema_and_table(name)
name = SQL::QualifiedIdentifier.new(sch, table_name) if sch
from(name).first
true
rescue DatabaseError => e
if e.to_s =~ /Operation not allowed for reason code "7" on table/ && v == false
# table probably needs reorg
reorg(name)
v = true
retry
end
false
end
So as per the from(name).first line, the test which this method is performing is just a simple select statement, which, in SQL, looks like: SELECT TOP 1 MAX(CAST(MAX_CALL_ATTEMPT_CNT AS BIGINT)) FROM ALL_WKSCRATCHPAD_DB.MAX_CALL_ATTEMPT_PARAMETERS
The above SQL statement executes perfectly fine within Teradata SQL Assistant, so it's not a SQL syntax issue. The generic ID which our testing suite (Rubymine) uses is also not the issue; that ID has select access to the arm_custom_db.
The exeption which I can see is being thrown (within the builds console output on Hudson) is
Sequel::DatabaseError: Java::ComTeradataJdbcJdbc_4Util::JDBCException. Since this execption is a subclass of DatabaseError, the exception shouldn't be the problem either.
Also: We use unless statements like this every day for hundreds of different tables, and all except this one work correctly. This statement just seems to be a problem.
The complete error message which appears in the builds console output of Hudson is as follows:
[2015-01-07T13:56:37.947000 #16702] ERROR -- : Java::ComTeradataJdbcJdbc_4Util::JDBCException: [Teradata Database] [TeraJDBC 13.10.00.17] [Error 3807] [SQLState 42S02] Object 'ALL_WKSCRATCHPAD_DB.MAX_CALL_ATTEMPT_PARAMETERS' does not exist.: SELECT TOP 1 MAX(CAST(MAX_CALL_ATTEMPT_CNT AS BIGINT)) FROM ALL_WKSCRATCHPAD_DB.MAX_CALL_ATTEMPT_PARAMETERS
Sequel::DatabaseError: Java::ComTeradataJdbcJdbc_4Util::JDBCException: [Teradata Database] [TeraJDBC 13.10.00.17] [Error 3807] [SQLState 42S02] Object 'ALL_WKSCRATCHPAD_DB.MAX_CALL_ATTEMPT_PARAMETERS' does not exist.
I don't understand why this specific bit of code is giving me issues...there does not appear to be anything special about this table or database, and all SQL code executes perfectly fine in Teradata when I am signed in with the same exact user ID that is being used to execute the code from Hudson.
Related
I am working with Apache Kylin and using Sqoop to connect to my PostgreSQL database. I have a cube created based on a fact table that references the same dimensional table twice. So the problem arises when I try to build the cube, I get the following error on the first step of the job (#1 Step Name: Sqoop To Flat Hive Table):
ERROR manager.SqlManager: Error executing statement: org.postgresql.util.PSQLException: ERROR: table name "d_date" specified more than once
The problem is that Sqoop generates SQL and references the table d_date twice and gives it the same alias both times so the SQL statement fails... Can I configure it in any way to fix this issue?
Edit: If the answer is no, that is also helpful, I just really need to know whether there is anything I can do to fix this...
This is the generated SQL (bold is where the problem is):
SELECT f_exam.course_id as F_EXAM_COURSE_ID ,f_exam.academic_year_id as F_EXAM_ACADEMIC_YEAR_ID ,f_exam.semester_id as F_EXAM_SEMESTER_ID ,f_exam.exam_id as F_EXAM_EXAM_ID ,f_exam.exam_app_user_created_id as F_EXAM_EXAM_APP_USER_CREATED_ID ,f_exam.exam_available_from_date_id ,f_exam.exam_available_from_time_id as F_EXAM_EXAM_AVAILABLE_FROM_TIME_ID ,f_exam.exam_available_to_date_id as F_EXAM_EXAM_AVAILABLE_TO_DATE_ID ,f_exam.exam_available_to_time_id as F_EXAM_EXAM_AVAILABLE_TO_TIME_ID ,f_exam.exam_ordinal_id as F_EXAM_EXAM_ORDINAL_ID ,d_time_day.time_day_id as D_AVAILABLE_FROM_TIME_TIME_DAY_ID ,d_time_day.hour_minutes_seconds as D_AVAILABLE_FROM_TIME_HOUR_MINUTES_SECONDS ,d_time_day.the_seconds as D_AVAILABLE_FROM_TIME_THE_SECONDS ,d_time_day.the_minutes as D_AVAILABLE_FROM_TIME_THE_MINUTES ,d_time_day.the_hours as D_AVAILABLE_FROM_TIME_THE_HOURS ,d_time_day.period_of_day as D_AVAILABLE_FROM_TIME_PERIOD_OF_DAY ,d_time_day.time_day_id as D_AVAILABLE_TO_TIME_TIME_DAY_ID ,d_time_day.hour_minutes_seconds as D_AVAILABLE_TO_TIME_HOUR_MINUTES_SECONDS ,d_time_day.the_seconds as D_AVAILABLE_TO_TIME_THE_SECONDS ,d_time_day.the_minutes as D_AVAILABLE_TO_TIME_THE_MINUTES ,d_time_day.the_hours as D_AVAILABLE_TO_TIME_THE_HOURS ,d_time_day.period_of_day as D_AVAILABLE_TO_TIME_PERIOD_OF_DAY ,f_exam.number_of_questions as F_EXAM_NUMBER_OF_QUESTIONS ,f_exam.duration_in_seconds as F_EXAM_DURATION_IN_SECONDS ,f_exam.number_of_students_participated as F_EXAM_NUMBER_OF_STUDENTS_PARTICIPATED ,f_exam.is_forward_only_01 as F_EXAM_IS_FORWARD_ONLY_01 ,f_exam.max_score_possible as F_EXAM_MAX_SCORE_POSSIBLE ,f_exam.max_score as F_EXAM_MAX_SCORE ,f_exam.min_score as F_EXAM_MIN_SCORE ,f_exam.pass_percentage as F_EXAM_PASS_PERCENTAGE ,f_exam.max_score_percentage as F_EXAM_MAX_SCORE_PERCENTAGE ,f_exam.min_score_percentage as F_EXAM_MIN_SCORE_PERCENTAGE ,f_exam.avg_score as F_EXAM_AVG_SCORE ,f_exam.median as F_EXAM_MEDIAN ,f_exam.first_quartile as F_EXAM_FIRST_QUARTILE ,f_exam.third_quartile as F_EXAM_THIRD_QUARTILE ,f_exam.interquartile_range as F_EXAM_INTERQUARTILE_RANGE ,f_exam.minimum_without_outliers as F_EXAM_MINIMUM_WITHOUT_OUTLIERS ,f_exam.maximum_without_outliers as F_EXAM_MAXIMUM_WITHOUT_OUTLIERS FROM public.f_exam f_exam INNER JOIN public.d_course d_course ON f_exam.course_id = d_course.course_id INNER JOIN public.d_academic_year d_academic_year ON f_exam.academic_year_id = d_academic_year.academic_year_id INNER JOIN public.d_semester d_semester ON f_exam.semester_id = d_semester.semester_id INNER JOIN public.d_exam d_exam ON f_exam.exam_id = d_exam.exam_id INNER JOIN public.d_app_user d_app_user ON f_exam.exam_app_user_created_id = d_app_user.app_user_id INNER JOIN public.d_date d_date ON f_exam.exam_available_from_date_id = d_date.date_id INNER JOIN public.d_time_day d_time_day ON f_exam.exam_available_from_time_id = d_time_day.time_day_id INNER JOIN public.d_date d_date ON f_exam.exam_available_to_date_id = d_date.date_id INNER JOIN public.d_time_day d_time_day ON f_exam.exam_available_to_time_id = d_time_day.time_day_id INNER JOIN public.d_ordinal d_ordinal ON f_exam.exam_ordinal_id = d_ordinal.ordinal_id WHERE 1=1 AND (f_exam.exam_available_from_date_id >= 20120101 AND f_exam.exam_available_from_date_id < 20170101) AND (1 = 0)
The long long SQL is generated by Kylin and submitted to sqoop for execution. So you really need is to fix the duplicated alias of the two public.d_date d tables in Kylin model definition.
In Kylin model designer, around your fact table f_exam, there must be two public.d_date d. Set their alias to different names, save the model and try build again. This shall change the generated SQL and let the sqoop step pass.
I am trying to run the below code using H2DB (via junit test), while doing so i get error message as below. I understand that, there are no function available as "days" in H2. So i am trying to write a custom function, but it does not work out, can any one help on writing this function.
SQLBuilder class code:
public String dummy() {
return new StringBuilder(new SQL() {
{
SELECT("date(CREATE_TMS)");
SELECT("CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_B");
FROM("Q.DUMMY");
}
}.toString().concat(" FOR READ ONLY WITH UR")).toString();
}
Error message:
org.springframework.jdbc.BadSqlGrammarException:
### Error querying database. Cause: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "SELECT DATE(CREATE_TMS), CASE WHEN DATE(CREATE_TMS) >= (CURRENT DATE - CAST('1' AS INTEGER) DAYS[*]) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR "; expected "[, ::, *, /, %, +, -, ||, ~, !~, NOT, LIKE, ILIKE, REGEXP, IS, IN, BETWEEN, AND, OR, ,, )"; SQL statement:
SELECT date(CREATE_TMS), CASE WHEN date(CREATE_TMS) >= (CURRENT DATE - cast('1' AS integer) days) THEN 'Y' ELSE 'N' END NEW_BILLING
FROM Q.DUMMY FOR READ ONLY WITH UR [42001-199]
For some reason days are converted to DAYS[*], we can see that in error message.
Customer method i tried in schema-db2.sql:
drop ALIAS if exists days;
CREATE ALIAS days as '
import java.lang.String;
#CODE
java.lang.String days() throws Exception {
return "days";
}
';
applicaiton.properties:
spring.datasource.url=jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false;Mode=DB2
DAYS is not a function and is not a something that other databases support. Db2 also uses non-standard interval literals.
If you can build H2 from its current sources, you can use cast('1' AS integer) day in it (not the days) and such construction is also supported by Db2. You can also simply use 1 DAY, it is supported by current H2 and Db2 too.
(CURRENT_DAY - 1 DAY)
Sources of H2 are available on GitHub:
https://github.com/h2database/h2database
Building instructions are here:
https://h2database.com/html/build.html#building
You need a jar target.
To compile H2 from the current sources you need JDK 8, 9, 10, 11, or 12. Compiled jar will be compatible with more recent versions.
When executing below code in phpMyAdmin:
use db;
DELIMITER $$
DROP PROCEDURE IF EXISTS McaTest3$$
CREATE PROCEDURE McaTest3()
BEGIN
SELECT
cl.*
FROM `condition_library` cl
LEFT JOIN condition_custom cc on cl.condition_library_id = cc.condition_library_id
and cc.active = 1
AND (cc.permit_application_id = 20231 OR cc.permit_id = NULL)
WHERE FIND_IN_SET(cl.`condition_library_id`, '13070')
AND cl.active = 1
and cc.condition_library_id IS NULL;
END$$
DELIMITER ;
call McaTest3();
Getting error:
Error
Static analysis:
1 errors were found during analysis.
Missing expression. (near "ON" at position 25)
SQL query: Edit Edit
SET FOREIGN_KEY_CHECKS = ON;
MySQL said: Documentation
#2014 - Commands out of sync; you can't run this command now
This happens when there is no record found in the table which is at LEFT JOIN.
When the same is ran in MySQL Workbench: NO ERROR and return empty dataset.
Same procedure when executed from Application (Appian) is failing as well… Any clues?
Another question on Stackoverflow answered my issue:
link: MySQL error #2014 - Commands out of sync; you can't run this command now
I'm currently writing a moped log parser in order to monitor moped queries runtime.
It's work great for QUERY command using the runtime parameter, but INSERT and UPDATE have no runtime parameter. All INSERT and UPDATE are followed by a getLastError COMMAND which contains a runtime.
Here are some samples of moped logs:
QUERY with runtime
MOPED: 127.0.0.1:27017 QUERY database=X collection=X selector=X
flags=[] limit=-1 skip=0 batch_size=nil fields=nil runtime: 0.6950ms
INSERT without runtime but with COMMAND
MOPED: 127.0.0.1:27017 INSERT database=X collection=X documents=X flags=[]
COMMAND database=X command={:getlasterror=>1, :w=>1}
runtime: 0.4750ms
I'm pretty sure that COMMAND runtime is for the getlasterror call and not for my INSERT one.
So is there a way to get this runtime info for an INSERT query?
Instead of using a log parser, I use something like this and it works great:
ActiveSupport::Notifications.subscribe('query.moped') do |name, start, finish, id, payload|
runtime = (finish - start)*1000
moped_ops = payload[:ops]
moped_ops.each do |op|
unless op.collection == '$cmd'
query = op.class.name.split('::').last.downcase
query = op.selector.first[0].to_s.gsub(/\$/, '') if query == 'command'
DO SOMETHING WITH #{op.database}.#{op.collection}.#{query}
end
end
end
I develop with Ruby 2, Rails 4.0, and Oracle 11,
When starting with Tests, I couldn't purge neither load the Test Database, many errors appeared:
ActiveRecord::StatementInvalid: OCIError: ORA-00900: SQL sentence not valid
ActiveRecord::StatementInvalid: OCIError: ORA-00933: SQL command not properly ended:
Additional Warning appears:
DEPRECATION WARNING: This database tasks were deprecated,
because this tasks should be served by the 3rd party adapter.
(called from mon_synchronize at /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/monitor.rb:211)
I've modified the next files and now I can execute
rake db:test:load and rake test and database is correctly generated, but warning still appears.
I suppose someone should modify the gems in origin, but who? And if these changes are correct, how can I report this?
File:
usr/local/rvm/gems/ruby-2.0.0-p353#402/gems/activerecord-4.0.2/lib/active_record/tasks/oracle_database_tasks.rb
Added: if (ddl.size>2)
def purge
establish_connection(:test)
connection.structure_drop.split(";\n\n").each { |ddl|
connection.execute(ddl) if (ddl.size>2) }
end
File:
/usr/local/rvm/gems/ruby-2.0.0-p353#402/gems/activerecord-oracle_enhanced-adapter-1.5.5/lib/active_record/connection_adapters/oracle_enhanced_structure_dump.rb
1st Change: Remove / from STATEMENT_TOKEN
# STATEMENT_TOKEN = "\n\n/\n\n"
STATEMENT_TOKEN = "\n\n\n\n"
2nd Change: Add ; after "DROP SEQUENCE..." AND "DROP TABLE....."
# Albert
def structure_drop #:nodoc:
statements = select_values("SELECT sequence_name FROM user_sequences ORDER BY 1").map do |seq|
"DROP SEQUENCE \"#{seq}\";" #Added -->;
end
select_values("SELECT table_name from all_tables t
WHERE owner = SYS_CONTEXT('userenv', 'session_user') AND secondary = 'N'
AND NOT EXISTS (SELECT mv.mview_name FROM all_mviews mv WHERE mv.owner = t.owner AND mv.mview_name = t.table_name)
AND NOT EXISTS (SELECT mvl.log_table FROM all_mview_logs mvl WHERE mvl.log_owner = t.owner AND mvl.log_table = t.table_name)
ORDER BY 1").each do |table|
statements << "DROP TABLE \"#{table}\" CASCADE CONSTRAINTS;" # Added -->;
end
join_with_statement_token(statements)
end
File:
/usr/local/rvm/gems/ruby-2.0.0-p353#402/gems/activerecord-oracle_enhanced-adapter-1.5.5/lib/active_record/connection_adapters/oracle_enhanced_schema_statements
(I don't know if this was correct, or I broke this)
Remove: ;
execute "DROP SEQUENCE #{quote_table_name(seq_name)}" rescue nil
there was a ;--> "seq_name)};" recue nil
and in same folder
oracle_enhanced_adapter.rb
there was also a ";", in "...sequence_name)} ; "), that I removed:
execute ("DROP SEQUENCE #{quote_table_name(sequence_name)}")