The database version we are using is MonetDB 5 server v11.37.7 (Jun2020) on Debian 4.9.210-1+deb9u1.
I have a regular table with about 22M entries. I added a Boolean column with:
alter table conversations add column "virtual" Boolean Null ;
Now when I try to query the table I get this error:
sql>select * from conversations where created > '2020-08-25 10:00';
GDK reported error: project_sht: does not match always
Deleted the added column, restarted Monet, but still the same error. The file merovingian.log Shows:
2020-08-25 16:07:22 ERR reports[23151]: #DFLOWworker3: project_sht: !ERROR: does not match always
2020-08-25 16:07:22 ERR reports[23151]: #DFLOWworker2: project_bte: !ERROR: does not match always
2020-08-25 16:07:22 ERR reports[23151]: #DFLOWworker3: createExceptionInternal: !ERROR: MALException:algebra.projection:GDK reported error: project_sht: does not match always
2020-08-25 16:07:22 ERR reports[23151]: #DFLOWworker2: createExceptionInternal: !ERROR: MALException:algebra.projection:GDK reported error: project_bte: does not match always
Any ideas how to fix this problem or how to debug it further?
Thanks!
This sounds like a bug but I could not reproduce it:
sql>create table foo(i int);
operation successful
sql>insert into foo select * from generate_series(0,22000000);
22000000 affected rows
sql>alter table foo add column b boolean null;
operation successful
sql>select * from foo where i % 2200000 = 42;
+----------+-------+
| i | b |
+==========+=======+
| 42 | null |
| 2200042 | null |
| 4400042 | null |
| 6600042 | null |
| 8800042 | null |
| 11000042 | null |
| 13200042 | null |
| 15400042 | null |
| 17600042 | null |
| 19800042 | null |
+----------+-------+
10 tuples
This looks about right, but maybe I misunderstood your scenario.
If possible, please try to create a standalone script that reproduces the issue along the lines of the above example, and submit it as a bug at the MonetDB Bug Tracker
Best regards,
Joeri
Related
I am a newcomer to Go. I have an old tool to check and compare data in the Mysql database to my device, and I want to rewrite the tool in Go.
Since the tables and data have been already in the Mysql, I try to use GORM to auto map the existing tables. But I am not sure how to do that? I did not find any description of automapping an existing table in the GORM documentation.
I redeclare the existing table model and try to query data. The procedure is as below:
For example one of my tables is like this:
MariaDB [neutron]> desc lbaas_loadbalancers;
+---------------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+--------------+------+-----+---------+-------+
| project_id | varchar(255) | YES | MUL | NULL | |
| id | varchar(36) | NO | PRI | NULL | |
| name | varchar(255) | YES | | NULL | |
| description | varchar(255) | YES | | NULL | |
| vip_port_id | varchar(36) | YES | MUL | NULL | |
| vip_subnet_id | varchar(36) | NO | | NULL | |
| vip_address | varchar(36) | YES | | NULL | |
| admin_state_up | tinyint(1) | NO | | NULL | |
| provisioning_status | varchar(16) | NO | | NULL | |
| operating_status | varchar(16) | NO | | NULL | |
| flavor_id | varchar(36) | YES | MUL | NULL | |
+---------------------+--------------+------+-----+---------+-------+
11 rows in set (0.002 sec)
MariaDB [neutron]> select * from lbaas_loadbalancers \G;
*************************** 1. row ***************************
project_id: 346052548d924ee095b3c2a4f05244ac
id: f6638d02-29f8-41aa-9433-179bf49f5fbd
name: test1
description:
vip_port_id: 21cebbd5-fa4c-4d20-9858-d14ba3eacea8
vip_subnet_id: 0916f471-afcd-48ee-afc5-56bcb0efa963
vip_address: 172.168.1.6
admin_state_up: 1
provisioning_status: ACTIVE
operating_status: ONLINE
flavor_id: NULL
1 row in set (0.003 sec)
Then I try to use GORM mapping the table. I just chosen two fields ID and Name for the test.
package main
import (
"log"
"gorm.io/driver/mysql"
"gorm.io/gorm"
)
// declare only two attribute in the model for test purpose
type Lbaas_loadbalancers struct {
ID string
Name string
}
func main() {
var lb Lbaas_loadbalancers
dsn := "test:test#tcp(192.168.0.17:3306)/test?charset=utf8mb4&parseTime=True&loc=Local"
db, err := gorm.Open(mysql.Open(dsn), &gorm.Config{})
if err != nil {
log.Fatal("connection error")
}
test := db.Take(&lb)
log.Println("test err is ", test.Error)
log.Println(test.RowsAffected)
// this line report error: ./db.go:25:6: test.ID undefined (type *gorm.DB has no field or method ID)
log.Println(test.ID)
// if I comment the above line, this print out 'mysql', but the actual name is 'test1'.
log.Println(test.Name())
}
Finally, I run go run db.go, I got this error:
➜ test git:(main) ✗ go run db.go
# command-line-arguments
./db.go:27:20: cannot convert test.Config.Dialector.Name (type func() string) to type string
It seems not the right way to do it. what is the correct way to auto map an existing database in Mysql by using GORM module?
If the below code is the correct way, why I cannot get the ID attribute from the return value of db.Take method directly? Do I need to do data conversion?
Please give me some hints, thanks.
I know what is wrong here, I should not get ID and Name from the db.Take return, It takes the address of lb variable, and change the lb.
I am so silly, just realized the problem. :)
I'm updating an oracle table with a UDT (user defined type), and then querying the V$LOGMNR_CONTENTS view. I'm seeing that for each row updated there are 2 records - UPDATE and INTERNAL. I need to figure out how to link between them as the UPDATE operation has a temporary value in the ROW_ID and the correct value appears only in the INTERNAL operation, and I'm not sure how do their SCN numbers relate. The way I'm thinking about is to make a queue of UPDATEs per DATA_OBJ#, and link them to the INTERNALs FIFO. Is there something nicer I'm missing?
Script:
CREATE TYPE srulon AS OBJECT (name VARCHAR2(30),phone VARCHAR2(20) );
create table root.udt_table (myrowid rowid, myudt srulon);
BEGIN rdsadmin.rdsadmin_util.switch_logfile;END;
insert into root.udt_table values (null, srulon('small', '1234'));
commit;
BEGIN rdsadmin.rdsadmin_util.switch_logfile;END;
insert into root.udt_table values (null, srulon('small', '1234'));
update root.udt_table set myrowid=rowid, myudt = srulon('smaller', rowid);
commit;
BEGIN rdsadmin.rdsadmin_util.switch_logfile;END;
Query (after START_LOGMNR for the last log):
select scn, SEQUENCE#,operation, SQL_REDO, ROW_ID from V$LOGMNR_CONTENTS
where session# = 6366 and not operation like '%XML%'
order by scn, SEQUENCE#;
Results:
| SCN | SEQUENCE# | OPERATION | ROW\_ID | SQL\_REDO |
| :--- | :--- | :--- | :--- | :--- |
| 240676056 | 1 | INTERNAL | AAB1avAAAAAAwT7AAA | NULL |
| 240676056 | 1 | UPDATE | AAAAAAAAAAAAAAAAAA | update "ROOT"."UDT\_TABLE" a set a."MYROWID" = 'AAB1avAAAAAAwT7AAA' where a."MYROWID" IS NULL; |
| 240676057 | 5 | INTERNAL | AAB1avAAAAAAwT7AAA | NULL |
| 240676058 | 1 | UPDATE | AAAAAAAAAAAAAAAAAA | update "ROOT"."UDT\_TABLE" a set a."MYROWID" = 'AAB1avAAAAAAwT7AAB' where a."MYROWID" IS NULL; |
| 240676059 | 5 | INTERNAL | AAB1avAAAAAAwT7AAB | NULL |
| 240676069 | 1 | COMMIT | AAAAAAAAAAAAAAAAAA | commit; |
System Change Number (SCN) is the main controlling function that is used to keep track of database transactional activity. SCN is a stamp that defines a committed version of a database at a particular point in time. Every committed transaction gets a unique SCN assigned. DB keeps records of all database changes with the help of SCN numbers. SCN is a running number for the database changes
To get current SCN use
DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER()
So there is no other Connection between the UPDATE and INTERNAL Operation then the Fact that UPDATE SCN is lower then INTERNAL SCN - but no calculated or logical connection
the mistake was to order by scn, SEQUENCE#.
once you remove the order by clause, each INTERNAL statement follows its corresponding UPDATE.
credit goes to srulon.
There is a database that contains several views and tables.
I need create a report (documentation of database) with a list of all the fields in these tables indicating the type and, if possible, an indication of the minimum/maximum values and values from first row. For example:
.------------.--------.--------.--------------.--------------.--------------.
| Table name | Column | Type | MinValue | MaxValue | FirstRow |
:------------+--------+--------+--------------+--------------+--------------:
| Table1 | day | date | ‘2010-09-17’ | ‘2016-12-10’ | ‘2016-12-10’ |
:------------+--------+--------+--------------+--------------+--------------:
| Table1 | price | double | 1030.8 | 29485.7 | 6023.8 |
:------------+--------+--------+--------------+--------------+--------------:
| … | | | | | |
:------------+--------+--------+--------------+--------------+--------------:
| TableN | day | date | ‘2014-06-20’ | ‘2016-11-28’ | ‘2016-11-16’ |
:------------+--------+--------+--------------+--------------+--------------:
| TableN | owner | string | NULL | NULL | ‘Joe’ |
'------------'--------'--------'--------------'--------------'--------------'
I think the execution of many queries
SELECT MAX(column_name) as max_value, MIN(column_name) as min_value
FROM table_name
Will be ineffective on the huge tables that are stored in Hadoop.
After reading documentation found an article about "Statistics in Hive"
It seems I must use request like this:
ANALYZE TABLE tablename COMPUTE STATISTICS FOR COLUMNS;
But this command ended with error:
Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.ColumnStatsTask
Do I understand correctly that this request add information to the description of the table and not display the result? Will this request work with view?
Please suggest how to effectively and automatically create documentation for the database in HIVE?
I have an Oracle Forms 6i form with a data block that consists of several columns.
------------------------------------------------------------------------------
| FIRST_NAME | LAST_NAME | DEPARTMENT | BIRTH_DATE | JOIN_DATE | RETIRE_DATE |
------------------------------------------------------------------------------
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
------------------------------------------------------------------------------
The user can press F7 (to Enter in Query Mode, for example, he/she types JOH% in the first_name and H% in the DEPARTMENT field) , then F8 to execute the query and see the results. In this example, a list of all employees with their last name starting with JOH and working in any department starting with H will be listed. Here is a sample output of that query
------------------------------------------------------------------------------
| FIRST_NAME | LAST_NAME | DEPARTMENT | BIRTH_DATE | JOIN_DATE | RETIRE_DATE |
------------------------------------------------------------------------------
| MIKE | JOHN | HUMAN RES. | 05-MAY-82 | 02-FEB-95 | |
| BEN | JOHNATHAN | HOUSING | 23-APR-76 | 16-AUG-98 | |
| SMITH | JOHN | HOUSING | 11-DEC-78 | 30-JUL-91 | |
| | | | | | |
------------------------------------------------------------------------------
I then added a small button on top of each column to allow the user to sort the data by the desired column, by executing WHEN-BUTTON-PRESSED trigger:
set_block_property('dept', order_by, 'first_name desc');
The good news is that the ORDER_BY does change. The bad news is that the user never notice the change because he/she will need to do another query and execute to see the output ordered by the column they selected. In other words, user will only notice the change in the next query he/she will execute.
I tried to automatically execute the query upon changing the ORDER_BY clause like this:
set_block_property('dept', order_by, 'first_name desc');
go_block('EMPLOYEE');
do_key('EXECUTE_QUERY');
/* EXECUTE_QUERY -- same thing */
but what happens is that all data from the table is selected, ignoring the criteria that the user has initially set during the query mode entry.
I also searched for a solution to this problem and most of them deal with SYSTEM.LAST_QUERY and default_where. The problem is, last_query can refer to a different block from a different form, that is not valid on the currently displayed data bloc.
How can do the following in just one button press:
1- Change the ORDER_BY clause of the currently active datablock
and: 2- Execute the last query that the user has executed, using the same criteria that was set?
Any help will be highly appreciated.
You can get the last query of the block with get_block_property built-in function:
GET_BLOCK_PROPERTY('EMPLOYEE', LAST_QUERY);
Another option is to provide separate search field(s) on the form, instead of using the QBE functionality.
I'd like to insert the data after unpivoting it. The statement needs to be a merge statemenet. However, I am getting ora-30926 error, and I can't really figure out how to solve it.
Here the data table:
------------------------------------------------------------------------------------
|Employee_id | work_experience_1 | work_experience_2 | work_experience_3 | language |
-------------------------------------------------------------------------------------
| 123 | C&S | Deloitte | TCS | FI |
| 211 | E&Y | Microsoft | | FI |
| 213 | C&S | | | FI |
-------------------------------------------------------------------------------------
So first before entering the data, I need to unpivot it.
----------------------------------
|Employee_id | work_experience |
----------------------------------
| 123 | C&S |
| 123 | Deloitte |
| 123 | TCS |
| 211 | E&Y |
| 211 | Microsoft |
| 213 | C&S |
----------------------------------
Here is what I have done. The inserting part works ok but updating part fails.
MERGE INTO arc_hrcs.user_multi_work_exp work_exp
USING (SELECT user_id, work_experience_lang, work_exp_fi FROM
(SELECT ext.user_id, tmp_work.employee_id, tmp_work.work_experience_1, tmp_work.work_experience_2, tmp_work.work_experience_3, tmp_work.work_experience_4, tmp_work.work_experience_5, tmp_work.work_experience_6, tmp_work.work_experience_7, tmp_work.work_experience_8, tmp_work.work_experience_9, tmp_work.work_experience_10, tmp_work.work_experience_lang FROM arc_hrcs.hr_extension_data ext
JOIN aa_work_exp_tmp tmp_work ON tmp_work.employee_id = ext.employee_id)
UNPIVOT (work_exp_fi FOR work_code IN (work_experience_1 AS 'a', work_experience_2 AS 'b', work_experience_3 AS 'c', work_experience_4 AS 'd', work_experience_5 AS 'e', work_experience_6 AS 'f', work_experience_7 AS 'g', work_experience_8 AS 'h', work_experience_9 AS 'i', work_experience_10 AS 'j'))) r
ON (work_exp.user_id = r.user_id AND r.work_experience_lang LIKE '%FI%' )
WHEN NOT MATCHED THEN
INSERT (work_exp.user_id, work_exp.work_experience_fi)
VALUES (r.user_id, r.work_exp_fi)
WHEN MATCHED THEN
UPDATE SET work_exp.work_experience_fi = r.work_exp_fi
What can I do to make it working?
Cheers and thx in advance :-)
afaik, the MERGE statement needs UNIQUE or PRIMARY KEY columns specified in the ON clause and also on the target table. Looking at your data sample you are probably missing it (them) on the source table.
Essentially, the query in the USING cause is a multiple-row subquery, when it needs to be a single-row subquery. I would try running the subquery in isolation and attempt to fix the logic of the WHERE cause so that you bring back a unique row.
http://blog.mclaughlinsoftware.com/2010/03/05/stable-set-of-rows/