'thrust::system::system_error' error when running GPGPU database engine Alenka - gpgpu

I am trying to run alenka (https://github.com/antonmks/Alenka) by loading a custom table test.tbl and fire select queries on it.
It works fine with 3 or 4 rows.
But when I increase number of entries beyond 6 or 10 rows, it does not show any error while loading(./alenka load_test.sql), however when i run query(./alenka testquery.sql), it gives an error:
terminate called after throwing an instance of 'thrust::system::system_error' what(): invalid argument Aborted (core dumped)
---test.tbl---
1|2.12345|3|4|5|6|7|
1|2|3|4|5|6|7|
1|2|3|4|5|6|7|
1|2|3|4|5|6|7|
1|2|3|4|5|6|7|
1|2|3|4|5|6|7|
1|2|3|4|5|6|7|
1|2|3|4|5|6|7|
1|2|3|4|5|6|7|
This is the load_test.sql query
A := LOAD 'test.tbl' USING ('|') AS (var1{1}:int, var2{2}:int, var3{4}:int,
var4{5}:int,var5{6}:int, var6{7}:int, var7{8}:int);
STORE A INTO 'test' BINARY;
And testquery.sql
B := FILTER test BY id <= 19980902;
D := SELECT var2 AS var2
FROM B;
STORE D INTO 'mytest.txt' USING ('|');
Can someone explain, what is the reason for such error?
Thank you

The problem was raised due to minor errors which summed up to this confusion.
When a load command is fired on alenka, it creates binary files containing data from each column of the table.
These files will be overwritten if they are load again, however if the column names are changed it would create a new file along side the old ones.
So its a good idea to delete those files after renaming columns in a table in-order to avoid using them again.
Hence, I got this error because I had loaded data with different column names earlier and forgot to delete those files( test.id*) files from its folder.
Along with that, I also committed one more blunder of filtering it with "id" instead of 'var1' in query(testquery.sql) file.
Since the id files had 9 entries(from the previous schema), it ran perfectly for 9 rows but when the database size increased beyond that ,the thrust library threw system error.
Hope this helps someone from wasting time like I did.

Related

How to remove unwanted nested columns?

I've been tasked to alter the company's Event Export from our PlayFab Environment into Azure. Initially, we've set it up to Export all events but after looking at the data we do have some data exported that we don't want for legal reasons. I was exploring the Use custom query method and was trying to build the query to get all data except the columns I want to exclude. The problem is that these columns are nested. I've tried using the project-away query to exclude one column for now but when I run the below query
['events.all']
| project-away EventData.ColumnToExclude
| limit 100
I get this error
I'm assuming it's because it is not supporting nested columns. Is there an easy way to exclude the column without having to flatten the data or list all my columns (our developers might create new events without notice so that won't work)?
UPDATE 1:
I've found that project-away is the syntax to remove a column from a table but what I needed is a way to remove a key from a json/dynamic object so found that using bag_remove_keys() is the correct approach
['events.all']
| project EventData=bag_remove_keys(EventData, dynamic(['Key1', 'Key2', '$.Key3.SubKey1'])
But now I am facing another issue. When I use the '$.' notation for subkeys I get the below error
Query execution has resulted in error (0x8000FFFF): Partial query failure: Catastrophic failure (message: 'PropertyBagEncoder::Add: non-contiguous entries: ', details: '').
[0]Kusto.Data.Exceptions.KustoDataStreamException: Query execution has resulted in error (0x8000FFFF): Partial query failure: Catastrophic failure (message: 'PropertyBagEncoder::Add: non-contiguous entries: ', details: '').
Timestamp=2022-01-31T13:54:56.5237026Z
If I don't list any subkeys I don't get this issue and I can't understand why
UPDATE 2:
I found that bag_remove_keys has a bug. On the below query I get the described error in UPDATE 1
datatable(d:dynamic)
[
dynamic(
{
"test1": "val",
"test2": {},
"test3": "val"
}
)
]
| extend d1=bag_remove_keys(d, dynamic(['$.SomeKey.Sub1', '$.SomeKey.Sub2']))
However, if I move the "test2" key at the end I don't get an error but d1 will not show the "test2" key in the output.
Also, if I have a key in bag_remove_keys() that matches a key from the input like | extend d1=bag_remove_keys(d, dynamic(['$.SomeKey.Sub1', '$.SomeKey.Sub2', 'test1'])) then, again it will not error but will remove "test2" from the output
Thanks for reporting it Andrei, it is a bug and we are working on a fix.
Update - fix had been checked in and will be deployed within two weeks, please open a support ticket if you need it earlier.

Snowflake Copy Command Result Scan Inconsistency

I am using Copy command to load data from CSV file to a table using internal Stage.
After loading data I am using below code to get number of rows loaded and failed.
Select * from table(Result_Scan('Copy_Query_ID'))
I am also using below query to get actual failed records:
select * from table(validate("Table_Name",job_id=>'Copy_Query_ID'))
it worked fine few times. But I noticed today that first query shows as below:
Rows_Parsed Rows_Loaded Error_Seen
10000 9600 400
So I was expecting 400 rows in second query result but instead I see 10400 records:
All rows once and additional 400 records for some other errors. If all rows are error rows then why are they loaded? Can i not use this queries for this purpose?
Note- In my file I have 6 fields but I am using only 4 of them in Copy and rest two fields I am getting using SYSdate(), may be this is the reason for mismatch?
Copy into table(col1,col2,col3,col4,col5,col6) from ( select $1,$2,$3,$4, sysdate(),'10/20/2020' from %#table)
so I am guessing validate is not looking at my new values for field 5,6 instead it is taking these values from file?

QGIS 3.14 : Inserting into Oracle database does not work when metadata is set with three dimensions

I´m trying to set up an Oracle Connection from QGIS. The tables are set with metadata for Three dimensions in Oracle. When I try to add objects to the database, however, I get the following error:
Oracle error while adding features: Oracle error: Could not insert feature -27
SQL: ORA-29875: failed in the execution of the ODCIINDEXINSERT routine
ORA-13364: layer dimensionality does not match geometry dimensions
ORA-06512: at "MDSYS.SDO_INDEX_METHOD_10I", line 976
ORA-06512: at "MDSYS.SDO_INDEX_METHOD_10I", line 488
Unable to execute statement
When I remove the Z-dimension from the metadata it seems to work. Any help on what can resolve the problem is appreciated.
The Z-value is usually added by convention at the organization and not always used, i.e. set to 0.
ironic, I was actually having a similar error today when attempting to save data into a geopackage. (That is, i don't get the error on insert, i get it when hitting save).
I got a slightly different error:
'QGIS error creating feature -16: failed to prepare SQL: INSERT INTO..... etc.
However my error was related to the fact that the geometry attribute name had somehow been created as 'geom'....and the GDAL code is looking for a geometry attribute called 'geometry'.
I recreated my geopackage table, with the attribute named 'geometry' and z values, and no more problems, (even if source data has no z).
however I don't think this is your problem - it could be that source data doesn't have a z - You could try maybe setting a procedure somewhere or derived field, such that z value is populated upon insert.
The error code makes it sounds like either the source has no z value (ie: Geometry dimensions) or the layer indexing is not setup to handle z (ie: ODCIINDEXINSERT routine layer dimensionality).
That may be the starting point for yourself.

ORA-01861 literal does not match format error when running within larger query

I'm a newbie using Oracle SQL Developer, and running into some issues. When I run the following smaller sample query by itself, it runs fine:
SELECT ((SYSDATE - TO_DATE(table1.a_certain_date_field, 'DD-MON-RR')) / 365.0)
FROM &a_table
WHERE rownum < 5 AND
((SYSDATE - TO_DATE(table1.a_certain_date_field, 'DD-MON-RR')) > 365.0);
But once I embed this into a larger query, I get an ORA-01861 error: literal does not match format string. For example, when I include it with other WHERE conditions, such as
...AND NOT (adr.state = 'WA')
AND
((SYSDATE - TO_DATE(table1.a_certain_date_field, 'DD-MON-RR')) > 365.0)...[more conditions]
What are some reasons why this would occur? My guess is that the table field changed the way that dates were inputted (ie. from DD-MM-YYYY to DD-MON-RR) at some point and caused this error to be thrown, but wouldn't take likewise throw a similar error in the smaller sample code?
This is always (in my experience) a data issue. you can try to narrow it down by looking at various subsets of the data to find out which row/rows are causing it.

Insert VS (Select and Insert)

I am writing a simple program to insert rows into a table.But when i started writing the program i got a doubt. In my program i will get duplicate input some times. That time i have to notify the user that this already exists.
Which of the Following Approaches is good to Use to achieve this
Directly Perform Insert statement will get the primary key violation error if it is duplicate notify otherwise it will be inserted. One Query to Perform
First make a search for the primary key values. If found a Value Prompt User. Otherwise perform insert operation.For a non-duplicate row this approach takes 2 queries.
Please let me know trade-offs between these approaches. Which one is best to follow ?
Regards,
Sunny.
I would choose the 2nd approach.
The first one would cause an exception to be thrown which is known to be very expensive...
The 2nd approach would use a SELECT count(*) FROM mytable WHERE key = userinput which will be very fast and the INSERT statement for which you can use the same DB connection object (assuming OO ;) ).
Using prepared statements will pre-optimize the queries and I think that will make the 2nd approach much better and mre flexible than the first one.
EDIT: depending on your DBMS you can also use a if not exists clause
EDIT2: I think Java would throw a SQLExcpetion no matter what went wrong, i.e. using the 1st approach you wouldn't be able to differ between a duplicate entry or an unavailable database without having to parse the error message - which is again a point for using SELECT+INSERT (or if not exists)

Resources