ERROR: relation "table" does not exist, even though both the database and the table exist - cockroachdb

I'm using cockroachdb, which is essentially a superset of postgres and I can't understand why the statement:
select * from _a66df261120b6c23.tabDefaultValue;
results sin the error:
ERROR: relation "_a66df261120b6c23.tabdefaultvalue" does not exist
show databases gives me:
database_name | owner | primary_region | regions | survival_goal
--------------------+-------+----------------+---------+----------------
_a66df261120b6c23 | root | NULL | {} | NULL
defaultdb | root | NULL | {} | NULL
postgres | root | NULL | {} | NULL
root | root | NULL | {} | NULL
sammy | root | NULL | {} | NULL
system | node | NULL | {} | NULL
test123 | root | NULL | {} | NULL
test3 | root | NULL | {} | NULL
test4 | root | NULL | {} | NULL
test5 | root | NULL | {} | NULL
test9 | root | NULL | {} | NULL
and show tables from _a66df261120b6c23 gives me:
schema_name | table_name | type | owner | estimated_row_count | locality
--------------+-------------------+-------+-------+---------------------+-----------
public | __Auth | table | root | 0 | NULL
public | tabDefaultValue | table | root | 0 | NULL
The database and the table both exist, so why does select * from _a66df261120b6c23.tabDefaultValue; fail? Strange thing is when I run \dt all I get is:
schema_name | table_name | type | owner | estimated_row_count | locality
--------------+------------+-------+-------+---------------------+-----------
public | sammy | table | root | 0 | NULL
How do I actually get the select statement to work? Thank you

SQL statements are case sensitive and incoming identifiers are lowercased unless specifically forced with double quotes.
This means that to refer to your tabDefaultValue, you need to use the following statement:
select * from _a66df261120b6c23."tabDefaultValue";
Note that the quotes are around the table name only, if you quote dbname.tablename together, this will be considered a single identifier with a dot in the middle.

Related

How to drop hive partitions with hivevar passed as partition variable?

I have been trying to run this piece of code to drop current day's partition from hive a table and for some reason it does not drop the partition from the hive table. Not sure what's worng.
Table Name : prod_db.products
desc:
+----------------------------+-----------------------+-----------------------+--+
| col_name | data_type | comment |
+----------------------------+-----------------------+-----------------------+--+
| name | string | |
| cost | double | |
| load_date | string | |
| | NULL | NULL |
| # Partition Information | NULL | NULL |
| # col_name | data_type | comment |
| | NULL | NULL |
| load_date | string | |
+----------------------------+-----------------------+-----------------------+--+
## I am using the following code
SET hivevar:current_date=current_date();
ALTER TABLE prod_db.products DROP PARTITION(load_date='${current_date}');
Before and After picture of partitions:
+-----------------------+--+
| partition |
+-----------------------+--+
| load_date=2022-04-07 |
| load_date=2022-04-11 |
| load_date=2022-04-18 |
| load_date=2022-04-25 |
+-----------------------+--+
It runs without any error but doesn't work but won't drop the partition. Table is internal/managed.
I tried different ways mentioned on stack but it is just not working for me.
Help.
You dont need to set a variable. You can directly drop using direct sql.
Alter table prod_db.products
drop partition (load_date= current_date());

Sqoop import having SQL query with where clause and parallel processing

I have a table as below in mysql :
Order_Details :
+---------+------------+-------------------+--------------+
| orderid | order_date | order_customer_id | order_status |
+---------+------------+-------------------+--------------+
| A001 | 10/30/2018 | C003 | Completed |
| A002 | 10/30/2018 | C005 | Completed |
| A451 | 11/02/2018 | C376 | Pending |
| P9209 | 10/30/2018 | C234 | Completed |
| P92099 | 10/30/2018 | C244 | Pending |
| P9210 | 10/30/2018 | C035 | Completed |
| P92398 | 10/30/2018 | C346 | Pending |
| P9302 | 10/30/2018 | C034 | Completed |
+---------+------------+-------------------+--------------+
and the description as below :
mysql> desc Order_Details_Sankha;
+-------------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+-------------+------+-----+---------+-------+
| orderid | varchar(20) | NO | PRI | | |
| order_date | varchar(20) | YES | | NULL | |
| order_customer_id | varchar(20) | YES | | NULL | |
| order_status | varchar(20) | YES | | NULL | |
+-------------------+-------------+------+-----+---------+-------+
I am using the below sqoop import with parallel processing :
sqoop import
--connect jdbc:mysql://ip-10-0-1-10.ec2.internal/06july2018_new
--username labuser
--password abc123
--driver com.mysql.jdbc.Driver
--query "select * from Order_Details where order_date = '10/30/2018' AND \$CONDITIONS"
--target-dir /user/sankha087_gmail_com/outputs/EMP_Sankha_1112201888
--split-by ","
--m 3
and I am getting the below error message
18/12/15 17:15:14 WARN security.UserGroupInformation: PriviledgedActionException as:sankha087_gmail_com (auth:SIMPLE) cause:java.io.IOException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You hav
e an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '), MAX(,) FROM (select * from Order_Details_Sankha where order_date = '10/30/201' a
t line 1
18/12/15 17:15:14 ERROR tool.ImportTool: Import failed: java.io.IOException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use near '), MAX(,) FROM (select * from Order_Details_Sankha where order_date = '10/30/201' at line 1
at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:207)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
Please advise what needs to be changed in my import statement .
Sqoop parallel execution doesn't happen with vertical split, it happens with horizontal split.
--split-by should be a column name. column should be one which is evenly distributed.
https://sqoop.apache.org/docs/1.4.0-incubating/SqoopUserGuide.html#id1765770
Read: 7.2.4. Controlling Parallelism

Return filtered records from a returned set of data from two tables

I have three tables:
- Venue
- Space (belongs to Venue)
- Included Space (belongs to Space)
I receive the id of a Venue in the route and return all the related spaces that I know have Included Spaces(a field called num_included_spaces__c on the Space record that maintains a count of its children). Now that I have all the related parent Spaces for that Venue, I need to find all of the Included Spaces for them.
An Included Space is still a Space, it just happens to have a parent that resides in the same table. I'm trying to turn this:
Venue = Rockdog
- Space = Upstairs
- Space = Media Room
- Space = Courtyard
- Space = Downstairs
- Space = Front Patio
- Space = Indoor Bar
Into this:
Venue = Rockdog
- Space = Upstairs
-- Included Space = Media Room
-- Included Space = Courtyard
- Space = Downstairs
-- Included Space = Front Patio
-- Included Space = Indoor Bar
The Included Spaces table has belongs_to__c and space__c as fields, where belongs_to__c is the id of the parent space and space__c is the id of the child. So i'm looking to find all the Included Spaces where belongs_to_c matches the id of any #spaces returned below
#sub_spaces = Space.where("venue__c = ? AND num_included_spaces__c = ?", params[:venue],0)
#spaces = Space.where("venue__c = ? AND num_included_spaces__c > ?", params[:venue],0)
How would I write this Active Record Query for #included_spaces?
my database schema.
mysql> describe venues;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| created_at | datetime | NO | | NULL | |
| updated_at | datetime | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
4 rows in set (0,00 sec)
mysql> describe spaces;;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| venue_id | int(11) | YES | | NULL | |
| created_at | datetime | NO | | NULL | |
| updated_at | datetime | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
5 rows in set (0,00 sec)
ERROR:
No query specified
mysql> describe included_spaces;;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| space_id | int(11) | YES | | NULL | |
| created_at | datetime | NO | | NULL | |
| updated_at | datetime | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
5 rows in set (0,00 sec)
ERROR:
No query specified
Below function will somehow give you the result you need (in console ofcourse ) however it's not a good solution - it queries database more than needed. However it is the easy -))
def foo id
v = Venue.find(id)
puts v.name
v.spaces.each do |space|
puts space.name
space.included_spaces.each do |spis|
puts spis.name
end
end
end
You can also try a more complex query sth like,
mysql> SELECT spaces.name, included_spaces.name FROM `spaces` INNER JOIN `venues` ON `venues`.`id` = `spaces`.`venue_id` INNER JOIN `included_spaces` ON `included_spaces`.`space_id` = `spaces`.`id` WHERE `spaces`.`venue_id` = 1
-> ;
+------------+-----------+
| name | name |
+------------+-----------+
| Upstairs, | Front |
| Upstairs, | Patio, |
| Upstairs, | Indoor |
| Upstairs, | Bar |
| Downstairs | Media |
| Downstairs | Room, |
| Downstairs | Courtyard |
+------------+-----------+
7 rows in set (0,00 sec)
which should be translated to active record as
Space.joins(:venue)
.joins(:included_spaces)
.where(venue_id: 1)
.select('spaces.name, included_spaces.name')

Materialized View having UNKNOWN staleness - Oracle 11G

I am working on Oracle 11G.
One of my Materialized view has become UNKNOWN (MY_MAT_VW1). You can check the output of the ALL_MVIEWS below.
OWNER | MVIEW_NAME | CONTAINER_NAME | QUERY | QUERY_LEN | UPDATABLE | UPDATE_LOG | MASTER_ROLLBACK_SEG | MASTER_LINK | REWRITE_ENABLED | REWRITE_CAPABILITY | REFRESH_MODE | REFRESH_METHOD | BUILD_MODE | FAST_REFRESHABLE | LAST_REFRESH_TYPE | LAST_REFRESH_DATE | STALENESS | AFTER_FAST_REFRESH | UNKNOWN_PREBUILT | UNKNOWN_PLSQL_FUNC | UNKNOWN_EXTERNAL_TABLE | UNKNOWN_CONSIDER_FRESH | UNKNOWN_IMPORT | UNKNOWN_TRUSTED_FD | COMPILE_STATE | USE_NO_INDEX | STALE_SINCE | NUM_PCT_TABLES | NUM_FRESH_PCT_REGIONS | NUM_STALE_PCT_REGIONS
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MY_DB | MY_MAT_VW1 | MY_MAT_VW1 | select.. | 6728 | N | | | | N | GENERAL | DEMAND | COMPLETE | IMMEDIATE | NO | COMPLETE | 14-Nov-16 | UNKNOWN | NA | N | Y | N | N | N | N | VALID | N | 0 | | |
MY_DB | MY_MAT_VW2 | MY_MAT_VW2 | select.. | 7074 | N | | | | N | TEXTMATCH | DEMAND | COMPLETE | IMMEDIATE | NO | COMPLETE | 13-Nov-16 | FRESH | NA | N | N | N | N | N | N | FRESH | N | 0 | 0 | |
The queries for the materialized view contain complex joins between multiple tables, inline views and unions.
As per my understanding (UNKNOWN_PLSQL_FUNC column) I guess there is a PLSQL Function which is causing the staleness to become UNKNOWN. However I am not sure which one.
I tried re-compiling and refreshing it but no luck.
Can anyone provide me some information on how to detect the root cause and make sure it does not become UNKNOWN again.
Also is there any implication of it on the data stored within it?
Below is just a sample I've created to replicate the scenario.
SELECT * FROM ENTITY_T;
ID | ENTITY_TYPE | FIRST_NAME | LAST_NAME | LEGAL_NAME
--------------------------------------------------
1 | INDIVIDUAL | JOHN | LESSEN |
2 | INDIVIDUAL | ROSAN | MEL |
3 | CORP | SIGMA | | SIGMA CORPORATION
--Function to get name base upon type
CREATE OR REPLACE FUNCTION GET_NAME (P_ID IN NUMBER)
RETURN VARCHAR2
DETERMINISTIC
AS
LV_NAME VARCHAR2(200);
BEGIN
SELECT CASE ENTITY_TYPE WHEN 'INDIVIDUAL' THEN FIRST_NAME ||' '|| LAST_NAME
WHEN 'CORP' THEN LEGAL_NAME
ELSE 'NONE'
END INTO LV_NAME
FROM ENTITY_T
WHERE ID=P_ID;
RETURN LV_NAME;
EXCEPTION
WHEN NO_DATA_FOUND THEN
RETURN 'NO ID FOUND';
WHEN OTHERS THEN
RETURN 'OTHER ERROR';
END;
--Materialized view creation
CREATE MATERIALIZED VIEW TEST_MV
AS
SELECT ID,ENTITY_TYPE,GET_NAME(ID) NAME
FROM ENTITY_T;
SELECT MVIEW_NAME,STALENESS,AFTER_FAST_REFRESH,UNKNOWN_PLSQL_FUNC,COMPILE_STATE,STALE_SINCE
FROM ALL_MVIEWS WHERE MVIEW_NAME='TEST_MV';
MVIEW_NAME | STALENESS | AFTER_FAST_REFRESH | UNKNOWN_PLSQL_FUNC | COMPILE_STATE | STALE_SINCE
----------------------------------------------------------------------------------------------
TEST_MV | UNKNOWN | NA | Y | VALID |
The Oracle Issue/Doc ID 757537.1 mentioned by JSapkota states clearly, that this is not a bug, but correct/expected behaviour:
STALENESS of the mview, refering to PL/SQL function is set to UNKOWN
as one cannot determine PL/SQL function changes. Current behaviour is
correct as per the design & code.
I guess using DETERMINISTIC functions instead of the default scope could prevent it.
As per the My Oracle Support this could be a bug(7582462).
As there is no solution to this bug, you have to deal with fact that staleness will show unknown, or not use functions on Materialized View definition.
Reference:DBA_MVIEWS Shows STALENESS Value of UNKNOWN After Refresh (Doc ID 757537.1)

How to use ResultSet to fetch the ID of the record

I have got a table with name table_listnames whose structure is given below
mysql> desc table_listnames;
+-------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | NO | | NULL | |
+-------+--------------+------+-----+---------+----------------+
2 rows in set (0.04 sec)
It has got sample data as shown
mysql> select * from table_listnames;
+----+------------+
| id | name |
+----+------------+
| 6 | WWW |
| 7 | WWWwww |
| 8 | WWWwwws |
| 9 | WWWwwwsSSS |
| 10 | asdsda |
+----+------------+
5 rows in set (0.00 sec)
I have a requirement where if name not found under the table , i need to insert or else do nothing
I am achieving it this way
String sql = "INSERT INTO table_listnames (name) SELECT name FROM (SELECT ?) AS tmp WHERE NOT EXISTS (SELECT name FROM table_listnames WHERE name = ?) LIMIT 1";
pst = dbConnection.prepareStatement(sql);
pst.setString(1, salesName);
pst.setString(2, salesName);
pst.executeUpdate();
Is it possible to know the id of the record of the given name in this case

Resources