After executing a pg_dump from PostgeSQL, I attempted to import the .sql file into CockroachDB, but received the following errors:
ERROR: unknown variable: "STATEMENT_TIMEOUT"
ERROR: unknown variable: "LOCK_TIMEOUT"
ERROR: unknown variable: "IDLE_IN_TRANSACTION_SESSION_TIMEOUT"
SET
SET
ERROR: unknown variable: "CHECK_FUNCTION_BODIES"
SET
ERROR: unknown variable: "ROW_SECURITY"
SET
ERROR: unknown variable: "DEFAULT_TABLESPACE"
ERROR: unknown variable: "DEFAULT_WITH_OIDS"
CREATE TABLE
ERROR: syntax error at or near "OWNER"
Any guidance?
CockroachDB has special support for using psql, which supports the COPY command (which is faster than batched INSERT statements).
You'll need to do two things:
Clean up the SQL file
Import it into CockroachDB (which sounds like you tried, but I'll include the steps here for anyone else who needs them):
Clean up the SQL File
After generating the .sql file, you need to perform a few editing steps before importing it:
Remove all statements from the file besides the CREATE TABLE and COPY statements.
Manually add the table's PRIMARY KEY constraint to the CREATE TABLE statement.
This has to be done manually because PostgreSQL attempts to add the primary key after creating the table, but CockroachDB requires the primary key be defined upon table creation.
Review any other constraints to ensure they're properly listed on the table.
Remove any unsupported elements, such as arrays.
Import Data
After reformatting the file, you can import it through psql:
$ psql -p [port] -h [node host] -d [database] -U [user] < [file name].sql
For reference, CockroachDB uses these defaults:
[port]: 26257
[user]: root
in greenplum how to configure to generate gphdfs.so?
when I
postgres=# CREATE EXTERNAL TABLE csv_hdfs_lineitem (like a) LOCATION ( 'gphdfs://xxxxx/gptest/lineitem.csv' ) FORMAT 'text' (delimiter E'|' null E'\N' escape E'off' fill missing fields) ENCODING 'UTF8' ;
it shows
ERROR: protocol "gphdfs" does not exist
You need to run a SQL Script included in Greenplum Package. You will find the same script in $GPHOME/share/postgresql/cdb_external_extensions.sql
Run this:
psql -f $GPHOME/share/postgresql/cdb_external_extensions.sql
Hope this helps..
My hive table name is in the below format:
schema_name.hive_table_name
eg: schema1.abc;
Now when I try to do MSCK repair table on the above hive table it throws below error.
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
FAILED: ParseException line 1:28 missing EOF at '.' near 'schema_name'
Below is the command I used:
hive -e "MSCK repair table schema_name.hive_table_name"
Could any one help on this?
I tried the below statement:
hive -e "use schema_name;MSCK repair table hive_table_name"
This allows to add partition to hive with the specific schema mentioned .
It worked for me.
Thanks
I am not sure what I am doing wrong here:
hive> CREATE TABLE default.testtbl(int1 INT,string1 STRING)
stored as orc
tblproperties ("orc.compress"="NONE")
LOCATION "/user/hive/test_table";
FAILED: ParseException line 1:107 missing EOF at 'LOCATION' near ')'
while the following query works perfectly fine:
hive> CREATE TABLE default.testtbl(int1 INT,string1 STRING)
stored as orc
tblproperties ("orc.compress"="NONE");
OK
Time taken: 0.106 seconds
Am I missing something here. Any pointers will help. Thanks!
Try put the "LOCATION" in front of "tblproperties" like below, worked for me.
CREATE TABLE default.testtbl(int1 INT,string1 STRING)
stored as orc
LOCATION "/user/hive/test_table"
tblproperties ("orc.compress"="NONE");
It seems even the sample SQL from book "Programming Hive" got the order wrong. Please reference to the official definition of create table command:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTable
#Haiying Wang pointed out that LOCATION is to be put in front of tblproperties.
But I think the error also occurs when location is specified above stored as.
Its better to stick to the correct order:
CREATE [TEMPORARY] [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name -- (Note: TEMPORARY available in Hive 0.14.0 and later)
[(col_name data_type [COMMENT col_comment], ... [constraint_specification])]
[COMMENT table_comment]
[PARTITIONED BY (col_name data_type [COMMENT col_comment], ...)]
[CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name [ASC|DESC], ...)] INTO num_buckets BUCKETS]
[SKEWED BY (col_name, col_name, ...) -- (Note: Available in Hive 0.10.0 and later)]
ON ((col_value, col_value, ...), (col_value, col_value, ...), ...)
[STORED AS DIRECTORIES]
[
[ROW FORMAT row_format]
[STORED AS file_format]
| STORED BY 'storage.handler.class.name' [WITH SERDEPROPERTIES (...)] -- (Note: Available in Hive 0.6.0 and later)
]
[LOCATION hdfs_path]
[TBLPROPERTIES (property_name=property_value, ...)] -- (Note: Available in Hive 0.6.0 and later)
[AS select_statement]; -- (Note: Available in Hive 0.5.0 and later; not supported for external tables)
Refer: Hive Create Table
Check this post:
Loading Data from a .txt file to Table Stored as ORC in Hive
And check your source files present at the specified directory /user/hive/test_table. Incase the files are in .txt or some other non ORC format then you can follow the steps in the above post to come out of the error.
ParseException line lineNumber missing EOF at '.' near 'schemaName':
Got the above error while trying to execute the following command from linux script to truncate a hive table
dse -u username -p password hive -e "truncate table keyspace.tablename;"
Fix:
Need to separate the commands within the script line as follows -
dse -u username -p password hive -e "use keyspace; truncate table keyspace.tablename;"
Happy coding!
Got the same error while creating a table in hive.
I used the drop command to drop the table and then run the create table command that I had again.
Worked for me.
If you see this error when running the HiveQL from a file with the command "hive -f file.hql". And that it points the first line of your query most definitely this is because of a forgotten semicolon(;) for a previous query.
Since parser looks for semicolon(;) as a terminator for each query.
for example:
DROP TABLE IF EXISTS default.emp
create table default.emp (
field1 type,
field2 type)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
STORED AS TEXTFILE
LOCATION 's3://gts-promocube/source-data/Lowes/POS/';
If you save the above in a file and execute it with hive -f, then you'll get the error:
FAILED: ParseException line 2:0 missing EOF at 'CREATE' near emp.
Solution: Put a semicolon(;) for the DROP TABLE command above.
I ran the below create script and it created the table:-
Create writable external table FLTR (like dbname.FLTR)
LOCATION ('gpfdist://172.90.38.190:8081/fltr.out')
FORMAT 'CSV' (DELIMITER ',' NULL '')
DISTRIBUTED BY (fltr_key);
But when I tried inserting into the file like insert into fltr.out select * from dbname.fltr
I got the below error, cannot find server connection.
Please help me out
I think your gpfdist is probably not running try:
gpfdist -p 8081 -l ~/gpfdist.log -d ~/ &
on 172.90.38.190.
This will start gpfidist using your home directory as the data directory.
When I do that my inserts work and create a file ~/fltr.out