I have docker container with clickhouse database. And I have backups from our production. But I tried many cases with restore. When I using command (in container): clickhouse-client --format Raw --file 21_02_04_db.backup, I get error Bad arguments: unrecognised option '--file'. Okay, I tried without this option: clickhouse-client --format Raw < 21_02_04_db.backup and I get an error:
Code: 62. DB::Exception: Syntax error: failed at position 1 ('') (line 1, col 1): �TimDateTime�#�]�{�]�k�]u �]0&�] 9�]�-�]��]�I�]U��]���]yy�]���]�\�]�\�]��]{B�]�j�]v)�]�d�]J��]�F�]�]�]��]t%�]t%�]���]��]F?�]�B�]��]W3�]��]E��]P��]a�. Unrecognized token: '' I use other backups but in many cases I get similar error. Maybe someone tried restore backup clickhouse with help binary data? I try many cases but I didn't search solution.
it looks like a Native format
it should something like
clickhouse-client -q 'insert into mytable format Native' < 21_02_04_db.backup
Related
When importing a file into Greenplum,one lines fails,and the whole file is not imported successfully.Is there a way can skip the wrong line and import other data into Greenplum successfully?
Here are my SQL execution and error messages:
copy cjh_test from '/gp_wkspace/outputs/base_tables/error_data_test.csv' using delimiters ',';
ERROR: invalid input syntax for integer: "FE00F760B39BD3756BCFF30000000600"
CONTEXT: COPY cjh_test, line 81, column local_city: "FE00F760B39BD3756BCFF30000000600"
Greenplum has an extension to the COPY command that lets you log errors and set up a certain amount of errors that can occur that won't stop the load. Here is an example from the documentation for the COPY command:
COPY sales FROM '/home/usr1/sql/sales_data' LOG ERRORS
SEGMENT REJECT LIMIT 10 ROWS;
That tells COPY that 10 bad rows can be ignored without stopping the load. The reject limit can be # of rows or a percentage of the load file. You can check the full syntax in psql with: \h copy
If you are loading a very large file into Greenplum, I would suggest looking at gpload or gpfdist (which also support the segment reject limit syntax). COPY is single threaded through the master server where gpload/gpfdist load the data in parallel to all segments. COPY will be faster for smaller load files and the others will be faster for millions of rows in a load file(s).
I am trying to move data from a oracle instance to postgres RDS using DMS. I am only doing a full load operation and I have disabled all the foreign keys on the target. I also made sure that the datatypes are not mismatched between columns for the same tables. I tried both 'Do Nothing' and 'Truncate' for the Target Table preparation mode and when I run the task, several tables are failing with below error messages:
[TARGET_LOAD ]E: Command failed to load data with exit error code 1, Command output: <truncated> [1020403] (csv_target.c:981)
[TARGET_LOAD ]E: Failed to wait for previous run [1020403] (csv_target.c:1578)
[TARGET_LOAD ]E: Failed to load data from csv file. [1020403] (odbc_endpoint_imp.c:5648)
[TARGET_LOAD ]E: Handling End of table 'public'.'SKEWED_VALUES' loading failed by subtask 6 thread 1 [1020403] (endpointshell.c:2416)
DMS doesn't give the correct error information and I am not able to understand what the above error messages mean.
When I use 'Drop tables on target' for the Target table preparation mode, it works but it creates the datatypes of the columns in a different way which I don't want.
Any help would be appreciated.
To troubleshoot my case, I created a copy of the task that only loaded the one problem table, and upped all the logging severities to "Detailed debug". Then I was able to see this:
[TARGET_LOAD ]D: RetCode: SQL_SUCCESS_WITH_INFO SqlState: 42622 NativeError: -1 Message: NOTICE: identifier "diagnosticinterpretationrequestdata_diagnosticinterpretationcode" will be truncated to "diagnosticinterpretationrequestdata_diagnosticinterpretationcod" (ar_odbc_stmt.c:4720)
In the RDS logs for the target DB I found:
2021-10-11 14:30:36 UTC:...:[19259]:ERROR: invalid input syntax for integer: ""
2021-10-11 14:30:36 UTC:...:[19259]:CONTEXT: COPY diagnosticinterpretationrequest, line 1, column diagnosticinterpretationrequestdata_diagnosticinterpretationcod: ""
2021-10-11 14:30:36 UTC:...:[19259]:STATEMENT: COPY "myschema"."diagnosticinterpretationrequest" FROM STDIN WITH DELIMITER ',' CSV NULL 'attNULL' ESCAPE '\'
I found that if I added a table mapping rule to explicitly rename the column to truncate the name within Postgres's limit for identifier length, then things ran ok.
{
"rule-type": "transformation",
"rule-id": "1",
"rule-name": "1",
"rule-target": "column",
"object-locator": {
"schema-name": "%",
"table-name": "%",
"column-name": "diagnosticinterpretationrequestdata_diagnosticinterpretationcode"
},
"rule-action": "rename",
"value": "diagnosticinterpretationrequestdata_diagnosticinterpretationcod",
"old-value": null
},
After executing a pg_dump from PostgeSQL, I attempted to import the .sql file into CockroachDB, but received the following errors:
ERROR: unknown variable: "STATEMENT_TIMEOUT"
ERROR: unknown variable: "LOCK_TIMEOUT"
ERROR: unknown variable: "IDLE_IN_TRANSACTION_SESSION_TIMEOUT"
SET
SET
ERROR: unknown variable: "CHECK_FUNCTION_BODIES"
SET
ERROR: unknown variable: "ROW_SECURITY"
SET
ERROR: unknown variable: "DEFAULT_TABLESPACE"
ERROR: unknown variable: "DEFAULT_WITH_OIDS"
CREATE TABLE
ERROR: syntax error at or near "OWNER"
Any guidance?
CockroachDB has special support for using psql, which supports the COPY command (which is faster than batched INSERT statements).
You'll need to do two things:
Clean up the SQL file
Import it into CockroachDB (which sounds like you tried, but I'll include the steps here for anyone else who needs them):
Clean up the SQL File
After generating the .sql file, you need to perform a few editing steps before importing it:
Remove all statements from the file besides the CREATE TABLE and COPY statements.
Manually add the table's PRIMARY KEY constraint to the CREATE TABLE statement.
This has to be done manually because PostgreSQL attempts to add the primary key after creating the table, but CockroachDB requires the primary key be defined upon table creation.
Review any other constraints to ensure they're properly listed on the table.
Remove any unsupported elements, such as arrays.
Import Data
After reformatting the file, you can import it through psql:
$ psql -p [port] -h [node host] -d [database] -U [user] < [file name].sql
For reference, CockroachDB uses these defaults:
[port]: 26257
[user]: root
I am trying to do a bulk insert into tables from a CSV file using Oracle11. My problem is that the database is on a remote machine which I can sqlpl to using this:
sqlpl username#oracle.machineName
Unfortunately the sqlldr has trouble connecting using the following command:
sqlldr userid=userName/PW#machinename control=BULK_LOAD_CSV_DATA.ctl log=sqlldr.log
Error is:
Message 2100 not found; No message file for product=RDBMS, facility=ULMessage 2100 not found; No message file for product=RDBMS, facility=UL
Now having given up on this approach I tried writing a basic sql script, but I am unsure of the proper Oracle keyword for BULK. I know this works in MySql but I get:
unknown command beginning "BULK INSER..."
When running the script:
BULK INSERT <TABLE_NAME>
FROM 'CSVFILE.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
I don't care which one works! Either one will do, I just need a little help.
Sorry I am a dumb dumb! I forgot to add oracle/bin to my path!
If you have found this post, add the bin directory to your path (linux) using the following commands:
export ORACLE_HOME=/path/to/oracle/client
export PATH=$PATH:$ORACLE_HOME/bin
Sorry if I wasted anyone's time ....
For some reason I am having trouble with the make test statement while installing ```Vowpal Wabbit``. I am getting the following error:
RunTests: test 59: '/usr/bin/timeout 20 ../vowpalwabbit/vw -d train-sets/argmax_data -k -c --passes 20 --search_rollout oracle --search_alpha 1e-8 --search_task argmax --search 2 --holdout_off' failed (exitcode=1)
Anyone have a clue what this could be?
Just run the command which failed (in single quotes) directly from the test directory, and the reason would become obvious.
It is missing data file:
Reading datafile = test/train-sets/argmax_data
can't open: test/train-sets/argmax_data, error = No such file or directory
vw: std::exception
The issue was introduced in a recent check-in and should soon be fixed (hopefully).
Update (2014-05-31): fixed in the most recent commit.