I am creating new partition of type "LIST COLUMNS" to the table in mysql.
I also need to add the comment to the partition as some identification.
Can someone tell me the mysql syntax that how can I do this?
Atul
I found the syntax. It is not a big thing but even in documentation the syntax is not provided so struggled a bit on it.
Anyway the problem has been resolved and the syntax to add comment to the partition while creating is:
CONCAT('ALTER TABLE <tableName> ADD PARTITION (PARTITION ',<partitionName>,' VALUES in ( ',<value>,' ) COMMENT = "',<anyComment>,'")');
This is just when you want to concat something. The plain syntax is:
ALTER TABLE <tableName> ADD PARTITION (PARTITION <partitionName> VALUES in ( <value> ) COMMENT = "<anyComment>");
Related
I am new to Hive and have some problems. I try to find a answer here and other sites but with no luck... I also tried many different querys that come to my mind, also without success.
I have my source table and i want to create new table like this.
Were:
id would be number of distinct counties as auto increment numbers and primary key
counties as distinct names of counties (from source table)
You could follow this approach.
A CTAS(Create Table As Select)
with your example this CTAS could work
CREATE TABLE t_county
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
STORED AS TEXTFILE AS
WITH t AS(
SELECT DISTINCT county, ROW_NUMBER() OVER() AS id
FROM counties)
SELECT id, county
FROM t;
You cannot have primary key or foreign keys on Hive as you have primary key on RBDMSs like Oracle or MySql because Hive is schema on read instead of schema on write like Oracle so you cannot implement constraints of any kind on Hive.
I can not give you the exact answer because of it suppose to you must try to do it by yourself and then if you have a problem or a doubt come here and tell us. But, what i can tell you is that you can use the insertstatement to create a new table using data from another table, I.E:
create table CARS (name string);
insert table CARS select x, y from TABLE_2;
You can also use the overwrite statement if you desire to delete all the existing data that you have inside that table (CARS).
So, the operation will be
CREATE TABLE ==> INSERT OPERATION (OVERWRITE?) + QUERY OPERATION
Hive is not an RDBMS database, so there is no concept of primary key or foreign key.
But you can add auto increment column in Hive. Please try as:
Create table new_table as
select reflect("java.util.UUID", "randomUUID") id, countries from my_source_table;
Is there any way to ALTER a table and change the column name in clickhouse?
I only found to change tha table name but not for an individual column in a straight forward way.
Thanks.
The feature has been introduced here into v20.4.
ALTER TABLE table1 RENAME COLUMN old_name TO new_name
You can also rename multiple columns at on:
ALTER TABLE table1
RENAME COLUMN old_name1 TO new_name1,
RENAME COLUMN old_name2 TO new_name2
Old answer:
ClickHouse doesn't have that feature yet.
Implementation is not trivial, because ALTERs that changing columns
are processed outside of usual replication queue, and adding rename
without reworking of ALTERs will introduce race conditions in
replicated tables.
https://github.com/yandex/ClickHouse/issues/146#issuecomment-255631384
As #Slash said, the solution for now is to create new table and
INSERT INTO `new_table` SELECT * FROM `old_table`
Do not forget that column aliasing won't work there (AS).
INSERT INTO `new_table` SELECT a, b AS c, c AS b FROM `old_table`
That will still insert a into first column, b into second column and c into third column. AS has no effect there.
You can try use CREATE TABLE new_table with another field name
and run INSERT INTO new_table SELECT old_field AS new_field FROM old_table
If you created the table using Engine=log, it won't allow you to alter or rename the column.
connection_string = f'clickhouse://{username}:{password}#{host}:{port}/{database}'
engine = create_engine(connection_string)
conn = engine.connect()
table = "table1"
schema = 'Parameter String, Key UInt8'
engine.execute("CREATE TABLE IF NOT EXISTS {}({}) ENGINE = Log".format(table,schema))
If you created table using the mergeTree engine, it’s allowed to rename the column:
engine.execute("CREATE TABLE IF NOT EXISTS {}({}) ENGINE =MergeTree ORDER BY Key".format(table,schema))
I am trying to add a new column in a monetDB database and I want it positioned after a specific one. In mysql this is possible using the AFTER keyword.
ALTER TABLE myTable ADD myNewColumn VARCHAR(255) AFTER myOtherColumn
I am trying this in the mclient:
sql>ALTER TABLE dbname.table_name ADD COLUMN new_name AFTER existing_name SET DEFAULT NULL;
What I get is a syntax error:
syntax error, unexpected AFTER in: "ALTER TABLE dbname.table_name ADD COLUMN new_name AFTER"
It is true that the ALTER documentation does not specify that AFTER exists, but I am hoping that anybody knows an alternative.
The safe way is to create an new table with the columns properly ordered and move the data; you probably know this already.
However if you really cannot do that, create a view:
CREATE VIEW AS SELECT [order the columns however you want here] FROM your_table;
When I need to change a column into a partition (convert normal column as partition column in hive), I want to create a new table to copy all columns except one. I currently have >50 columns in the original table. Is there any clean way of doing that?
Something like:
CREATE student_copy LIKE student EXCEPT age and hair_color;
Thanks!
You can use a regex:
CTAS using REGEX column spec. :
set hive.support.quoted.identifiers=none;
CREATE TABLE student_copy AS SELECT `(age|hair_color)?+.+` FROM student;
set hive.support.quoted.identifiers=column;
BUT (as mentioned by Kishore Kumar Suthar :
this will not create a partitioned table, as that is not supported with CTAS (Create Table As Select).
Only way I see for you to get your partitioned table is by getting the complete create statement of the table (as mentioned by Abraham):
SHOW CREATE TABLE student;
Altering it to create a partition on the column you want. And after that you can use the select with regex when inserting into the new table.
If your partition column is already part of this select, then you need to make sure it is the last column you insert. If it is not you can exclude that column in the regex and including it as last. Also if you expect several partitions to be created based on your insert statement you need to enable 'dynamic partitioning':
set hive.support.quoted.identifiers=none;
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
INSERT INTO TABLE student_copy PARTITION(partcol1) SELECT `(age|hair_color|partcol1)?+.+`, partcol1 FROM student;
set hive.support.quoted.identifiers=column;
the 'hive.support.quoted.identifiers=none' is required to use the backticks '`' in the regex part of the query. I set this parameter to it's original value after my statement: 'hive.support.quoted.identifiers=column'
CREATE TABLE student_copy LIKE student;
It just copies the source table definition.
CREATE TABLE student_copy AS select name, age, class from student;
Target cannot be partitioned table.
Target cannot be external table.
It copies the structure as well as the data
I use below command to get the create statement of existing table.
SHOW CREATE TABLE student;
Copy the result and modify that based on your requirement for new table and run the modified command to get the new table.
I need to insert new rows to Cassandra, to a table that has only primary key columns, e.g.:
CREATE TABLE users (
user_id bigint,
website_id bigint,
PRIMARY KEY (user_id, website_id)
)
The obvious way to do it would be by INSERT:
INSERT INTO users(user_id, website_id) VALUES(1,2);
But I want to do it with use of Hadoop CqlOutputFormat and CqlRecordWriter only supports UPDATE statements. That's usually not a problem as UPDATE is in theory semantically the same as INSERT. (It will create rows if given primary key does not exist).
But here... I don't know how to construct UPDATE statement - it seems that CQL just does not
support my case, where there are non-primary key columns. See what I tried:
> update users set where user_id=3 and website_id=2 ;
Bad Request: line 1:18 no viable alternative at input 'where'
> update users set website_id=2 where user_id=3;
Bad Request: PRIMARY KEY part website_id found in SET part
> update users set website_id=2 where user_id=3 and website_id=2;
Bad Request: PRIMARY KEY part website_id found in SET part
> update users set website_id=2,user_id=1;
Bad Request: line 1:40 mismatched input ';' expecting K_WHERE
Some ideas on how to resolve it?
Many thanks.
Not sure if you can do this with update like that. But why not just create a new dummy column that you never use for anything else? Then you could do
update users set dummy=1 where user_id=3 and website_id=2;
You can't update primary key values in Cassandra as you have explained. As a solution you could also delete the row and insert a new one with the correct value in it. It's just a bit cleaner than creating two rows with one incorrect.