Is there any way to ALTER a table and change the column name in clickhouse?
I only found to change tha table name but not for an individual column in a straight forward way.
Thanks.
The feature has been introduced here into v20.4.
ALTER TABLE table1 RENAME COLUMN old_name TO new_name
You can also rename multiple columns at on:
ALTER TABLE table1
RENAME COLUMN old_name1 TO new_name1,
RENAME COLUMN old_name2 TO new_name2
Old answer:
ClickHouse doesn't have that feature yet.
Implementation is not trivial, because ALTERs that changing columns
are processed outside of usual replication queue, and adding rename
without reworking of ALTERs will introduce race conditions in
replicated tables.
https://github.com/yandex/ClickHouse/issues/146#issuecomment-255631384
As #Slash said, the solution for now is to create new table and
INSERT INTO `new_table` SELECT * FROM `old_table`
Do not forget that column aliasing won't work there (AS).
INSERT INTO `new_table` SELECT a, b AS c, c AS b FROM `old_table`
That will still insert a into first column, b into second column and c into third column. AS has no effect there.
You can try use CREATE TABLE new_table with another field name
and run INSERT INTO new_table SELECT old_field AS new_field FROM old_table
If you created the table using Engine=log, it won't allow you to alter or rename the column.
connection_string = f'clickhouse://{username}:{password}#{host}:{port}/{database}'
engine = create_engine(connection_string)
conn = engine.connect()
table = "table1"
schema = 'Parameter String, Key UInt8'
engine.execute("CREATE TABLE IF NOT EXISTS {}({}) ENGINE = Log".format(table,schema))
If you created table using the mergeTree engine, it’s allowed to rename the column:
engine.execute("CREATE TABLE IF NOT EXISTS {}({}) ENGINE =MergeTree ORDER BY Key".format(table,schema))
Related
I have a specific scenario where i have to insert two new columns in an existing table in Oracle. I can not do the dropping and recreating the table. So can it be achieved by any means??
Amit-
I don't believe you can add a column anywhere but at the end of the table once the table is created. One solution might be to try this:
CREATE TABLE MY_TEMP_TABLE AS
SELECT *
FROM TABLE_TO_CHANGE;
Drop the table you want to add columns to:
DROP TABLE TABLE_TO_CHANGE;
It's at the point you could rebuild the existing table from scratch adding in the columns where you wish.
Let's assume for this exercise you want to add the columns named "COL2 and COL3".
Now insert the data back into the new table:
INSERT INTO TABLE_TO_CHANGE (COL1, COL2, COL3, COL4)
SELECT COL1, 'Foo', 'Bar', COL4
FROM MY_TEMP_TABLE;
When the data is inserted into your "new-old" table, you can drop the temp table.
DROP TABLE MY_TEMP_TABLE;
This is often what I do when I want to add columns in a specific location. Obviously if this is a production on-line system, then it's probably not practical, but just one potential idea.
-CJ
In 12c you can make use of the fact that columns which are set from invisible to visible are displayed as the last column of the table:
Tips and Tricks: Invisible Columns in Oracle Database 12c
Maybe that is the 'trick' #jeffrey-kemp was talking about in his comment, but the link there does not work anymore.
Example:
ALTER TABLE my_tab ADD (col_3 NUMBER(10));
ALTER TABLE my_tab MODIFY (
col_1 invisible,
col_2 invisible
);
ALTER TABLE my_tab MODIFY (
col_1 visible,
col_2 visible
);
Now col_3 would be displayed first in a SELECT * FROM my_tab statement.
Note: This does not change the physical order of the columns on disk, but in most cases that is not what you want to do anyway. If you really want to change the physical order, you can use the DBMS_REDEFINITION package.
Although this is somewhat old I would like to add a slightly improved version that really changes column order. Here are the steps (assuming we have a table TAB1 with columns COL1, COL2, COL3):
Add new column to table TAB1:
alter table TAB1 add (NEW_COL number);
"Copy" table to temp name while changing the column order AND rename the new column:
create table tempTAB1 as select NEW_COL as COL0, COL1, COL2, COL3 from TAB1;
drop existing table:
drop table TAB1;
rename temp tablename to just dropped tablename:
rename tempTAB1 to TAB1;
You (still) can not choose the position of the column using ALTER TABLE: it can only be added to the end of the table. You can obviously select the columns in any order you want, so unless you are using SELECT * FROM column order shouldn't be a big deal.
If you really must have them in a particular order and you can't drop and recreate the table, then you might be able to drop and recreate columns instead:-
First copy the table
CREATE TABLE my_tab_temp AS SELECT * FROM my_tab;
Then drop columns that you want to be after the column you will insert
ALTER TABLE my_tab DROP COLUMN three;
Now add the new column (two in this example) and the ones you removed.
ALTER TABLE my_tab ADD (two NUMBER(2), three NUMBER(10));
Lastly add back the data for the re-created columns
UPDATE my_tab SET my_tab.three = (SELECT my_tab_temp.three FROM my_tab_temp WHERE my_tab.one = my_tab_temp.one);
Obviously your update will most likely be more complex and you'll have to handle indexes and constraints and won't be able to use this in some cases (LOB columns etc). Plus this is a pretty hideous way to do this - but the table will always exist and you'll end up with the columns in a order you want. But does column order really matter that much?
I want to add a new column to a specific location in hive table. when i add new column it goes to the last position.
You need to recreate table. If the table is external and data already contains new column, then issue drop and create table statements. General solution is to:
1. create new_table...;
2. insert overwrite new_table select from old_table;
3. drop old_table;
4. alter new_table rename to old_table;
Also if datafiles already contain new column in some position you can
1. Alter table add column
Change column position using this example:
2. ALTER TABLE test_change CHANGE old_name new_name STRING AFTER other_col CASCADE;
See docs here: Change Column Name/Type/Position/Comment
How frequently are people running SELECT *?? Typically, people list out each column in the select statement. Just add the column to the end, and adjust like SELECT last_col, first_col, second_col ...
Alternatively, create a VIEW that runs a select statement with the column ordering you want.
Rename the table to something else, and name the view to the table, and no one would know any different
I have two columns Id and Name in Hive table, and I want to delete the Name column. I have used following command:
ALTER TABLE TableName REPLACE COLUMNS(id string);
The result was that the Name column values were assigned to the Id column.
How can I drop a specific column of the table and is there any other command in Hive to achieve my goal?
In addition to the existing answers to the question : Alter hive table add or drop column
As per Hive documentation,
REPLACE COLUMNS removes all existing columns and adds the new set of columns.
REPLACE COLUMNS can also be used to drop columns. For example, ALTER TABLE test_change REPLACE COLUMNS (a int, b int); will remove column c from test_change's schema.
The query you are using is right. But this will modify only schema i.e, the metastore. This will not modify anything on data side.
So, before you are dropping the column you should make sure that you hav correct data file.
In your case the data file should not contain name values.
If you don't want to modify the file then create another table with only specific column that you need.
Create table tablename as select id from already_existing_table
let me know if this helps.
When I need to change a column into a partition (convert normal column as partition column in hive), I want to create a new table to copy all columns except one. I currently have >50 columns in the original table. Is there any clean way of doing that?
Something like:
CREATE student_copy LIKE student EXCEPT age and hair_color;
Thanks!
You can use a regex:
CTAS using REGEX column spec. :
set hive.support.quoted.identifiers=none;
CREATE TABLE student_copy AS SELECT `(age|hair_color)?+.+` FROM student;
set hive.support.quoted.identifiers=column;
BUT (as mentioned by Kishore Kumar Suthar :
this will not create a partitioned table, as that is not supported with CTAS (Create Table As Select).
Only way I see for you to get your partitioned table is by getting the complete create statement of the table (as mentioned by Abraham):
SHOW CREATE TABLE student;
Altering it to create a partition on the column you want. And after that you can use the select with regex when inserting into the new table.
If your partition column is already part of this select, then you need to make sure it is the last column you insert. If it is not you can exclude that column in the regex and including it as last. Also if you expect several partitions to be created based on your insert statement you need to enable 'dynamic partitioning':
set hive.support.quoted.identifiers=none;
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
INSERT INTO TABLE student_copy PARTITION(partcol1) SELECT `(age|hair_color|partcol1)?+.+`, partcol1 FROM student;
set hive.support.quoted.identifiers=column;
the 'hive.support.quoted.identifiers=none' is required to use the backticks '`' in the regex part of the query. I set this parameter to it's original value after my statement: 'hive.support.quoted.identifiers=column'
CREATE TABLE student_copy LIKE student;
It just copies the source table definition.
CREATE TABLE student_copy AS select name, age, class from student;
Target cannot be partitioned table.
Target cannot be external table.
It copies the structure as well as the data
I use below command to get the create statement of existing table.
SHOW CREATE TABLE student;
Copy the result and modify that based on your requirement for new table and run the modified command to get the new table.
How can I change DATA TYPE of a column from number to varchar2 without deleting the table data?
You can't.
You can, however, create a new column with the new data type, migrate the data, drop the old column, and rename the new column. Something like
ALTER TABLE table_name
ADD( new_column_name varchar2(10) );
UPDATE table_name
SET new_column_name = to_char(old_column_name, <<some format>>);
ALTER TABLE table_name
DROP COLUMN old_column_name;
ALTER TABLE table_name
RENAME COLUMN new_column_name TO old_coulumn_name;
If you have code that depends on the position of the column in the table (which you really shouldn't have), you could rename the table and create a view on the table with the original name of the table that exposes the columns in the order your code expects until you can fix that buggy code.
You have to first deal with the existing rows before you modify the column DATA TYPE.
You could do the following steps:
Add the new column with a new name.
Update the new column from old column.
Drop the old column.
Rename the new column with the old column name.
For example,
alter table t add (col_new varchar2(50));
update t set col_new = to_char(col_old);
alter table t drop column col_old cascade constraints;
alter table t rename column col_new to col_old;
Make sure you re-create any required indexes which you had.
You could also try the CTAS approach, i.e. create table as select. But, the above is safe and preferrable.
The most efficient way is probably to do a CREATE TABLE ... AS SELECT
(CTAS)
alter table table_name modify (column_name VARCHAR2(255));
Since we can't change data type of a column with values, the approach that I was followed as below,
Say the column name you want to change type is 'A' and this can be achieved with SQL developer.
First sort table data by other column (ex: datetime).
Next copy the values of column 'A' and paste to excel file.
Delete values of the column 'A' an commit.
Change the data type and commit.
Again sort table data by previously used column (ex: datetime).
Then paste copied data from excel and commit.