I have a Hive table whose schema is as below, the col is of map type:
select
col
from table
col
{"name":"abc", "value":"val_1"}
What I need to do is change the val_1 to val_2 and create another table from it.
create table table_2 as
select
col -- TODO: need to do something here
from table
Any suggestions? Thanks!
with t as (select map("name","abc","value","val_1") as col)
select map("name",col["name"],"value","val_2") as col
from t
+--------------------------------+
| col |
+--------------------------------+
| {"name":"abc","value":"val_2"} |
+--------------------------------+
Related
I have below existing table emp with partition column as as_of_date(current_date -1).
CREATE EXTERNAL TABLE IF NOT EXISTS emp(
student_ID INT,
name STRING)
partitioned by (as_of_date date)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION '/user/emp';
Below are the existing partition path
user/emp/as_of_date=2021-09-02
user/emp/as_of_date=2021-09-03
user/emp/as_of_date=2021-09-04
In emp table, I have to add new partition column as businessdate(current_date) and change partition column (as_of_date) to non-partition column.
Expected output should be as below.
describe table emp;
CREATE EXTERNAL TABLE IF NOT EXISTS emp(
student_ID INT,
Name STRING,
as_of_date date)
partitioned by (businessdate date)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION '/user/emp';
After update, below will be hdfs path
user/emp/buinessdate=2021-09-03
user/emp/buinessdate=2021-09-04
user/emp/businessdate=2021-09-05
Expected output table:
|student_ID |name |as_of_date | business_date |
|--| --- | --- |----|
|1 | Sta |2021-09-02| 2021-09-03 |
|2 |Danny|2021-09-03| 2021-09-04 |
|3 |Elle |2021-09-04| 2021-09-05 |
Create new table, load data from old table, remove old table, rename new table.
--1 Create new table emp1
CREATE EXTERNAL TABLE emp1(
student_ID INT,
Name STRING,
as_of_date date)
partitioned by (businessdate date)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION '/user/emp1';
--2 Load data into emp1 from the emp with new partition column calculated
--dynamic partition mode
SET hive.exec.dynamic.partition=true;
SET hive.exec.dynamic.partition.mode=nonstrict;
insert overwrite table emp1 partition (business_date)
select
student_ID,
Name,
as_of_date,
date_add(as_of_date,1) as business_date
from emp;
Now you can drop old (make it managed first to drop location as well) and rename new table if necessary.
I have a column which i set enum earlier,
$table->enum('column_name', ['value1', 'value2']);
Now i want to change that into string without losing data.
I am using postgres database.
please help me,
thanks
You Actually cant change type of enum column with simple migration
How i achieved it was using the DB::statement to alter the column type
DB::statement('ALTER TABLE <table_name> MODIFY <column_name> VARCHAR(200)');
I'm not sure about the postgres You can modify query according to your need. Do make backup 1st we aren't sure if it will make you lose your data.
In plain SQL the solution is more something like "ALTER TABLE table_name ALTER COLUMN column_name SET DATA TYPE VARCHAR(20)".
For example:
postgres=# create table tenum(x serial, y enumval);
CREATE TABLE
postgres=# select relfilenode from pg_class where relname='tenum';
relfilenode
-------------
61038
(1 row)
postgres=# insert into tenum(y) values ('val1');
INSERT 0 1
postgres=# insert into tenum(y) values ('val2');
INSERT 0 1
postgres=# select * from tenum;
x | y
---+------
1 | val1
2 | val2
(2 rows)
postgres=# alter table tenum alter column y set data type varchar(20);
ALTER TABLE
postgres=# select relfilenode from pg_class where relname='tenum';
relfilenode
-------------
61042
(1 row)
postgres=# select * from tenum;
x | y
---+------
1 | val1
2 | val2
(2 rows)
postgres=#
Note that PostgreSQL will rewrite the table due to data type change.
My database performance skills are not really good - I could not find any 'good' Google result, so I need your help.
I am trying to convert all columns of a table. All data in this table are datatype varchar.
I do have a reference table which has wrong data but correct meta data like Column_Name, Data_Type etc. ==> So I try to use the table with the correct metadata to convert the table with the correct data. As in the following example, the dynamic script wants to convert a column that should actually be datetime:
IF #Datatype IN ('datetime')
Begin
set #sqlDate = ('
Update dbo.'+#Table+'
SET '+#Column+' = TRY_CONVERT( datetime, '+#Column+', 105)
Alter Table dbo.'+#Table+'
Alter Column '+#Column+' datetime;
')
exec (#sqlDate);
End
So my goal is to convert a Table like this :
+----------------+----------------+
| Col1 (varchar) | Col2 (varchar) |
+----------------+----------------+
| '01.01.2000' | '124.5' |
+----------------+----------------+
To this:
+-------------------------+--------------+
| Col1(datetime) | Col2 (float) |
+-------------------------+--------------+
| jjjj-mm-tt hh:mi:ss.mmm | 124.5 |
+-------------------------+--------------+
(based on the correct metadata table)
Do you think its better to first convert data into #TempTable and Update the original Column via the pre-converted #TempTable? Any better solution?
Thanks a lot!
Here's how I would do it.
First, create and populate sample table (Please save is this step in your future questions):
CREATE TABLE Sample
(
DateTimeColumn varchar(50),
FloatColumn varchar(50)
);
INSERT INTO Sample(DateTimeColumn, FloatColumn) VALUES ('01.01.2000', '124.5');
Then - Alter the table to add the columns with the correct data type.
ALTER TABLE Sample
ADD AsDateTime DateTime,
AsFloat float;
Populate the new columns:
UPDATE Sample
SET AsDateTime = TRY_CONVERT(datetime, DateTimeColumn, 105),
AsFloat = TRY_CAST(FloatColumn AS float);
At this point, you should pause and check if you really did get correct values.
Once the new columns data is verified, you can delete the old columns
ALTER TABLE Sample
DROP COLUMN DateTimeColumn;
ALTER TABLE Sample
DROP COLUMN FloatColumn;
and rename the new columns:
EXEC sp_rename 'dbo.Sample.AsDateTime', 'DateTimeColumn', 'COLUMN';
EXEC sp_rename 'dbo.Sample.AsFloat', 'FloatColumn', 'COLUMN';
A quick select to verify the change:
SELECT DateTimeColumn, FloatColumn
FROM Sample;
Results:
DateTimeColumn FloatColumn
2000-01-01 00:00:00 124.5
I have a query sequence in Vertica below:
drop table [table_name];
create [table_name] as select ...;
drop table [table_name];
create [table_name] as select ...;
select ...;
My resulting file looks like
DROP TABLE
CREATE TABLE
DROP TABLE
CREATE TABLE
column1 | column2 | column3 |
Does Vertica have a configuration parameter that can eliminate the 'DROP' and 'CREATE' table and retains only the SELECT output?
I have created a cluster in Oracle
CREATE CLUSTER myLovelyCluster (clust_id NUMBER(38,0))
SIZE 1024 SINGLE TABLE HASHKEYS 11;
Than a table for the cluster
CREATE TABLE Table_cluster
CLUSTER myLovelyCluster (columnRandom)
AS SELECT * FROM myTable ;
the columnRandom is well defined as NUMBER(38,0) but why I am getting an error assuming incompatible column definition?
Thanks
Are you sure that columnRandom is number(38,0)? In oracle NUMBER != NUMBER(38,0)
Let's create two table.
create table src_table ( a number);
create table src_table2( a number(38,0));
select column_name,data_precision,Data_scale from user_tab_cols where table_name like 'SRC_TABLE%';
Result of query is. Definitions of column are different.
+-------------+----------------+------------+
| Column_name | Data_Precision | Data_scale |
+-------------+----------------+------------+
| A | | |
| A | 38 | 0 |
+-------------+----------------+------------+
And if i try creat cluster for first table.
CREATE TABLE Table_cluster
CLUSTER myLovelyCluster (a)
AS SELECT * FROM src_table ;
ORA-01753: column definition incompatible with clustered column definition
For 2-nd every thing is ok.
CREATE TABLE Table_cluster
CLUSTER myLovelyCluster (a)
AS SELECT * FROM src_table2 ;
If you add cast into select. Execution also is correct.
CREATE TABLE Table_cluster CLUSTER myLovelyCluster (a)
AS SELECT cast(a as number(38,0)) FROM src_table;