Per the documentation on Analytical store.
What is Azure Cosmos DB Analytical Store?
“The first non-null occurrence defines the column data type. Any document not following the first non-null datatype won't be represented in analytical store.”
I want to confirm my DateTime columns are interpreted as DateTime and not stored as string.
How can I view the Well-defined schema representation?
There are two solutions:
SQL Serverless:
a) Create a view say MyView that gets the data.
b) Run the following against that view.
sp_describe_first_result_set #tsql = N'select * from MyView where 1=0'
Spark: df.printSchema
Related
Can we insert into a view in Hive?
I have done this in the past with Oracle and Teradata.
But, doesn't seem to work in Hive.
create table t2 (id int, key string, value string, ds string, hr string);
create view v2 as select id, key, value, ds, hr from t2;
insert into v2 values (1,'key1','value1','ds1','hr1')
***Error while compiling statement: FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: Unable to determine if null is encrypted: java.lang.NullPointerException***
These seems to be some sort of update support in view. But, I can't see anything on insert into a view.
https://cwiki.apache.org/confluence/display/Hive/UpdatableViews
Thanks for the feedback. Makes sense. The reason behind needing this functionality is, we use an ETL tool that has problems with handling high precision decimals (>15 digits). If the object(table->column in this case) is represented as string within the tool, we don't have a problem. So, i thought i'll define a bunch of views with string datatypes and use that in the tool instead. But, can't do inserts in hive to view. So, may be i need to think of something else. Have done this way before with oracle and teradata.
Can we have two tables with different structures point to the same underlying hdfs content? Probably wouldn't work because fo the parquet storage which stores schema. Sorry, not a hadoop expert.
Thanks a lot for your time.
It is not possible to insert data in a Hive view, Hive view is just a projection of a Hive table (you can see it as presaved query). From Hive documentation
Note that a view is a purely logical object with no associated
storage. (No support for materialized views is currently available in
Hive.) When a query references a view, the view's definition is
evaluated in order to produce a set of rows for further processing by
the query. (This is a conceptual description; in fact, as part of
query optimization, Hive may combine the view's definition with the
query's, e.g. pushing filters from the query down into the view.)
The link (https://cwiki.apache.org/confluence/display/Hive/UpdatableViews) seems to be for a proposed feature.
Per the official documentation:
Views are read-only and may not be used as the target of LOAD/INSERT/ALTER.
Is it possible to do updates via Resultset to an Oracle view? Asking as my code is giving me insufficient priviledge error when it does the rs.updateRow() call. I have checked and I definitely have access to the table/view.
The code looks like:
white (rs.next()) {
int updateStatus = getPSCforAction(status);
rs.updateInt("SPSC", updateStatus);
rs.updateRow;
}
The SELECT statement changes depends on operation but it will always be querying an Oracle view (and in some cases multiple views). My main question is whether updating via resultSet can be done to an Oracle view (or views)?
To answer your question one must see a definition of you view and a SELECT statament that is used to produce the resulset in your Java code. Without looking at this it is hard to give the answer
Anyway, generar rules and limitations are described in the Oracle Database JDBC Developer's guide:
Result Set Limitations
The following limitations are placed on queries for enhanced result
sets. Failure to follow these guidelines results in the JDBC driver
choosing an alternative result set type or concurrency type.
To produce an updatable result set:
A query can select from only a single table and cannot contain any
join operations.
In addition, for inserts to be feasible, the query must select all
non-nullable columns and all columns that do not have a default value.
A query cannot use SELECT * . However, there is a workaround for this.
A query must select table columns only.
It cannot select derived columns or aggregates, such as the SUM or MAX
of a set of columns.
To produce a scroll-sensitive result set:
A query cannot use SELECT *. However, there is a workaround for this.
A query can select from only a single table.
Scrollable and updatable result sets cannot have any column as Stream.
When the server has to fetch a Stream column, it reduces the fetch
size to one and blocks all columns following the Stream column until
the Stream column is read. As a result, columns cannot be fetched in
bulk and scrolled through.
They vaguely write that:
A query can select from only a single table and cannot contain any
join operations.
Could be that they mean "exlusively from tables, but not views", but also they could mean: "from tables and views", nobody knows, one must test this.
Another possible problem - your view may not be updatable, that is it doesn't conform to the following rules:
Notes on Updatable Views The following notes apply to updatable views:
An updatable view is one you can use to insert, update, or delete base
table rows. You can create a view to be inherently updatable, or you
can create an INSTEAD OF trigger on any view to make it updatable.
To learn whether and in what ways the columns of an inherently
updatable view can be modified, query the USER_UPDATABLE_COLUMNS data
dictionary view. The information displayed by this view is meaningful
only for inherently updatable views. For a view to be inherently
updatable, the following conditions must be met:
Each column in the view must map to a column of a single table. For
example, if a view column maps to the output of a TABLE clause (an
unnested collection), then the view is not inherently updatable.
The view must not contain any of the following constructs:
A set operator
A DISTINCT operator
An aggregate or analytic function
A GROUP BY, ORDER BY, MODEL, CONNECT BY, or START WITH clause
A collection expression in a SELECT list
A subquery in a SELECT list
A subquery designated WITH READ ONLY Joins, with some exceptions, as
documented in Oracle Database Administrator's Guide
In addition, if an
inherently updatable view contains pseudocolumns or expressions, then
you cannot update base table rows with an UPDATE statement that refers
to any of these pseudocolumns or expressions.
If you want a join view to be updatable, then all of the following
conditions must be true:
The DML statement must affect only one table underlying the join.
For an INSERT statement, the view must not be created WITH CHECK
OPTION, and all columns into which values are inserted must come from
a key-preserved table. A key-preserved table is one for which every
primary key or unique key value in the base table is also unique in
the join view.
For an UPDATE statement, the view must not be created WITH CHECK
OPTION, and all columns updated must be extracted from a key-preserved
table.
For a DELETE statement, if the join results in more than one
key-preserved table, then Oracle Database deletes from the first table
named in the FROM clause, whether or not the view was created WITH
CHECK OPTION.
I need to create a sequence whose value is going to be read by call to .NEXTVAL in PL/SQL code and saved in more then one record of a specific table column, so my design doesn't require to define a PK on the aforementioned column.
I cannot find out how to edit the sequence tab in Oracle Data Modeler (I'm on version 4.1.1) when the PK checkbox is not selected (all the sequence related fields are disabled).
Any idea?
In the relational model, choose your DB and within that you will find sequence as an item to create. You can also create other types of object here.
i have a table t1 in database A and another table t2 in another database B.
how can we select data from them without using qualifiers directly..
just as database1.table1.something.
You should be able to query it with fully qualified names such as SCHEMA.TABLE#DBLINK.
If you don't want to use the #DBLINK notation while querying from database A you can mask the #DBLINK in a view (In database A) and query that view instead.
CREATE OR REPLACE VIEW remote_table [(column_list)]
AS select * FROM SCHEMA.TABLE#DBLINK;
I have a Data Flow from an Oracle table to an MSSQL table with one field of data type BIT. The Oracle table is using the characters Y and N at the moment (I'm unsure of the data type and have no way of checking), but the MSSQL table needs to be data type BIT. What type of cast can I use on the Oracle query so that the data is pulled smoothly over?
Use char(1) and then use a derived column transformation like this:
(DT_BOOL)(OracleField == "Y"?1:0)
Give this column a name like OracleFieldAsBool
and then use it instead of the original column in the rest of your data flow.