I can't find a way to extract table expiration date that is not via the console (https://console.cloud.google.com/).
We maintain thousands of tables in BQ and we want to enforce the use of table expiration date - so the only way is to collect the data automatically.
is this possible via query/cli/go/python/perl/whatever?
this can be done via querying the INFORMATION_SCHEMA.TABLE_OPTIONS:
SELECT * FROM `my_project.my_dataset.INFORMATION_SCHEMA.TABLE_OPTIONS`
where option_name='expiration_timestamp'
the value will be in the option_name column.
If you want to extract the expiration time from a number of tables in a dataset, you can refer to this documentation [1] (the same query provided by #EldadT), which returns the catalog of tables in a dataset with the expiration time option.
Therefore, if you want to create a script in Python to get the result of this query, you can also check a Client Bigquery Library [2] to run that query and get the expiration time for each table in a dataset.
[1] https://cloud.google.com/bigquery/docs/tables#example_1_2
[2] https://cloud.google.com/bigquery/docs/reference/libraries#client-libraries-usage-python
You can use BigQuery CLI:
bq show mydataset.mytable
Output:
Last modified Schema Total Rows Total Bytes Expiration Time Partitioning Clustered Fields Labels
----------------- ------------------- ------------ ------------- ----------------- ------------------- ------------------ --------
16 Aug 10:42:13 |- col_1: integer 7 106 21 Aug 10:42:13
|- col_2: string
Related
I have a DB and an application server located in NYC. I want to write a mobile application, which will be used by users from different cities (Los Angeles, New York, Miami). I use Postgresql as database, Spring MVC for backend and Ionic for mobile application. My question is - when user A from Los Angeles inserts a data into database for time 10:00 AM, how should I show this data to the user from New York, because there is a timezone difference? How should I store this kind of data in Postgres, and how should I process it in Spring MVC? Should my rest return time in milliseconds, or should I use timestamp or timestampz?
Thanks in advance!
You should store the data as TIMESTAMPTZ. This is what that data type is for. This blog post explains it a bit.
TIMESTAMPTZ stores all timestamps in absolute time, and displays them based on the user's timezone settings. For example:
postgres=# create table times ( some_time timestamptz );
CREATE TABLE
postgres=# set timezone = 'US/Eastern';
SET
postgres=# insert into times values ( '2017-09-15 10:00:00' );
INSERT 0 1
postgres=# set timezone = 'US/Pacific';
SET
postgres=# select * from times;
some_time
------------------------
2017-09-15 07:00:00-07
(1 row)
postgres=# set timezone = 'US/Eastern';
SET
postgres=# select * from times;
some_time
------------------------
2017-09-15 10:00:00-04
(1 row)
This allows you to display the correct time, from the user's perspective, even for a user who themselves moves around and changes time zones. You just need to remember to set the time zone in JDBC when you connect.
I must generate a table of calendar dates from dateIni to dateEnd in Powercenter Designer.
dateIni is is fixed, for example '2013-01-01'
dateEnd is sysdate + 'n' months
I'm trying to generate from a java tranformation, that can generate several dynamic rows but needs an input row and I do not have any input... it there any other better approach using seq generator???
As an example table content result must be
date
=======
'2013-01-01'
'2013-01-02'
'2013-01-03'
...
...
'2016-03-10'
You can pass a single input row from any source into the Java transformation and then generate rows with consecutive dates in a loop.
You can create a simple table with two columns - dateIni and dateEnd. It will contain a single row that will both kickstart the Java code and provide configuration for the mapping.
When working with an Oracle database you can also use the following query in your source qualifier:
SELECT level
FROM dual
CONNECT BY
level <= 1000 --(or any other number)
This will generate 1000 rows.
With an Expression-transformation you can change this into dates:
ADD_TO_DATE(to_date('20190101','yyyymmdd'), 'DAY',Level)
how to get data from ms access database when u only know the column name and row number ?
example
select empID
from table
where row no is x
One way to get the 15th records is to use the TOP command twice. First, get the 15 records order by id asc. Then take the top 1 record order by id. That assumes you know a field in the record (table) that you can order by.
SELECT TOP 1 * FROM
(
SELECT top 15 *
FROM [Order Details] d
ORDER BY d.[Order Id] asc
) q
ORDER BY d.[Order Id] desc
The above query works fine in MS Access 2007.
I will see if there is some indicator in the MS Access system tables that are not well documented.
While there is a MSysObjects hidden table and LvProp (L-Value Property), it is a long binary data type.
It looks to me that MS Access is storing the data in a binary format. However, there is no DBCC PAGE to view the internal record structure.
In short, I think the solution above using TOP and/or COUNT is the only way to go.
I'm creating a fairly in-depth temporal database prototype in which we are using Oracle's Total Recall to manage transaction times.
My test dataset has about 150k current rows along with 170k retired rows loaded into a FLASHBACK ARCHIVE enabled table. The augmented SQL queries (like the first one below) are executing correctly providing the appropriate data results.
select *
from CUT_BLOCK_COMBO AS OF TIMESTAMP FROM_TZ(TO_TIMESTAMP('2007-08-28 00:00:00','YYYY-MM-DD HH24:MI:SS'), 'UTC')
WHERE CB_SKEY = 4141;
This select statement returns the following data:
CB_SKEY HVA_SKEY FOREST_FILE_ID CUTTING_PERMIT_ID TIMBER_MARK CUT_BLOCK_ID
----------- ----------- -------------- ----------------- ----------- ------------
4141 53094 A80053 80053 29025
However once I wrap the table up into a view, I can no longer query the data with the 'AS OF TIMESTAMP' clause.
create or replace view CUT_BLOCK_COMBO_VW as select * from CUT_BLOCK_COMBO;
select *
from CUT_BLOCK_COMBO_VW AS OF TIMESTAMP FROM_TZ(TO_TIMESTAMP('2007-08-28 00:00:00','YYYY-MM-DD HH24:MI:SS'), 'UTC')
WHERE CB_SKEY = 4141;
The select statement from the view returns the following error: ORA-01466: unable to read data - table definition has changed
Any ideas what I missed when creating the view definition? I couldn't find anything in the docs (Oracle Total Recall 11G R2)
Does it work as an in-line view?
I would suspect that the view would have to have existed as-of the timestamp. Possibly the code is looking for the create date or last DDL time on the object against which you are using AS OF.
Just a theory, but you could test it pretty easily I think, and possibly prove the theory by tracing the execution.
If that's the case then I can't think of a safe workaround I'm afraid.
I'm trying to take a sample from a insurance claims database.
For example 20% random, sample from 1 million claims data where provider type is '25' and year is '2012'. Data is in sqldeveloper. I am a statistician with basic SQL knowledge.
You can use SAMPLE to get a random set of rows from a table.
SELECT *
FROM claim SAMPLE(20)
WHERE type ='25'
AND year = 2012;
SQL has a SAMPLE command built in. Example:
SELECT * FROM emp SAMPLE(25)
means each row in emp has a 25% chance of being included in the resulting set. NOTE: this does not mean that exactly 25% of the rows are necessarily selected
this blog was a quick read on more details on sampling
With this you get a single line of a sample that is shown random.
SELECT * FROM TABLE# SAMPLE(10)
FETCH NEXT 1 ROWS ONLY