The documentation attests that it is possible to insert data older than 7 days (and not older than 1 year) into partitioned tables.
But whenever I try streaming one month old data using Go BigQuery Client, it returns an error: "You can only stream to date range within 7 days in the past and 3 days in the future relative to the current date."
How can I stream data older than 7 days using the Go client?
Edit 1: Here is the table schema:
bigquery.TableMetadata{
Schema: bigquery.Schema{
{Name: "page_id", Required: true, Type: bigquery.IntegerFieldType},
{Name: "user_id", Required: false, Type: bigquery.IntegerFieldType},
{Name: "hit_time", Required: true, Type: bigquery.TimestampFieldType},
},
TimePartitioning: &bigquery.TimePartitioning{Field: "hit_time", RequirePartitionFilter: true},
}
According to this issue, this is a new feature that "should be fully rolled out very soon". I got my project whitelisted to try it and I was able to insert older data into the tables.
Related
In the data I have there are multiple Boolean columns that categorise each row, for example:
Object
Fruit
Yellow
Round
Chair
FALSE
FALSE
FALSE
Banana
TRUE
TRUE
FALSE
Apple
TRUE
FALSE
TRUE
Ball
FALSE
FALSE
TRUE
I am trying to make a KPI chart that would go alongside a table of this data in which it would have 'Fruit', 'Yellow', & 'Round' as categories and a count of how many are true, so that when you click on them it will mark and limit the data in the table. In this example 'Fruit' would have a count value of 2, 'Yellow' 1, and 'Round' 2.
How can I make a single KPI chart look at multiple columns and count their True values? I tried making a second data table to unpivot the columns into rows and have their count, which works, but then I am unsure as to how to make a relation between the two table such that I can mark and limit the original data table?
Please let me know if you require any information. I am using Spotfire 10.10, and this is how the data is presented to me, I cannot change it.
I have a bunch of indices A,B,C,D containing large amounts of data per each let's say 50 million records per index and the record contains data as follows:
user: amine, age: 22, state: Michigan
and I want to create a new index E and bulk new data into it from a csv file. What I hope for is that if a user from E already exists in one of the other indices but with different information for example:
user: amine, age: 25, state: California
I want the user amine to be updated in all other indices with the new data.
I tried pulling all the data processing it and re-ingesting it but the process took so long.
I have two data tables in my project. First one is taking data from an array in my hand,
data table #1:
$('#table1').dataTable({
"sPaginationType": "full_numbers",
"iDisplayLength": 25,
"aaSorting": [[7,'desc'],[2,'desc']],
"bStateSave": true
});
In above table I got the result sorted as expected. But for my second table, which uses an ajax request for data, is not sorted as expected,
data table #2
$('#table2').dataTable({
processing: true,
serverSide: true,
ajax: 'a valid url',
"aaSorting": [[3,'asc'],[0,'desc']],
"bStateSave": false,
"iDisplayLength": 25
});
I got the column with index 3 sorted but column with index 0 is not sorted for the same values of column 3. Do anyone is facing this kind of issue? Any help or suggestions invited.
If you use aDataSort instead of aaSorting , you can tell a column that it should do a multi-column sort rather than just by itself. - as said over here by datatable site admin allan.
For more detail about aDataSort, go at datatable
I have the following data structure stored in RethinkDB table:
{
id: string,
parentId: string,
timestamp: number,
data: Object
}
This data structure forms a tree, it can be depicted using the following diagram (white records represent ordinary data carrying records, the red ones have their data property equal to null which represents delete operation):
Now for every record in the table I would like to be able to compute the nextRecord. Which is the closest record in time to the current one. The task seems simple when there is only one record pointing back to a parent:
1 => 2
4 => 9
5 => 6
6 => 8
...
But it becomes more difficult to compute such value when parent record is being referenced by several child records:
2 => 3
3 => 5
7 => 11
The is also case when there is no child reference in which case the result should be null (for example record #8 has no child records, and so null should be returned).
So I'm not asking to write the query itself (which on the other hand would be really great to me) but at least point out the direction in which I can find solution to this problem.
Thank you in advance!
You can do this efficiently with a compound index on parentId and timestamp. You can create the index like this:
r.table('data').indexCreate('parent_timestamp', function(row) {
return [row('parentId'), row('timestamp')];
})
After you've done that, you can find the earliest item with parent PARENT like so:
r.table('data')
.between([PARENT, r.minval], [PARENT, r.maxval], {index: 'parent_timestamp'})
.orderBy({index: 'parent_timestamp'})
.nth(0).default(null)
I have a profile table that saves all profiles for all user.
I have different types of users and want each type of user to have different select options for choosing a certain field.
So both user types can choose how long they want to register for, but the have different options - one can choose 2 years and the other cant.
The schema.yml looks something like this:
UserProfile:
columns:
username:
type: string(255)
notnull: true
unique: false
WriterUserProfile:
inheritance:
type: column_aggregation
extends: UserProfile
columns:
register_time:
type: enum
values:
- 6 months
- 1 year
- 2 years
- Other
default: other
ReaderUserProfile:
inheritance:
type: column_aggregation
extends: UserProfile
columns:
register_time:
type: enum
values:
- 6 months
- 1 year
- Other
default: other
For some reason I am unable to select the '2 year' option - the form gives an 'invalid' error.
Does the '2 years' and 'Other' coincide with eachother because they are both the 3rd option?
Are there other fields which are not common? This one field only not enough cause to use column aggregation. Anyway, if the same field appears in multiple sub-classes than the field should be moved up and field names should be unique among all related classes (UserProfile, WriterUserProfile, ReaderUserProfile in your case).
You can change the options of the choice field in a form:
$choices = array('0' => 'a', '1' => 'b');
$this->getWidget('register_time')->setOption('choices', $choices);
$this->getValidator('register_time')->setOption('choices', array_keys($choices));