I'm having a problem with what should be a very simple solution.
https://i.stack.imgur.com/QFTbm.jpg
How can i get the latest value xyz and abc in the field C .
Very new to quicksight.
There might be a simple solution that
I might not know about.
If you want to have a new calculated field which has the latest value of C for each column (A, B etc..), then you could add these as follows:
latest A
maxOver(
ifelse(
{Date} = maxOver({Date}, [A], PRE_FILTER), C,
null
),
[]
PRE_FILTER
)
latest B
maxOver(
ifelse(
{Date} = maxOver({Date}, [B], PRE_FILTER), C,
null
),
[]
PRE_FILTER
)
and so on
Related
I am trying to create the following format of relation using Pig Latin:
userid, day, {(pid,fulldate, x,y),(pid,fulldate, x,y), ...}
Relation description: Each user (userid) in each day (day) has purchased multiple products (pid)
I am Loading the data into:
A= LOAD '**from a HDFS URL**' AS (pid: chararray,userid:
chararray,day:int,fulldate: chararray,x: chararray,y:chararray);
B= GROUP A BY (userid, day);
Describe B;
B: {group: (userid: chararray,day: int),A: {(pid: chararray,day: int,fulldate: chararray,x: chararray,userid: chararray,y: chararray)}}
C= FOREACH B FLATTEN(B) AS (userid,day), $1.pid, $1.fulldate,$1.x,$1.y;
Describe C;
C: {userid: chararray,day: int,{(pid: chararray)}},{(fulldate: chararray)},{(x: chararray)},{(y: chararray)}}
The result of Describe C does not give the format I want ! What I am doing wrong?
You are correct till the GROUP BY part. After that however you are trying to do something messy. I'm actually not sure what is happening for your alias C. To arrive at the format you are looking for, you will need a nested foreach.
C = FOREACH B {
data = A.pid, A.fulldate, A.x, A.y;
GENERATE FLATTEN(group), data;
}
This allows C to have one record for each (userid, day) and all the corresponding (pid,fulldate, x, y) tuples in a bag.
You can read more about nested foreach here: https://www.safaribooksonline.com/library/view/programming-pig/9781449317881/ch06.html (Search for nested foreach in that link).
My understanding is that B is almost what you're looking for, except you would like the tuple containing userid and day to be flattened, and you would like only pid, fulldate, x, and y to appear in the bag.
First, you want to flatten the tuple group which has fields userid and day, not the bag A which contains multiple tuples. Flattening group unnests the tuple, which only has 1 set of unique values for each row, whereas flattening the bag A would effectively ungroup your previous GROUP BY statement since the values in the bag A are not unique. So the first part should read C = FOREACH B GENERATE FLATTEN(group) AS (userid, day);
Next, you want to keep pid, fulldate, x, and y in separate tuples for each record, but the way you've selected them essentially makes a bag of all the pid values, a bag of all the fulldate values, etc. Instead, try selecting these fields in a way that keeps the tuples nested in the bag:
C = FOREACH B GENERATE
FLATTEN(group) AS (userid, day),
A.(pid, fulldate, x, y) AS A;
I am creating a "temporary" (transition) table : "Abis" which must be the copy (identical structure) of a table "A" with the addition of data and the update of several fields via other tables more recent (B, C, D, E).
I have a primary key based on 2 fields in "A" (A.a and A.b) that is present in "Abis" (Abis.a and Abis.b) as well as in "B" (B.a and B.b).
I made a full join between A and B: A.a = B.a and A.b = B.b.
What mapping I have to put to feed my "Abis" table on Abis.a and Abis.b, recovering all key combinations of A (A.a + A.b) as well as all key combinations of B (B.a + B.b) that aren't present in A.
I tested with
"Case When A.a Not In B.a Than A.a Else B.a End"
But the query turns indefinitely.
To sum up:
Target Datastore: Abis
Diagram: A, B, C, D, E
Join: A.a = B.a and A.b = B.b (Full join)
Number of row: Table A ~ 6000, Table B ~ 40000
Software: ODI 10.1.3.5 (Oracle Data Integrator)
Thanks :)
Ok I almost solved my problem with the DECODE function
I tried the NVL function but it did not give exactly what I wanted.
The similar function to NVL is DECODE.
https://www.techonthenet.com/oracle/functions/decode.php
Mapping on Abis.a > Decode (A.a, 0, B.a, A.a)
Mapping on Abis.b > Decode (A.b, 0, B.b, A.b)
I've got a small problem.
I made calculations and all my ids are not following each other anymore (because of some delete during the calculus). Unfortunately, to make my result valid, I need this order... :/
So in order to simply the task, I made an external function, that will "rename" all the IDs but I do not know how to perform this.
Here is the function that I've got :
fun setId (W {id, ...}) =
let
in
print( "[" ^ Int.toString (id) ^ "]");
print( "[" ^ Int.toString (!nextId) ^ "]\n");
Ref.incr nextId
end
(for the one who are wandering appis just a homemade function to perform the same calcul on each element of a list, but it's not important.)
When I'm execute this code, I obtain in output :
[0][0]
[1][1]
[2][2]
[3][3]
[4][4]
[5][5]
[6][6]
[7][7]
[8][8]
[9][9]
[10][10]
[11][11]
[12][12]
[13][13]
[14][14]
[15][15]
[16][16]
[17][17]
[18][18]
[19][19]
[20][20]
[21][21]
[22][22]
[39][23]
[40][24]
[41][25]
[42][26]
[43][27]
[44][28]
[45][29]
[46][30]
[47][31]
[48][32]
[49][33]
[50][34]
[51][35]
[52][36]
[53][37]
as you can see there is a big problem [23] [39] the list does not have following numbers. :/
Basically, I would like the function setIdto be able to modify the ID of the Node. But I don't know how :/
Here is the datatype Node for understanding purposes :
datatype node =
W of {
id : int
, predId : int option
, creationDepcy : Dependency.depcy
, nominalDepcies : Dependency.depcy list ref
, pattern : Termstore.store
, propositions : Propstore.pstore
, nominals : Propstore.pstore
, diamonds : Termstore.store
, boxes : Termstore.store
, disjunctions : Termstore.store
, nglstore : Termstore.store
, lazyProps : Lazystore.store
, lazyNoms : Lazynomstore.store
, lazyBoxes : Lazyboxstore.store
, blockedDiamonds : (Term.index * int) list ref
, branchPoints : int list ref
}
Thanks in advance for your help !
Best Regards.
Since id is typed int it cannot be modified. If you change it to int ref then you can modify it, but you will also have to change the accessors to dereference the ref.
An alternative solution is to create an array that maps from old IDs to new IDs and use this array for presentation, but that seems even more complicated.
Since ints are immutable -- you could take your list of nodes and replace it with a new list of nodes:
fun newID (W(x), i) =
W({
id = i
, predId = #predId x
, creationDepcy = #creationDepcy x
, nominalDepcies = #nominalDepcies x
, pattern = #pattern x
, propositions = #propositions x
, nominals = #nominals x
, diamonds = #diamonds x
, boxes = #boxes x
, disjunctions = #disjunctions x
, nglstore = #nglstore x
, lazyProps = #lazyProps x
, lazyNoms = #lazyNoms x
, lazyBoxes = #lazyBoxes x
, blockedDiamonds = #blockedDiamonds x
, branchPoints = #branchPoints x
});
fun imap _ [] _ = []
| imap f (x::xs) i = f(x,i):: (imap f xs (i+1));
(imap stands for "increment map")
Then if xs is a list of nodes the function call
imap newID xs 0
will yield a new list of nodes with the id fields consecutive ints starting with 0
Disclaimer: I didn't try this on your setup of course, by I created a datatype of records which had an id field and successfully used this approach. Obviously this isn't something you would want to do a lot of because of all the copying, but if it is at one specific point in your code it should be okay.
Mostly equivalent to John's solution, here is how one would update a list of nodes without using references, using a fold instead. The function setId is the same.
fun setIds firstId ws =
#1 (foldr (fn (w,(ws',nextId)) => (setId w nextId::ws', nextId+1)) ([],firstId) ws)
Running setIds 1 [w1, w2, w3, ...] would yield [w1', w2', w3', ...].
I have something like this :
joined = JOIN A BY F1, B BY F1 ;
joinOutput = FOREACH joined GENERATE A::f3 AS f3, A::f4 AS f4, B::f5 AS f5 ;
grouped = GROUP joinOutput BY f3 ;
countOutput = FOREACH grouped FLATTEN(joinOutput) , count(f5) as COUNT ;
if I do """ DESCRIBE countOutput """ then I get following:
countOutput = { joinOutput::f3 :chararray, joinOutput::f4 :int, COUNT :int }
Now if I try to reference f3 with respect to "countOutput" i.e. countOutput.f3 I get error saying invalid field projection.
So my question is how do I project field f3 with respect to countOutput.
I haven't tried this is yet if this is correct but I could think of following ways -
countOutput.joinOutput::f3
Not sure though if this is correct way.
Any help is appreciated.
ok, found solution after trying out few things. I found that you can specify schema explicitly when you FLATTEN.
So this particular step can be re-written as follows :
countOutput = FOREACH grouped FLATTEN(joinOutput) AS ( f3 :chararray, f4: int) , count(f5) as COUNT ;
Now I can directly reference flattened fields with respect to outer relation.
Hope this helps if someone runs into same problem.
I am new to Prolog and wanted to access specific elements of the list.For eg-
L=[name,age,height,weight,gender]
How can I access age and height from this list as I want to compare them with some given values???
A list is not intended to be used as a record. Anyway, just use unification
L = [Name,Age,Height,Weight,Gender],
( Gender == male -> ... ; ... )