Use PowerQuery to iterate over list of record and call an API - powerquery

In Excel I am trying to call an API for a list of IDs.
The list of ids are stored as a Table in Power Query:
ID
00001
00002
00003
I want to call an API:
Source = Json.Document(Web.Contents("https://localhost:80890/data/datasets/timeseries/00001")),
metadata = Source[metadata]
in
metadata
which returns a table in Power Query like:
ID
MetaDataField1
MetaDataField2
00001
x
a
00002
y
b
00003
z
c
Issues:
How do I look through the list of IDs
Call the API subsisting the ID in the url
Collage the results returned by the API call into one power query table

The short answer is a function query.
To achieve that, start first by creating a new "static" query to retrieve the metadata for the first ID in your list (00001). Once this works and gives you the expected results, then post here the definition of your query and we'll make together some adjustments to your "static" query (the one retrieving just the metadata for ID 00001) in order to change it into a function that takes the ID as an input by defining it as a parameter.
Finally you'll be able to call this function in a custom column and then expand it to show your MetaDataField1, MetaDataField2, etc.

Related

Google Sheet Query: Select misses data when there are different data type in a column?

I have a table like this:
a
b
c
1
2
abc
2
3
4.00
note c2 is text while c3 is a number.
When I do
=QUERY(A1:C,"select *")
The result is like
a
b
c
1
2
2
3
4.00
The "text" in C2 has been missed. You can see the live sheet here:
https://docs.google.com/spreadsheets/d/1UOiP1JILUwgyYUsmy5RzQrpGj7opvPEXE46B3xfvHoQ/edit?usp=sharing
How to deal with this issue?
QUERY is very useful, but it has a main limitation: only can handle one kind of data per column. The other data is left as blank. There are usually ways to try to overcome this from inside the QUERY, but I've found them unfruitful. What you can do is just to use:
={A:C}
You can work with filters by its own, but as a step-by-step to adapt the main features of query: If you need to add conditions, use LAMBDA INDEX and FILTER
For example, to check where A is not null:
=LAMBDA(quer,FILTER(quer,INDEX(quer,,1)<>""))({A:C}) --> with INDEX(quer,,1), I've accesed the first column
Where B is more than one cell and less than other:
=LAMBDA(quer,FILTER(quer,INDEX(quer,,2)>D1,INDEX(quer,,2)<D2))({A:C})
For sorting and limiting an amount of items, use SORTN. For example, you want to sort by 3rd column and limit to 5 higher values in that column:
=LAMBDA(quer,SORTN(FILTER(quer,INDEX(quer,,1)<>""),5,1,3,0))({A:C})
Or, to limit to 5 elements without sorting use ARRAY_CONSTRAIN:
=ARRAY_CONSTRAIN(LAMBDA(quer,FILTER(quer,INDEX(quer,,1)<>""))({A:C}),5)
There are other options, you can use REGEXMATCH and other options, and emulate QUERYs functions without missing data. Let me know!
shenkwen,
If you are comfortable with adding an Google App Script in your sheet to give you a custom function, I have a QUERY replacement function that supports all standard SQL SELECT syntax. I don't analyze the column data to try and force to one type based on which is the most common data in the column - so this is not an issue.
The custom function code - is one file and is at:
https://github.com/demmings/gsSQL/tree/main/dist
After you save, you have a new function from your sheet. In your example, the syntax would be
=gsSQL("select a,b,c from testTable", {{"testTable", "F150:H152", 60, true}})
If your data is on a separate tab called 'testTable'(or whatever you want), the second parameter is not required.
I have typed in your example data into my test sheet (see line 150)
https://docs.google.com/spreadsheets/d/1Zmyk7a7u0xvICrxen-c0CdpssrLTkHwYx6XL00Tb1ws/edit?usp=sharing

how to get data from a view table(which has resource_full) from Google bigquery(google-cloud-ruby gem)

I am working on Sinatra,ruby Application.
I need to get data from one view table which is in Google-BigQuery and I am using google-cloud-bigquery gem in my Application.
here is how am Querying the google-bigquery(ruby code)
bigquery = Google::Cloud::Bigquery.new(<necessary credentials for the application>)
query = "select * from `dataset.table_name` limit 10"
bigquery.query query => #(this query will give me the exact output)
but, when I am querying without limit, like following
query = "select * from `dataset.table_name`"
bigquery.query query
I will get this kind of response.
Google::Cloud::InvalidArgumentError (resourcesExceeded: Resources exceeded during query execution: The query could not be executed)
so, here in this case how can I handle it. as, I need to get all the data from that table I shouldn't give any limit.
Instead of querying whole table - you should use Tabledata: list API (or respective method in client of your choice)
Using List is free and also comes with paging so you can get whole table data page by page

Access - Get the total of a column in a select query

I have an Access database set up that takes a bunch of raw data, splits things up in different 'select' queries and pipes the results into various CSV files, where a dashboard set up in Excel will pick it up.
There's some data that I'm trying to calculate in Access, namely I have a quantity field, and I need to calculate the percentage of for each record. In other words, quantity / total of quantity.
Using my rather limited Access abilities, I tried the following query:
SELECT [Sales].*, [Quantity] / Sum([Quantity]) AS QuantityPercent FROM [Sales];
Which comes up with an error:
Your query does not include the specified expression 'company_name' as part of an aggregate function.
Company_name is the first field of the table, and after some Googling and Binging, I'm still quite confused as to what it means in this context.
To sum it up, my question is this: Is there a way to calculate data based off the total of a column/field?
The easy method is to use DSum:
SELECT
[Sales].*,
[Quantity] / DSum("[Quantity]", "[Sales]") AS QuantityPercent
FROM
[Sales];

Get the root level of dimension in Mdx using olap4j Api

I have a problem that's really killing me. BTW I am new to Mdx and using olap4j Api to construct MDX Query. My problem is with the root element, Following is the code snippet:
Query myQuery = new Query("Generated Query", sales); // where sales is an object of type cube.
QueryDimension productDimension = myQuery.getDimension("Product");
So now I have an object with dimension Product to use. I want some thing like this [Product].[All Products] to be part of the MdxQuery when I add productDimension to an axis. I can hardcode [All Products] but if the dimensionName passed is other than Product, say Stores it would be an issue. SO what I want is, whether it would be possible to fetch some thing get the names like [All Products] or [All Stores] or [definite Measures] which is equivalent to [All Measures] dynamically using olap4j?
PS: This is a method which takes in a dimension name and returns the first Member of the dimension like [Product].[All Products] if product is passed or [Measures].[All Measures/some Measures] if measures are passed.
I was able to figure out on how to get the root element To get some thing like All Products or basically all member name then I need to do some thing as shown below:
Say, I want the all member name for productDimension[object of type Dimension for product] then,
Member allMember = productDimension.getDimension().getDefaultHierarchy().getRootMembers().get(0);
productDimension.include(Selection.Operator.MEMBER, allMember);
By doing so allMember will be part of the Mdx Query.

Can BIRT use the output of one query as the input for another query

I have a report that needs to be generated from two different queries.
The first query retrieves a list of information from one data source
The second query must then be called once for each of the records returned by the first query?
Each result set from the second query will then be displayed in separate report tables.
Any pointers are welcome.
It is not possible to perform a SQL join on the first and second query.
Yes this is possible lets assume you have bind your first query results to data table. you can create another table inside the parent table cell and assign your 2 data set to that table with filter of dataset 1
http://publib.boulder.ibm.com/infocenter/rmc/v7r5m0/index.jsp?topic=/org.eclipse.birt.doc/birt/birt-13-1.html
http://publib.boulder.ibm.com/infocenter/rmc/v7r5m0/index.jsp?topic=/org.eclipse.birt.doc/birt/birt-13-2.html
The functionality I require is provided by SubReports
http://www.eclipse.org/birt/phoenix/examples/reports/birt2.1/subreport/index.php

Resources