I am importing some JSON data into AWS QuickSight. It does some flattening of the file as described here:
https://docs.aws.amazon.com/quicksight/latest/user/supported-data-sources.html
Here is a simplified example of what I am importing:
{
"contact":{
"id":"1",
"level2":[
{
"name":"test"
}
]
}
}
The field id is imported with the name contact.id. However the field level2 is ignored, I presume this is because QuickSight will not flatten a nested array, although there are no error messages to confirm this.
Does anyone have any idea how I can import this data? I have considered using jsonParse to create a calculated field, however I can cannot even address the contact field, let alone the contact.level2 field.
Related
I am facing problems with nullable fields when trying to use the data returned when executing my query.
Schema
A simplified schema for demonstration purpose:
type Query {
members: [Instrument!]!
}
type Instrument {
series: SeriesType!
...
}
type SeriesType {
dividendYield: SeriesMethods
...
}
type SeriesMethods{
latest: Float!
...
}
There is nullable data that is resolved at dividendYield and a typical response I get when executing the query is shown in the following section.
Typical Response
A picture of the query and response can be seen here.
So what is the problem???
We are using this data to for visuals on a web app. The data is read in as an object such that you are able to access the data by data.series.dividendYield.latest. The problem comes in when dividendYield returns null as there is there no longer exists a latest field anymore.
For the client side it will always be necessary for the field to be present in the data, even if the parent resolver is null. Is it possible for the resolvers to be setup such that all children fields also return null rather than not appearing in the data response? Any other solutions would also be appreciated.
Let's say my graphql server wants to fetch the following data as JSON where person3 and person5 are some id's:
"persons": {
"person3": {
"id": "person3",
"name": "Mike"
},
"person5": {
"id": "person5",
"name": "Lisa"
}
}
Question: How to create the schema type definition with apollo?
The keys person3 and person5 here are dynamically generated depending on my query (i.e. the area used in the query). So at another time I might get person1, person2, person3 returned.
As you see persons is not an Iterable, so the following won't work as a graphql type definition I did with apollo:
type Person {
id: String
name: String
}
type Query {
persons(area: String): [Person]
}
The keys in the persons object may always be different.
One solution of course would be to transform the incoming JSON data to use an array for persons, but is there no way to work with the data as such?
GraphQL relies on both the server and the client knowing ahead of time what fields are available available for each type. In some cases, the client can discover those fields (via introspection), but for the server, they always need to be known ahead of time. So to somehow dynamically generate those fields based on the returned data is not really possible.
You could utilize a custom JSON scalar (graphql-type-json module) and return that for your query:
type Query {
persons(area: String): JSON
}
By utilizing JSON, you bypass the requirement for the returned data to fit any specific structure, so you can send back whatever you want as long it's properly formatted JSON.
Of course, there's significant disadvantages in doing this. For example, you lose the safety net provided by the type(s) you would have previously used (literally any structure could be returned, and if you're returning the wrong one, you won't find out about it until the client tries to use it and fails). You also lose the ability to use resolvers for any fields within the returned data.
But... your funeral :)
As an aside, I would consider flattening out the data into an array (like you suggested in your question) before sending it back to the client. If you're writing the client code, and working with a dynamically-sized list of customers, chances are an array will be much easier to work with rather than an object keyed by id. If you're using React, for example, and displaying a component for each customer, you'll end up converting that object to an array to map it anyway. In designing your API, I would make client usability a higher consideration than avoiding additional processing of your data.
You can write your own GraphQLScalarType and precisely describe your object and your dynamic keys, what you allow and what you do not allow or transform.
See https://graphql.org/graphql-js/type/#graphqlscalartype
You can have a look at taion/graphql-type-json where he creates a Scalar that allows and transforms any kind of content:
https://github.com/taion/graphql-type-json/blob/master/src/index.js
I had a similar problem with dynamic keys in a schema, and ended up going with a solution like this:
query lookupPersons {
persons {
personKeys
person3: personValue(key: "person3") {
id
name
}
}
}
returns:
{
data: {
persons: {
personKeys: ["person1", "person2", "person3"]
person3: {
id: "person3"
name: "Mike"
}
}
}
}
by shifting the complexity to the query, it simplifies the response shape.
the advantage compared to the JSON approach is it doesn't need any deserialisation from the client
Additional info for Venryx: a possible schema to fit my query looks like this:
type Person {
id: String
name: String
}
type PersonsResult {
personKeys: [String]
personValue(key: String): Person
}
type Query {
persons(area: String): PersonsResult
}
As an aside, if your data set for persons gets large enough, you're going to probably want pagination on personKeys as well, at which point, you should look into https://relay.dev/graphql/connections.htm
Let's say my graphql server wants to fetch the following data as JSON where person3 and person5 are some id's:
"persons": {
"person3": {
"id": "person3",
"name": "Mike"
},
"person5": {
"id": "person5",
"name": "Lisa"
}
}
Question: How to create the schema type definition with apollo?
The keys person3 and person5 here are dynamically generated depending on my query (i.e. the area used in the query). So at another time I might get person1, person2, person3 returned.
As you see persons is not an Iterable, so the following won't work as a graphql type definition I did with apollo:
type Person {
id: String
name: String
}
type Query {
persons(area: String): [Person]
}
The keys in the persons object may always be different.
One solution of course would be to transform the incoming JSON data to use an array for persons, but is there no way to work with the data as such?
GraphQL relies on both the server and the client knowing ahead of time what fields are available available for each type. In some cases, the client can discover those fields (via introspection), but for the server, they always need to be known ahead of time. So to somehow dynamically generate those fields based on the returned data is not really possible.
You could utilize a custom JSON scalar (graphql-type-json module) and return that for your query:
type Query {
persons(area: String): JSON
}
By utilizing JSON, you bypass the requirement for the returned data to fit any specific structure, so you can send back whatever you want as long it's properly formatted JSON.
Of course, there's significant disadvantages in doing this. For example, you lose the safety net provided by the type(s) you would have previously used (literally any structure could be returned, and if you're returning the wrong one, you won't find out about it until the client tries to use it and fails). You also lose the ability to use resolvers for any fields within the returned data.
But... your funeral :)
As an aside, I would consider flattening out the data into an array (like you suggested in your question) before sending it back to the client. If you're writing the client code, and working with a dynamically-sized list of customers, chances are an array will be much easier to work with rather than an object keyed by id. If you're using React, for example, and displaying a component for each customer, you'll end up converting that object to an array to map it anyway. In designing your API, I would make client usability a higher consideration than avoiding additional processing of your data.
You can write your own GraphQLScalarType and precisely describe your object and your dynamic keys, what you allow and what you do not allow or transform.
See https://graphql.org/graphql-js/type/#graphqlscalartype
You can have a look at taion/graphql-type-json where he creates a Scalar that allows and transforms any kind of content:
https://github.com/taion/graphql-type-json/blob/master/src/index.js
I had a similar problem with dynamic keys in a schema, and ended up going with a solution like this:
query lookupPersons {
persons {
personKeys
person3: personValue(key: "person3") {
id
name
}
}
}
returns:
{
data: {
persons: {
personKeys: ["person1", "person2", "person3"]
person3: {
id: "person3"
name: "Mike"
}
}
}
}
by shifting the complexity to the query, it simplifies the response shape.
the advantage compared to the JSON approach is it doesn't need any deserialisation from the client
Additional info for Venryx: a possible schema to fit my query looks like this:
type Person {
id: String
name: String
}
type PersonsResult {
personKeys: [String]
personValue(key: String): Person
}
type Query {
persons(area: String): PersonsResult
}
As an aside, if your data set for persons gets large enough, you're going to probably want pagination on personKeys as well, at which point, you should look into https://relay.dev/graphql/connections.htm
I created a field in a model named "Listing". The field in here is "category". This is set to enum of some values (screenshot below). Does graphql allow to query to fetch the enum list?
This is possible using a GraphQL introspection query.
Enum types on Graphcool are named "MODEL_FIELD", so in your case you can run this query:
{
__type(name: "LISTING_CATEGORY") {
name
enumValues {
name
}
}
}
You can find more information in this answer to a similar question.
An easier approach, when using Nexus.js, is to just create a new root type, and have it resolve to your array of ENUM values. This works because with Nexus.js you're writing your schema with code (TypeScript), which means you're able to import your array of ENUM values into the resolver — it's just code.
Graphql is great and I've started using it in my app. I have a page that displays summary information and I need graphql to return aggregate counts? Can this be done?
You would define a new GraphQL type that is an object that contains a list and a number. The number would be defined by a resolver function.
On your GraphQL server you can define the resolver function and as part of that, you would have to write the code that performs whatever calculations and queries are necessary to get the aggregate counts.
This is similar to how you would write an object serializer for a REST API or a custom REST API endpoint that runs whatever database queries are needed to calculate the aggregate counts.
GraphQL's strength is that it gives the frontend more power in determining what data specifically is returned. Some of what you write in GraphQL will be the same as what you would write for a REST API.
There's no automatic aggregate function in GraphQL itself.
You can add a field called summary, and in the resolve function calculate the totals.
You should define a Type of aggregated data in Graphql and a function you want to implement it. For example, if you want to write the following query:
SELECT age, sum(score) from student group by age;
You should define the data type that you want to return:
type StudentScoreByAge{
age: Int
sumOfScore: Float
}
and a Graphql function:
getStudentScoreByAge : [StudentScoreByAge]
async function(){
const res = await client.query("SELECT age, sum(score) as sumOfScore
from Student group by age");
return res.rows;
}
... need graphql to return aggregate counts? Can this be done?
Yes, it can be done.
Does GraphQL does it automatically for you? No, because it does not know / care about where you get your data source.
How? GraphQL does not dictate how you get / mutate the data that the user has queried. It's up to your implementation to get the requested aggregated data. You could get aggregated data directly from your MongoDB and serve it back, or you get all the data you need from your data source and do the aggregation yourself.
If you are using Hasura, in the explorer, you can definitely see an "agregate" table name, thus, your query would look something similar to the following:
query queryTable {
table_name {
field1
field2
}
table_name_aggregate {
aggregate { count }
}
}
In your results, you will see the total row count for the query
"table_name_aggregate": {
"aggregate": {
"count": 9973
}
This depends on whether you build the aggregator into your schema and are able to resolve the field.
Can you share what kind of GraphQL Server you're running? As different languages have different implementations, as well as different services (like Hasura, 8base, and Prisma).
Also, when you say "counts", I'm imagining a count of objects in a relation. Such as:
query {
user(id: "1") {
name
summaries {
count
}
}
}
// returns
{
"data": {
"user": {
"name": "Steve",
"summaries": {
"count": 10
}
}
}
}
8base provides the count aggregate by default on relational queries.