I've added a #searchable directive to my Amplify/GraphQL schema as follows:
type Card
#model
#searchable
{
name: String
id: ID!
}
I've added some items, which I can retrieve with listCards in my AppSync Console:
query MyQuery {
listCards {
items {
name
}
}
}
# Returns:
{
"data": {
"listCards": {
"items": [
{
"name": "hunter"
},
{
"name": "url1"
},
{
"name": "testThur"
},
{
"name": "testThur2"
},
...
}
Now, when I try to use searchCards I can't get it to return anything:
query MyQuery {
searchCards(filter: {name: {ne: "nonsense"}}) {
nextToken
total
items {
name
}
}
}
# Returns:
{
"data": {
"searchCards": {
"nextToken": null,
"total": null,
"items": []
}
}
}
How do I get this working?
I noticed that new cards that I add are returned, but ones that were added before adding the #searchable directive don't get returned.
There's a grey info paragraph in the docs https://docs.amplify.aws/cli/graphql/search-and-result-aggregations/:
Once the #searchable directive is added, all new records added to the model are streamed to OpenSearch. To backfill existing data, see Backfill OpenSearch index from DynamoDB table.
It looks like any previous items that I've created on the database won't be streamed to OpenSearch, and therefore won't be returned by 'search' AppSync calls.
We're directed here: https://docs.amplify.aws/cli/graphql/troubleshooting/#backfill-opensearch-index-from-dynamodb-table
We are instructed to use the provided python file with this command:
python3 ddb_to_es.py \
--rn 'us-west-2' \ # Use the region in which your table and OpenSearch domain reside
--tn 'Post-XXXX-dev' \ # Table name
--lf 'arn:aws:lambda:us-west-2:<...>:function:amplify-<...>-OpenSearchStreamingLambd-<...>' \ # Lambda function ARN, find the DynamoDB to OpenSearch streaming functions, copy entire ARN
--esarn 'arn:aws:dynamodb:us-west-2:<...>:table/Post-<...>/stream/2019-20-03T00:00:00.350' # Event source ARN, copy the full DynamoDB table ARN
(I've tried this with my region, ARN's, and DynamoDB references but when I hit enter in my CLI it just goes to the next command line and nothing happens? I've not used python before. Hopefully someone here has more luck?)
You should run the script like this :
python3 fileNameToYourScript.py --rn <region> --tn <fullTableName> --lf <arnToYourOpenSearchLambdaFunction> --esarn <arnToYourTableName>
Remove the angle brackets and replace them with the actual value no quotation marks...
Another thing, I kept getting an error that credentials couldn't be found, in case you also get it, I fixed it by going to .aws/credentials and duplicating my profile details but naming the copy [default]. Also did the same in the .aws/config, duplication my region details and naming the copy [default].
Related
I was able to perform the task to copy data from the source bucket to a destination bucket using lambda function, however, I got an error while executing the lambda function in Step functions. Below are the steps I followed from the scratch.
Region chosen is ap-south-1
Created 2 buckets. Source bucket: start.bucket & Destination bucket: final.bucket
Created a Lambda function with the following information:
Author from scratch
Function name: CopyCopy
Runtime: Python 3.8
Had created a lambda IAM role: LambdaCopy and gave the necessary policies(S3 full access and Step functions full access) and attached it to the function.
Added a trigger and chose:
S3
Bucket: start.bucket
Event type: All object create events
I found a python code in GeeksforGeeks and applied in the code section.
import json
import boto3
s3_client=boto3.client('s3')
# lambda function to copy file from 1 s3 to another s3
def lambda_handler(event, context):
#specify source bucket
source_bucket_name=event['Records'][0]['s3']['bucket']['name']
#get object that has been uploaded
file_name=event['Records'][0]['s3']['object']['key']
#specify destination bucket
destination_bucket_name='final.bucket'
#specify from where file needs to be copied
copy_object={'Bucket':source_bucket_name,'Key':file_name}
#write copy statement
s3_client.copy_object(CopySource=copy_object,Bucket=destination_bucket_name,Key=file_name)
return {
'statusCode': 3000,
'body': json.dumps('File has been Successfully Copied')
}
- I deployed the code and it worked. Uploaded a csv file in start.bucket and it was copied to final.bucket.
Then, I created a State machine in Step functions with the following information:
Design your workflow visually
Type: Standard
Dragged the AWS Lambda between the Start and End state.
Changed its name to LambdaCopy
Integration type: Optimized
Under API Parameters, Function name(I chose the Lambda function that I had created): CopyCopy:$LATEST
Next State: End
Next and then again Next
State machine name: StepLambdaCopy
IAM Role: Create a new role (Later gave it S3 full access, Lambdafullaccess and Step function fullaccess too).
It showed error when I tried to execute it.
I know I am missing out on something. I would really appreciate the help.
Step functions now allows you to utilize the S3 Copy SDK directly completely bypassing the need for Lambda and boto3. Take a look here for more information.
So in your case you would need a simple task that looks like this:
{
"Comment": "A description of my state machine",
"StartAt": "CopyObject",
"States": {
"CopyObject": {
"Type": "Task",
"End": true,
"Parameters": {
"ServerSideEncryption": "AES256",
"Bucket.$": "$.destination_bucket",
"CopySource.$": "$.source_path",
"Key.$": "$.key"
},
"Resource": "arn:aws:states:::aws-sdk:s3:copyObject"
}
}
}
Then your input state will need to feed in the parameters you would normally use to copy a file with the copy command. Source Path, Destination Bucket, and Object Key exactly the same as the boto3 command.
Note: Your state machine IAM role will need direct S3 permissions and will need to be in the same region as the buckets.
It's always confusing what exactly you have to pass as parameters. Here is a template I use to copy the output of an Athena query. You can adapt it to your needs:
"athena_score": {
"Type": "Task",
"Resource": "arn:aws:states:::athena:startQueryExecution.sync",
"Parameters": {
"QueryExecutionContext": {
"Catalog": "${AthenaCatalog}",
"Database": "${AthenaDatabase}"
},
"QueryString": "SELECT ...",
"WorkGroup": "${AthenaWorkGroup}",
"ResultConfiguration": {
"OutputLocation": "s3://${BucketName}/${OutputPath}"
}
},
"TimeoutSeconds": 300,
"ResultPath": "$.responseBody",
"Next": "copy_csv"
},
"copy_csv": {
"Type": "Task",
"Resource": "arn:aws:states:::aws-sdk:s3:copyObject",
"Parameters": {
"Bucket": "${BucketName}",
"CopySource.$": "States.Format('/${BucketName}/${OutputPath}/{}.csv', $.responseBody.QueryExecution.QueryExecutionId)",
"Key": "${OutputPath}/latest.csv"
},
"ResultPath": "$.responseBody.CopyObject",
"Ent": "true"
}
I am building a GraphQL Schema that has a Type Pod which contains 3 nested objects.
type Pod {
metadata: Metadata
spec: Spec
status: Status
}
My data source is an external API which returns an array of this data. In fact, I defined my schema around this API response. I have included a trimmed down version below.
[
{
"metadata": {
"name": "my-app-65",
"namespace": "default",
},
"spec": {
"containers": [
{
"name": "hello-world",
"image": "container-image.io/unique-id",
}
],
},
// more like-objects in the array
]
However, for each of these container objects, inside the array. I would like to add some extra information which this initial API call does not provide. However, I can query this information separately if I provide the name & namespace properties on the parent's metadata.
/container/endpoint/${namespace}/${name}
Returns...
[ {
name: "hello-world",
nestObj: {
//data
}
},
]
And I would like to add this nested Obj to the original space when that data is queried.
However, I don't have a clean way to access the pod.metadata.name inside the resolver for Container.
Currently my resolvers look like....
Query: {
pods: async () => {
//query that returns array of pod objects
return pods
}
},
Container: {
name: "hello-world",
nestedObj: async (parent, args, context, info) => {
//query that hits the second endpoint but requires name & namespace.
//however, I don't have access to those values.
}
}
Perfect world solution: I could access parent.parent.metadata.name inside the Container Resolver
Current Approach (brute force, repetitively add the property to the children)
Loop through every nested container objs in every Pod and add the podName & namespace as properties there.
pods.forEach(pod => pod.spec.containers.forEach(container => {
container.podName = pod.metadata.name;
container.namespace = pod.metadata.namespace;
}))
This feels very much like a hack, and really bogs down my query times. Especially considering this data won't always be requested.
I have two intuitive ideas but don't know how to implement them (I'm a bit new to graphQL).
Implement a Pod resolver that would then pass this data in through the rootValue as described here: https://github.com/graphql/graphql-js/issues/1098
Access it somewhere inside the info object
The problem the first one, is that my data source only sends me the data as an array, not individual pods and I'm unsure how to pass that array of data into resolvers for individual components.
The problem with second, is that object is very dense. I tried accessing it via the path. But path seems to only store the type, not the actual data.
It's also possible I'm just implementing this completely wrong and welcome such feedback.
Thanks for any guidance, suggestions, or resources.
I'm very new to GraphQL, and i'm trying to perform some example queries to this graph. In particular i'm trying the User schema.
According to that documentation, the schema is the following:
id: ID!
liquidityPositions: [LiquidityPosition!]
usdSwapped: BigDecimal!
And here is query i tried:
{
user (where: {id: "0x7c9C48b7cBEbBDA3268435F20c81f15A538C566C"}) {
id
liquidityPositions
usdSwapped
}
}
This query fails, i keep getting the following response:
{
"errors": [
{
"locations": [
{
"line": 0,
"column": 0
}
],
"message": "No value provided for required argument: `id`"
}
]
}
How can i provide the id field and where am i supposed to provide it? Thanks in advance!
You've got a couple of problems with that query. First, to get a user by id, remove the "where" and curly braces from your query. Secondly, the liquityPositions field needs a selection of subfields. Like so:
{
user (id: "0x7c9C48b7cBEbBDA3268435F20c81f15A538C566C") {
id
liquidityPositions {
id
}
usdSwapped
}
}
That website you linked to will show you errors with your query so you can interactively learn more about what is supported.
I would also suggest running through the introduction to GraphQL here: https://graphql.org/learn/ to get a handle on how things are done.
GraphQL lets you ask for specific fields, the response contains only the fields that you had asked for. For example:
a graphql query like:
{
hero {
name
}
}
will return:
{
"data": {
"hero": {
"name": "R2-D2"
}
}
}
where as a graphQl query like:
{
hero {
name
friends {
name
}
}
}
would return:
{
"data": {
"hero": {
"name": "R2-D2",
"friends": [
{
"name": "Luke"
},
{
"name": "Han Solo"
},
{
"name": "Leia"
}
]
}
}
}
Is there a similar mechanism/library/pattern that can be used in gRPC to achieve the same?
FieldMask is similar in protobuf. It is a list of fields to retain, so the first example would be paths: "hero.name" and the second would be paths: ["hero.name", "hero.friends.name"].
It is probably most frequently used to specify which fields should be changed in an update. But it can equally be used to specify the fields that should be returned.
The server can either process the FieldMask directly (e.g., only using the listed fields in a SELECT SQL query), or it can retrieve all the information and filter the result using FieldMaskUtil.merge() to copy just the requested fields into a new proto message to return to the client.
I am new to GraphQL and I wonder how I can explore an API without a possible wildcard (*) (https://github.com/graphql/graphql-spec/issues/127).
I am currently setting up a headless Craft CMS with GraphQL and I don't really know how my data is nested.
Event with the REST API I have no chance of just getting all the data, because I have to setup all the endpoints and therefore I have to know all field names as well.
So how could I easily explore my CraftCMS data structure?
Thanks for any hints on this.
Cheers
merc
------ Edit -------
If I use #simonpedro s suggestion:
{
__schema {
types {
name
kind
fields {
name
}
}
}
}
I can see a lot of types (?)/fields (?)...
For example I see:
{
"name": "FlexibleContentTeaser",
"kind": "OBJECT",
"fields": [
{
"name": "id"
},
{
"name": "enabled"
},
{
"name": "teaserTitle"
},
{
"name": "text"
},
{
"name": "teaserLink"
},
{
"name": "teaserLinkConnection"
}
]
But now I would like to know how a teaserLink ist structured.
I somehow found out that the teaserLink (it is a field with the type Entries, where I can link to another page) has the properties url & title.
But how would I set up query to explore the properties available within teaserLink?
I tried all sorts of queries, but I am always confrontend with messages like this:
I would be really glad if somebody could give me another pointer how I can find out which properties I can actually query...
Thank you
As far as I'm concerned currently there is no graphql implementation with that capability. However, if what you want to do is to explore the "data structure", i.e, the schema, you should use schema instrospection, which was thought for that (explore the graphql schema). For example, a simple graphql instrospection query would be like this:
{
__schema {
types {
name
kind
fields {
name
}
}
}
}
References:
- https://graphql.org/learn/introspection/
UPDATE for edit:
What you want to do I think is the following:
Make a query like this
{
__schema {
types {
name
kind
fields {
name
type {
fields {
name
}
}
}
}
}
}
And then find the wished type field to grab more information (the fields) from it. Something like this (I don't know if this works, just an idea):
const typeFlexibleContentTeaser = data.__schema.types.find(t => t === "FlexibleContentTeaser")
const teaserLinkField = typeFlexibleContentTeaser.fields.find(f => f.name === "teaserLink")
const teaserLinkField = teaserLinkField.type.fields;
i.e, you have to transverse recursively through the type field.