How to create local "copy" of remote Hasura server? - graphql

I want to set up a dev environment of Hasura on my local machine, that replicates my existing production (same tables, same schema, same data).
What are the required steps to achieve this task?

I've found this process to work well.
Create a clean empty local postgresql database and Hasura instance. To update an existing local database, drop it and recreate it.
Dump the schema and data from your existing Hasura server (as per the answer by #protob, but with clean_output set so that manual changes to the output do not have to be made. See pgdump for details.
curl --location --request POST 'https://example.com/v1alpha1/pg_dump' \
--header 'Content-Type: application/json' \
--header 'X-Hasura-Role: admin' \
--header 'Content-Type: text/plain' \
--header 'x-hasura-admin-secret: {SECRET}' \
--data-raw '{ "opts": ["-O", "-x","--inserts", "--schema", "public"], "clean_output": true}' > hasura-db.sql
Import the schema and data locally:
psql -h localhost -U postgres < hasura-db.sql
The local database has all the migrations because we copied the latest schema, so just mark them as applied:
# A simple `hasura migrate apply --skip-execution` may work too!
for x in $(hasura migrate status | grep "Not Present" | awk '{ print $1 }'); do
hasura migrate apply --version $x --skip-execution
done
# and confirm the updated status
hasura migrate status
Now finally apply the Hasura metadata using the hasura CLI:
hasura metadata apply
Enjoy your new instance!

Backup the database.
Run Hasura with the database.
Make sure Hasura metadata is synced.

Hasura has a special endpoint for executing pg_dump on the Postgres instance.
Here is a sample CURL request:
curl --location --request POST 'https://your-remote-hasura.com/v1alpha1/pg_dump' \
--header 'Content-Type: application/json' \
--header 'X-Hasura-Role: admin' \
--header 'Content-Type: text/plain' \
--data-raw '{
"opts": ["-O", "-x","--inserts", "--schema", "public"]
}'
It outputs the schema and data in psql format.
You can use a tool such as Postman for convenience to import, test and run the CURL query.
Please follow the pg_dump documentation to adjust needed opts.
i.e. the above query uses "--inserts" opt, which produces "INSERT INTO" statements in the output.
The output can be copied, pasted and imported directly to Hasura Panel SQL Tab ("COPY FROM stdin" statements result in errors when inserted in the panel).
http://localhost:8080/console/data/sql
Before import, comment out or delete the line CREATE SCHEMA public; from query, because it already exists.
You also have to select tables and relations to be tracked, during or after executing the query.
If the amout of data is bigger, it might be better to use CLI for import.

Related

How to create index-pattern for a specific tenant in Elasticsearch Kibana using API?

Hi I am following this https://opendistro.github.io/for-elasticsearch-docs/docs/security/access-control/api/#create-tenant documentation to create tenant at Elasticsearch.
We can also create an index using index API https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html.
Is there any way to crate index-pattern with an index for a specific tenant using any API?
(We can do the same manually from Elasticsearch Kibana UI)
Yes there is a way to do it.
You should first create a tenant using security plugin. (This should be installed.)
curl -X PUT $OPENDISTRO_HOSTURL/_opendistro/_security/api/tenants/[tenant_name] –d '{"description":"[description of the tenant]"}' –H "Content-Type: application/json" –u [user]:[pass]
And then you can call API
curl -k -v -X POST -u [user]:[pass] -H 'Content-Type: application/json' -H \"kbn-xsrf: reporting\" -d
'{\"attributes\":{\"title\": \"'\"$index_pattern_name\"'\",\"fields\": \"[{\\\"count\\\":0,\\\"name\\\":\\\"_id\\\",\\\"type\\\":\\\"string\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_index\\\",\\\"type\\\":\\\"string\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_score\\\",\\\"type\\\":\\\"number\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_source\\\",\\\"type\\\":\\\"_source\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_type\\\",\\\"type\\\":\\\"string\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false}]\"}}'
$KIBANA_HOSTURL/tenant-$TENANT_NAME

Elasticsearch read only user

I wanted to add a read only user to my cluster, my app prefixes all its indexes with myapp_.
Following https://www.elastic.co/blog/user-impersonation-with-x-pack-integrating-third-party-auth-with-kibana (what a strange title for the only actually usable blog post on this...) I have first added a role with
curl -XPOST '$ELASTIC_URL:9200/_xpack/security/role/name_of_readonly_role' \
-H 'Content-Type: application/json' \
-d'{"indices":[{"names":"myapp_*","privileges":["read"]}]}'
and then added it to a user:
curl -XPOST $ELASTIC_URL:9200/_xpack/security/user/name_of_user \
-H 'Content-Type: application/json' \
-d'{"roles":["name_of_readonly_role"],"password":"some_password"}'
but when opening $ELASTIC_URL:9200 I got
action [cluster:monitor/main] is unauthorized for user
what's next?
There's a complete dearth of examples for this as far as I can see, to fix this problem the role command needs to be re-run with -d'{"cluster":["monitor"], "indices":[{"names":"myapp_*","privileges":["read"]}]}' (same curl command works for creating or updating roles). This seems to leak the name of all indexes but not much else aside from their names and I was fine with that. And even that seems to be not enough for some apps like the ElasticSearch Head brower extension, I needed to add the index level monitor privilege as well: -d'{"cluster":["monitor"], "indices":[{"names":"myapp_*","privileges":["read", "monitor"]}]}'. Role changes are automatically applied to users.
I still have no idea what the "/main" relates to in the error message but this works.

Dynamodb connection: strange behaviour

I have created an Amazon DynamoDB database in a Docker container using this request:
curl -X POST http://192.168.99.100:8000/ -H 'accept-encoding: identity' -H 'authorization: AWS4-HMAC-SHA256 Credential=key/20170515/us-east-1/execute-api/aws4_request, SignedHeaders=accept-encoding;content-length;content-type;host;x-amz-date;x-amz-target, Signature=f2f21c6263ad5380aaa' -H 'cache-control: no-cache' -H 'content-type: application/json' -H 'x-amz-date: 20170515T151032Z' -H 'x-amz-target: DynamoDB_20120810.CreateTable' -d '{"AttributeDefinitions": [{"AttributeName": "userId","AttributeType": "S"}],"TableName": "User","KeySchema": [{"AttributeName": "userId","KeyType": "HASH"}],"ProvisionedThroughput": {"ReadCapacityUnits": 1,"WriteCapacityUnits": 1}}'
When I list the tables using a curl command like that:
curl -X POST http://192.168.99.100:8000/ -H 'authorization: AWS4-HMAC-SHA256 Credential=key/20170515/us-east-1/execute-api/aws4_request, SignedHeaders=accept-encoding;content-length;content-type;host;x-am z-date;x-amz-target' -H 'cache-control: no-cache' -H 'content-type: application/json' -H 'x-amz-date: 20 170515T151032Z' -H 'x-amz-target: DynamoDB_20120810.ListTables ' -d '{}'
All works fine. I get the list of the tables:
{"TableNames":["UserTable1","User", "TestTable]}
The problem is when I connect to this database using RazorSQL there is no table on it. I have the same problem with my application spring-boot it raise an exception:
Cannot do operations on a non-existent table (Service: AmazonDynamoDBv2; Status Code: 400;
Would you have any ideas about this strange behaviour ?
this is a screen shot of my connection profile:
When using DynamoDB locally, you should be aware of the following:
If you use the -sharedDb option, DynamoDB creates a single database file named shared-local-instance.db. Every program that connects to DynamoDB accesses this file. If you delete the file, you lose any data you have stored in it.
If you omit -sharedDb, the database file is named myaccesskeyid_region.db, with the AWS access key ID and region as they appear in your application configuration. If you delete the file, you lose any data you have stored in it.
So, make sure you're passing -shareDb.
Those who are using the official DynamoDB Local Docker image can do that like this:
docker run -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -inMemory -sharedDb
The original ENTRYPOINT and CMD used by the image can be seen in docker inspect amazon/dynamodb-local output and are:
"Entrypoint": [
"java"
]
"Cmd": [
"-jar",
"DynamoDBLocal.jar",
"-inMemory"
]
So we basically copied them and added -sharedDb.

Config not visible on Parse Server Dashboard?

So I've got a successful Parse-Server up and running on Heroku, and a functioning Parse-Server-Dashboard as well.
My issue is that through Parse-Server-Dashboard I'm not able to see the Config section. I've been searching endlessly, but I haven't been able to find any resources on this issue.
I've set my Config Var on Heroku PARSE_EXPERIMENTAL_CONFIG_ENABLED = 1, which allows me to get the config client-side just fine, but I still can't see it in dashboard.
Dashboard is V 1.0.15, and Parse-Sever is V 2.1.6
Here's what I'm seeing:
Here's (something similar to) what I want to see:
Cheers,
update parse-dashboard to the latest version as well as parse-server (you're running 2.1.6 and the current version is 2.2.18) it should be here...
At least I can see it.
You can view/edit your config parameters by using Parse RestAPI.
To view your config parameters:
curl -X GET \
-H "X-Parse-Application-Id: APP_ID_KEY" \
-H "X-Parse-REST-API-Key: REST_API_KEY" \
PATH_TO_PARSE_API + "/config"
Example:
curl -X GET \
-H "X-Parse-Application-Id: zirTLZa3h88Q6BBXj8WkkLxxxxxxxxxxxxxxxxxx" \
-H "X-Parse-REST-API-Key: ydcXhehk9wkz01bDBgpteVxxxxxxxxxxxxxxxxxx" \
https://api.parse.com/1/config
To edit your config parameters:
curl -X PUT \
-H "X-Parse-Application-Id: zirTLZa3h88Q6BBXj8WkkLxxxxxxxxxxxxxxxxxx" \
-H "X-Parse-Master-Key: OXuUPDbmAWOIcYZdzUXR0prxxxxxxxxxxxxxxxxxx" \
-d "{\"params\":{\"Test\":\"222\"}}" \
https://api.parse.com/1/config
NOTE:
To read the config you must to set the REST-API-KEY
To edit you must set your MASTER_KEY
Good Luck

Parse server response "Unexpected token"

I'm working on putting a Parse Server on Heroku. I'm using this app:
https://github.com/ParsePlatform/parse-server-example
Uploading to heroku using this guide:
https://devcenter.heroku.com/articles/getting-started-with-nodejs
I've updated the db and server URLs in the parse server code, and everything uploads and deploys properly. However, when I attempt to use cURL to test the server as indicated in this guide:
https://github.com/ParsePlatform/parse-server
I get the following error:
{"error":"Unexpected token '"}
I have copied and pasted the cURL command, modified for my url:
curl -X POST -H "X-Parse-Application-Id: myAppId" -H "Content-Type: application/json" -d '{"score":1337,"playerName":"Sean Plott","cheatMode":false}' http://my-app-name.herokuapp.com/parse/classes/GameScore
Heroku logs show the request coming in (so I know it's going to the right place) but no errors. I'm deploying from Windows 7, if that matters. This is my first experience with heroku and parse server so I'm kind of flying blind. Anybody see the problem?
Try to invert simple quote to double quote for your POST data field, it's work for me :
#here : "{'score':1337,'playerName':'Sean Plott','cheatMode':false}"
curl -X POST -H "X-Parse-Application-Id: myAppId" -H "Content-Type: application/json" -d "{'score':1337,'playerName':'Sean Plott','cheatMode':false}" http://my-app-name.herokuapp.com/parse/classes/GameScore
instead
#here : '{"score":1337,"playerName":"Sean Plott","cheatMode":false}'
curl -X POST -H "X-Parse-Application-Id: myAppId" -H "Content-Type: application/json" -d '{"score":1337,"playerName":"Sean Plott","cheatMode":false}' http://my-app-name.herokuapp.com/parse/classes/GameScore
I had that issue - for me the fix was is I was using a restApi key which is no longer needed in the Parse-Server.
It was also affecting my s3 adapter - was not allowing me to upload any images through dashboard.
I removed the restApi key and everything started working.

Resources