Dynamodb connection: strange behaviour - spring

I have created an Amazon DynamoDB database in a Docker container using this request:
curl -X POST http://192.168.99.100:8000/ -H 'accept-encoding: identity' -H 'authorization: AWS4-HMAC-SHA256 Credential=key/20170515/us-east-1/execute-api/aws4_request, SignedHeaders=accept-encoding;content-length;content-type;host;x-amz-date;x-amz-target, Signature=f2f21c6263ad5380aaa' -H 'cache-control: no-cache' -H 'content-type: application/json' -H 'x-amz-date: 20170515T151032Z' -H 'x-amz-target: DynamoDB_20120810.CreateTable' -d '{"AttributeDefinitions": [{"AttributeName": "userId","AttributeType": "S"}],"TableName": "User","KeySchema": [{"AttributeName": "userId","KeyType": "HASH"}],"ProvisionedThroughput": {"ReadCapacityUnits": 1,"WriteCapacityUnits": 1}}'
When I list the tables using a curl command like that:
curl -X POST http://192.168.99.100:8000/ -H 'authorization: AWS4-HMAC-SHA256 Credential=key/20170515/us-east-1/execute-api/aws4_request, SignedHeaders=accept-encoding;content-length;content-type;host;x-am z-date;x-amz-target' -H 'cache-control: no-cache' -H 'content-type: application/json' -H 'x-amz-date: 20 170515T151032Z' -H 'x-amz-target: DynamoDB_20120810.ListTables ' -d '{}'
All works fine. I get the list of the tables:
{"TableNames":["UserTable1","User", "TestTable]}
The problem is when I connect to this database using RazorSQL there is no table on it. I have the same problem with my application spring-boot it raise an exception:
Cannot do operations on a non-existent table (Service: AmazonDynamoDBv2; Status Code: 400;
Would you have any ideas about this strange behaviour ?
this is a screen shot of my connection profile:

When using DynamoDB locally, you should be aware of the following:
If you use the -sharedDb option, DynamoDB creates a single database file named shared-local-instance.db. Every program that connects to DynamoDB accesses this file. If you delete the file, you lose any data you have stored in it.
If you omit -sharedDb, the database file is named myaccesskeyid_region.db, with the AWS access key ID and region as they appear in your application configuration. If you delete the file, you lose any data you have stored in it.
So, make sure you're passing -shareDb.
Those who are using the official DynamoDB Local Docker image can do that like this:
docker run -p 8000:8000 amazon/dynamodb-local -jar DynamoDBLocal.jar -inMemory -sharedDb
The original ENTRYPOINT and CMD used by the image can be seen in docker inspect amazon/dynamodb-local output and are:
"Entrypoint": [
"java"
]
"Cmd": [
"-jar",
"DynamoDBLocal.jar",
"-inMemory"
]
So we basically copied them and added -sharedDb.

Related

How to create index-pattern for a specific tenant in Elasticsearch Kibana using API?

Hi I am following this https://opendistro.github.io/for-elasticsearch-docs/docs/security/access-control/api/#create-tenant documentation to create tenant at Elasticsearch.
We can also create an index using index API https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html.
Is there any way to crate index-pattern with an index for a specific tenant using any API?
(We can do the same manually from Elasticsearch Kibana UI)
Yes there is a way to do it.
You should first create a tenant using security plugin. (This should be installed.)
curl -X PUT $OPENDISTRO_HOSTURL/_opendistro/_security/api/tenants/[tenant_name] –d '{"description":"[description of the tenant]"}' –H "Content-Type: application/json" –u [user]:[pass]
And then you can call API
curl -k -v -X POST -u [user]:[pass] -H 'Content-Type: application/json' -H \"kbn-xsrf: reporting\" -d
'{\"attributes\":{\"title\": \"'\"$index_pattern_name\"'\",\"fields\": \"[{\\\"count\\\":0,\\\"name\\\":\\\"_id\\\",\\\"type\\\":\\\"string\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_index\\\",\\\"type\\\":\\\"string\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_score\\\",\\\"type\\\":\\\"number\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_source\\\",\\\"type\\\":\\\"_source\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false},{\\\"count\\\":0,\\\"name\\\":\\\"_type\\\",\\\"type\\\":\\\"string\\\",\\\"scripted\\\":false,\\\"searchable\\\":false,\\\"aggregatable\\\":false,\\\"readFromDocValues\\\":false}]\"}}'
$KIBANA_HOSTURL/tenant-$TENANT_NAME

How to create local "copy" of remote Hasura server?

I want to set up a dev environment of Hasura on my local machine, that replicates my existing production (same tables, same schema, same data).
What are the required steps to achieve this task?
I've found this process to work well.
Create a clean empty local postgresql database and Hasura instance. To update an existing local database, drop it and recreate it.
Dump the schema and data from your existing Hasura server (as per the answer by #protob, but with clean_output set so that manual changes to the output do not have to be made. See pgdump for details.
curl --location --request POST 'https://example.com/v1alpha1/pg_dump' \
--header 'Content-Type: application/json' \
--header 'X-Hasura-Role: admin' \
--header 'Content-Type: text/plain' \
--header 'x-hasura-admin-secret: {SECRET}' \
--data-raw '{ "opts": ["-O", "-x","--inserts", "--schema", "public"], "clean_output": true}' > hasura-db.sql
Import the schema and data locally:
psql -h localhost -U postgres < hasura-db.sql
The local database has all the migrations because we copied the latest schema, so just mark them as applied:
# A simple `hasura migrate apply --skip-execution` may work too!
for x in $(hasura migrate status | grep "Not Present" | awk '{ print $1 }'); do
hasura migrate apply --version $x --skip-execution
done
# and confirm the updated status
hasura migrate status
Now finally apply the Hasura metadata using the hasura CLI:
hasura metadata apply
Enjoy your new instance!
Backup the database.
Run Hasura with the database.
Make sure Hasura metadata is synced.
Hasura has a special endpoint for executing pg_dump on the Postgres instance.
Here is a sample CURL request:
curl --location --request POST 'https://your-remote-hasura.com/v1alpha1/pg_dump' \
--header 'Content-Type: application/json' \
--header 'X-Hasura-Role: admin' \
--header 'Content-Type: text/plain' \
--data-raw '{
"opts": ["-O", "-x","--inserts", "--schema", "public"]
}'
It outputs the schema and data in psql format.
You can use a tool such as Postman for convenience to import, test and run the CURL query.
Please follow the pg_dump documentation to adjust needed opts.
i.e. the above query uses "--inserts" opt, which produces "INSERT INTO" statements in the output.
The output can be copied, pasted and imported directly to Hasura Panel SQL Tab ("COPY FROM stdin" statements result in errors when inserted in the panel).
http://localhost:8080/console/data/sql
Before import, comment out or delete the line CREATE SCHEMA public; from query, because it already exists.
You also have to select tables and relations to be tracked, during or after executing the query.
If the amout of data is bigger, it might be better to use CLI for import.

Config not visible on Parse Server Dashboard?

So I've got a successful Parse-Server up and running on Heroku, and a functioning Parse-Server-Dashboard as well.
My issue is that through Parse-Server-Dashboard I'm not able to see the Config section. I've been searching endlessly, but I haven't been able to find any resources on this issue.
I've set my Config Var on Heroku PARSE_EXPERIMENTAL_CONFIG_ENABLED = 1, which allows me to get the config client-side just fine, but I still can't see it in dashboard.
Dashboard is V 1.0.15, and Parse-Sever is V 2.1.6
Here's what I'm seeing:
Here's (something similar to) what I want to see:
Cheers,
update parse-dashboard to the latest version as well as parse-server (you're running 2.1.6 and the current version is 2.2.18) it should be here...
At least I can see it.
You can view/edit your config parameters by using Parse RestAPI.
To view your config parameters:
curl -X GET \
-H "X-Parse-Application-Id: APP_ID_KEY" \
-H "X-Parse-REST-API-Key: REST_API_KEY" \
PATH_TO_PARSE_API + "/config"
Example:
curl -X GET \
-H "X-Parse-Application-Id: zirTLZa3h88Q6BBXj8WkkLxxxxxxxxxxxxxxxxxx" \
-H "X-Parse-REST-API-Key: ydcXhehk9wkz01bDBgpteVxxxxxxxxxxxxxxxxxx" \
https://api.parse.com/1/config
To edit your config parameters:
curl -X PUT \
-H "X-Parse-Application-Id: zirTLZa3h88Q6BBXj8WkkLxxxxxxxxxxxxxxxxxx" \
-H "X-Parse-Master-Key: OXuUPDbmAWOIcYZdzUXR0prxxxxxxxxxxxxxxxxxx" \
-d "{\"params\":{\"Test\":\"222\"}}" \
https://api.parse.com/1/config
NOTE:
To read the config you must to set the REST-API-KEY
To edit you must set your MASTER_KEY
Good Luck

Parse server response "Unexpected token"

I'm working on putting a Parse Server on Heroku. I'm using this app:
https://github.com/ParsePlatform/parse-server-example
Uploading to heroku using this guide:
https://devcenter.heroku.com/articles/getting-started-with-nodejs
I've updated the db and server URLs in the parse server code, and everything uploads and deploys properly. However, when I attempt to use cURL to test the server as indicated in this guide:
https://github.com/ParsePlatform/parse-server
I get the following error:
{"error":"Unexpected token '"}
I have copied and pasted the cURL command, modified for my url:
curl -X POST -H "X-Parse-Application-Id: myAppId" -H "Content-Type: application/json" -d '{"score":1337,"playerName":"Sean Plott","cheatMode":false}' http://my-app-name.herokuapp.com/parse/classes/GameScore
Heroku logs show the request coming in (so I know it's going to the right place) but no errors. I'm deploying from Windows 7, if that matters. This is my first experience with heroku and parse server so I'm kind of flying blind. Anybody see the problem?
Try to invert simple quote to double quote for your POST data field, it's work for me :
#here : "{'score':1337,'playerName':'Sean Plott','cheatMode':false}"
curl -X POST -H "X-Parse-Application-Id: myAppId" -H "Content-Type: application/json" -d "{'score':1337,'playerName':'Sean Plott','cheatMode':false}" http://my-app-name.herokuapp.com/parse/classes/GameScore
instead
#here : '{"score":1337,"playerName":"Sean Plott","cheatMode":false}'
curl -X POST -H "X-Parse-Application-Id: myAppId" -H "Content-Type: application/json" -d '{"score":1337,"playerName":"Sean Plott","cheatMode":false}' http://my-app-name.herokuapp.com/parse/classes/GameScore
I had that issue - for me the fix was is I was using a restApi key which is no longer needed in the Parse-Server.
It was also affecting my s3 adapter - was not allowing me to upload any images through dashboard.
I removed the restApi key and everything started working.

How to update a TeamCity build parameter using REST+cURL

I have a configuration parameter called "testing" in one of my build configurations in TeamCity. After taking a look at the TeamCity REST API doc here I could get information about this parameter using the following cURL command line commands on Windows:
(1) curl -X GET -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters
(2) curl -X GET -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
Response:
(1) <?xml version="1.0" encoding="UTF-8" standalone="yes"?><property name="testing" value="11"/></properties>
(2) 11
But then, when I try to update this "testing" build parameter using the following command, I get an error message:
curl -X PUT -d "1" -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
Response:
Error has occurred during request processing (Unsupported Media Type).
Error: javax.ws.rs.WebApplicationException
Not supported request. Please check URL, HTTP method and transfered data are correct.
I already successfully use a similar command to update the buildNumberCounter setting of the same build configuration:
curl -X PUT -d "1" -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/settings/buildNumberCounter
That's why I thought I can do the same with a build parameter in a similar way. What am I missing here?
UPDATE:
I managed to update the "testing" build parameter with value "1" using Fiddler. The request I composed had the following content:
Request: PUT
URL: http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
Request headers: Authorization: Basic (...)
Request body: 1
So the problem with my cURL command above is probably somewhere around the -d "1" option. But where?
UPDATE 2:
I'm not sure if that makes any difference, but I use this cURL build on Windows 7.
Instead of fixing the failing cURL command, as a workaround, we use now Node.js to compose and send the REST request to TeamCity.
The script that needs to be fed to node.exe is as follows:
// Equivalent cURL command:
// curl -X PUT -d "1" -H "Authorization: Basic (...)" http://teamcity:8080/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing
var http = require('http'),
options = {
hostname: 'teamcity',
port: 8080,
path: '/httpAuth/app/rest/buildTypes/id:bt7/parameters/testing',
method: 'PUT',
headers: { 'Authorization': 'Basic (...)' }
},
req;
req = http.request(options, function(res) { });
// write data to request body
req.write('1');
req.end();
Although the workaround works perfectly, I would still like to know what's wrong with the above cURL command?
For parameters that are non-XML like the one you are asking about, just add:
--Header "Content-Type: text/plain"
For parameters that are XML then you'll want to switch that to:
--Header "Content-Type: application/xml"
I was having a hard time figuring this out too but I found the answer. Instead of using -d and -H at the front. Use --data and --header at the end as shown below . I found this in the TeamCity docs buried in a "click to expand" example.
Set build number counter:
curl -v --basic --user <username>:<password> --request PUT http://<teamcity.url>/app/rest/buildTypes/<buildTypeLocator>/settings/buildNumberCounter --data <new number> --header "Content-Type: text/plain"
Set build number format:
curl -v --basic --user <username>:<password> --request PUT http://<teamcity.url>/app/rest/buildTypes/<buildTypeLocator>/settings/buildNumberPattern --data <new format> --header "Content-Type: text/plain"
I guess the REST API expect XML as input, add
-H "Content-type:text/xml"
and put XML as input. If you have a XML file file.xml :
curl -d "#/path/to/file.xml" -H "Content-type:text/xml" (...)

Resources