How to make a joined well-known full node into an authority node in Substrate, automatically became an validator node? - substrate

I use node-template to practise the tutorials.
I have finished the start a private network and Permissioned Network.
At the end, I can see Charlie joined the network as a full node.
I want to go further, I try to make Charlie to be an authority node, I want to make Charlie's role the same as Alice's and Bob's.
I want to let a node automatically joined and became an validator to generate blocks and finalize blocks.
Previously, Charlie runs as :
./node-template \
--chain=local \
--base-path /tmp/validator3 \
--name charlie \
--node-key=3a9d5b35b9fb4c42aafadeca046f6bf56107bd2579687f069b42646684b94d9e \
--port 30335 \
--ws-port=9946 \
--offchain-worker always
Now Charlie can act as a full node.
I try to modify and make Charlie run as an authority role.
What I do :
clear the db of Charlie.
./node-template purge-chain --chain=local --base-path /tmp/validator3
-y
run Charlie as an validator,
remove --offchain-worker always , add --validator --pruning archive
./node-template \
--chain=local \
--base-path /tmp/validator3 \
--name charlie \
--node-key=3a9d5b35b9fb4c42aafadeca046f6bf56107bd2579687f069b42646684b94d9e \
--port 30335 \
--ws-port 9946 \
--rpc-port 9935 \
--validator \
--pruning archive \
--rpc-methods=unsafe \
--bootnodes /ip4/127.0.0.1/tcp/30333/p2p/12D3KooWFzXtYJhUkMsWTXQodYaXhs6ah52xVExicuFPvmUQoZrE # connecto to Alice, if not specified, could not find peer.
submit auro grandpa keys:
input the auro key:
curl http://127.0.0.1:9935 -H "Content-Type:application/json;charset=utf-8" -d \
'{
"jsonrpc":"2.0",
"id":1,
"method":"author_insertKey",
"params": [
"aura",
"scheme sure language chef bacon second club wild suggest advice awful room",
"0x7a96031a623cb0057299dfe51e9421fde3cc80cf7e534bd7c1fbd84b0d520547"
]
}'
then input the grandpa key:
curl http://127.0.0.1:9935 -H "Content-Type:application/json;charset=utf-8" -d \
'{
"jsonrpc":"2.0",
"id":1,
"method":"author_insertKey",
"params": [
"gran",
"scheme sure language chef bacon second club wild suggest advice awful room",
"0x85935cef010757764ec07fd2ea8665a123283b7d81d2a063211b3d52d8cc215e"
]
}'
Doing this is to insert the SessionKeys for Charlie.
restart Charlie.
5). I stop Alice(or Bob, kill either Alice or Bob), leave Bob and Charlie running. The Best block# is continuing, the finalized# is hanged there, not growing.
I have tried many times and search for a day. Does it need a session pallet to support this? Where am I wrong? what I didn't do?

I got an answer substrate-validator-set. If you want to dynamically add validator on a PoA network, you need to add a session pallet, and let session to manage the auro key and grandpa key.

Related

kubectl patch secret with bash gcloud identity-token, heredoc and printf

I want to create a simple k8s cronjob to keep a gcp identity token fresh in a secret.
A relatively simple problem that I am not able to solve
Given
kubectl patch secret token-test --type json -p=<< END
[
{
"op": "replace",
"path": "/data/token.jwt",
"value": "$(gcloud auth print-identity-token | base64 )"
}
]
END
I want this to be applied with kubectl patch secret token-test --type json -p=$(printf "'%s'" "$json")
I have tried many variants, the weird thing is that if I paste in the result of the heredoc insted of printf it works. But all my efforts fails with (also tried a json doc on a single line)
$ kubectl patch secret token-test --type json -p=$(printf "'%s'" "$json")
error: unable to parse "'[{": yaml: found unexpected end of stream
Whereas this actually works:
printf "'%s'" "$json"|pbcopy
kubectl patch secret sudo-token-test --type json -p '[{ "op": "replace","path": "/data/token.jwt","value": "ZX...Zwo="}]'
secret/token-test patched
I cannot understand what is different when it fails. I understand bash is a bit tricky when it comes to string handling, but I am not sure if this is a bash issue or an issue in kubectl.
It's a slight different approach but, howabout:
printf "foo" > test
kubectl create secret generic freddie \
--namespace=default \
--from-file=./test
kubectl get secret/freddie \
--namespace=default \
--output=jsonpath="{.data.test}" \
| base64 --decode
foo
X="$(printf "bar" | base64)"
kubectl patch secret/freddie \
--namespace=default \
--patch="{\"data\":{\"test\":\"${X}\"}}"
kubectl get secret/freddie \
--namespace=default \
--output=jsonpath="{.data.test}" \
| base64 --decode
bar
NOTE it's not a best practice to use your user (gcloud auth print-identity-token) credentials in this way. Service Accounts are preferred. Service Accounts are intended for machine (rather than a human) auth and they can be more easily revoked.
User credentials grant the bearer all the powers of the user account (and this is likely extensive).
There's a portable alternative in which you create a Kubernetes secret from a Service Account key:
kubectl create secret generic your-key \
--from-file=your-key.json=/path/to/your-key.json
There's a cool-kids who use GKE-mostly approach called Workload Identity
$ json='[{ "op": "replace","path": "/data/token.jwt","value": "'"$(gcloud auth print-identity-token | base64 )"'"}]'
$ kubectl patch secret token-test --type json -p="$json"
secret/token-test patched
By appending the string insted of interpolating in the heredoc this was solved.
Still don't know why the other approach failed though.
EDIT:
The end result for this was to drop kubectl and use curl instead as it fit better inside the docker image.
# Point to the internal API server hostname
APISERVER=https://kubernetes.default.svc
# Path to ServiceAccount token
SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount
# Read this Pod's namespace
NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)
# Read the ServiceAccount bearer token
TOKEN=$(cat ${SERVICEACCOUNT}/token)
# Reference the internal certificate authority (CA)
CACERT=${SERVICEACCOUNT}/ca.crt
SECRET_NAME=$(cat /etc/scripts/secret-name)
# Explore the API with TOKEN
curl --fail --cacert ${CACERT} --header "Authorization: Bearer ${TOKEN}" --request PATCH ${APISERVER}/api/v1/namespaces/${NAMESPACE}/secrets/${SECRET_NAME} \
-H 'Accept: application/json' \
-H "Content-Type: application/json-patch+json" \
-d '[{ "op": "replace","path": "/data/token.jwt","value": "'"$(gcloud auth print-identity-token | base64 -w0 )"'"}]' \
--output /dev/null
As commente by me in DazWilkin answer as well, the user that calls gcloud auth print-identity-token is actually a serviceaccount on k8s authenticated via workload identity on GKE to a GCP ServiceAccount.
I need the token to be able to call AWS without storing the actual credentials by using it to call AWS via Workload Identity Federation

Turtle build:android fails in azuredevops pipeline, what am i missing?

I get this error when running turtle build:android
turtle[4754] ERROR: Failed to build standalone app
err: ConfigError: The expected package.json path: /Users/runner/work/1/s/<key_alias>/package.json does not exist
This is what the script looks like in my .yml
turtle setup:ios \
IOS_DIST_P12_PASSWORD=$(IOS_DIST_P12_PASSWORD) \
turtle build:ios -u $(EXPO_CLI_USER) -p $(EXPO_CLI_PASSWORD)\
--team-id $(APPLE_TEAM_ID) \
--dist-p12-path ./my-app_dist.p12 \
--provisioning-profile-path ./myapp.mobileprovision \
--release-channel deployment
turtle setup:android \
ANDROID_KEYSTORE_PASSWORD=$(ANDROID_KEYSTORE_PASSWORD) \
ANDROID_KEY_PASSWORD=$(ANDROID_KEY_PASSWORD) \
turtle build:android -u $(EXPO_CLI_USER) -p $(EXPO_CLI_PASSWORD)\
--keystore-path ./my-app.jks \
--keystore-alias $(KEY_ALIAS) \
-t apk \
--release-channel deployment
I have all the correct information in the yml and all the env variables are defined. Do I also need to run keytool command before i run turtle build? I have already done it for the app in the past and the filepaths are there.
Same issue for ios, it says the path/package.json doesnt exist.
As a workaround:
Remove the Expo variables in front of the build command (they were already declared as variables) and it worked fine.
For details, please refer to this case.
In addition, you can try to check whether the android.package key is included in the expo object of the app.json file
{
"expo": {
"sdkVersion": "26.0.0",
"name": "TongPos",
"description": "your app desc",
"android": {
"package": "com.sohagfaruque.tongpos"
}
}
}

How to create local "copy" of remote Hasura server?

I want to set up a dev environment of Hasura on my local machine, that replicates my existing production (same tables, same schema, same data).
What are the required steps to achieve this task?
I've found this process to work well.
Create a clean empty local postgresql database and Hasura instance. To update an existing local database, drop it and recreate it.
Dump the schema and data from your existing Hasura server (as per the answer by #protob, but with clean_output set so that manual changes to the output do not have to be made. See pgdump for details.
curl --location --request POST 'https://example.com/v1alpha1/pg_dump' \
--header 'Content-Type: application/json' \
--header 'X-Hasura-Role: admin' \
--header 'Content-Type: text/plain' \
--header 'x-hasura-admin-secret: {SECRET}' \
--data-raw '{ "opts": ["-O", "-x","--inserts", "--schema", "public"], "clean_output": true}' > hasura-db.sql
Import the schema and data locally:
psql -h localhost -U postgres < hasura-db.sql
The local database has all the migrations because we copied the latest schema, so just mark them as applied:
# A simple `hasura migrate apply --skip-execution` may work too!
for x in $(hasura migrate status | grep "Not Present" | awk '{ print $1 }'); do
hasura migrate apply --version $x --skip-execution
done
# and confirm the updated status
hasura migrate status
Now finally apply the Hasura metadata using the hasura CLI:
hasura metadata apply
Enjoy your new instance!
Backup the database.
Run Hasura with the database.
Make sure Hasura metadata is synced.
Hasura has a special endpoint for executing pg_dump on the Postgres instance.
Here is a sample CURL request:
curl --location --request POST 'https://your-remote-hasura.com/v1alpha1/pg_dump' \
--header 'Content-Type: application/json' \
--header 'X-Hasura-Role: admin' \
--header 'Content-Type: text/plain' \
--data-raw '{
"opts": ["-O", "-x","--inserts", "--schema", "public"]
}'
It outputs the schema and data in psql format.
You can use a tool such as Postman for convenience to import, test and run the CURL query.
Please follow the pg_dump documentation to adjust needed opts.
i.e. the above query uses "--inserts" opt, which produces "INSERT INTO" statements in the output.
The output can be copied, pasted and imported directly to Hasura Panel SQL Tab ("COPY FROM stdin" statements result in errors when inserted in the panel).
http://localhost:8080/console/data/sql
Before import, comment out or delete the line CREATE SCHEMA public; from query, because it already exists.
You also have to select tables and relations to be tracked, during or after executing the query.
If the amout of data is bigger, it might be better to use CLI for import.

Can Rasa chatbot initiate a conversation?

I'm planning to deploy the bot based on rasa with a monitoring engine. So if something goes wrong, I would like bot to start the conversation with a specific user. Is that possible?
An easy way to do so, would be to send a user message from your monitoring system to your input channel. So you basically mimic the user initiating the conversation. Note that I directly specify the intent using the / as prefix.
Start Rasa Core with the REST api exposed:
python -m rasa_core.run -d models --enable-api
Then you can send messages to it, e.g.:
curl --request POST \
--url http://localhost:5005/webhooks/rest/webhook \
--header 'content-type: application/json' \
--data '{
"sender": "<sender_id_of_your_user>",
"message": "/inform_about_failure"
}'
#Tobias's solution is the "old way" still valid to manage external events in a pull-based chatbot engine (not just RASA), when we want to push notifications to sender_id.
With the current RASA release, to run the RASA Core server the cli command is:
rasa run --debug --enable-api -m models
And the curl request #Tobias specified is still valid with current RASA version:
curl --request POST \
--url http://localhost:5005/webhooks/rest/webhook \
--header 'content-type: application/json' \
--data '{
"sender": "<sender_id_of_your_user>",
"message": "/inform_about_failure"
}'
On the other hand, the suggested way by RASA now (version 1.9.7), is to use external events management.
That way you can specify also the entities along with the intent to be triggered, as the example shows:
curl \
-H "Content-Type: application/json" \
-X POST \
-d '{"name": "EXTERNAL_dry_plant", "entities": {"plant": "Orchid"}}' \
http://localhost:5005/conversations/<sender_id_of_your_user>/trigger_intent
BTW in RASA, sender_id and conversation_id are different names for the same thing.
Definitely. But rasa_nlu got nothing to do with this, you will be sending a push_messagefrom your backend to the chat client from there rasa_nlu picks up the conversation.

Config not visible on Parse Server Dashboard?

So I've got a successful Parse-Server up and running on Heroku, and a functioning Parse-Server-Dashboard as well.
My issue is that through Parse-Server-Dashboard I'm not able to see the Config section. I've been searching endlessly, but I haven't been able to find any resources on this issue.
I've set my Config Var on Heroku PARSE_EXPERIMENTAL_CONFIG_ENABLED = 1, which allows me to get the config client-side just fine, but I still can't see it in dashboard.
Dashboard is V 1.0.15, and Parse-Sever is V 2.1.6
Here's what I'm seeing:
Here's (something similar to) what I want to see:
Cheers,
update parse-dashboard to the latest version as well as parse-server (you're running 2.1.6 and the current version is 2.2.18) it should be here...
At least I can see it.
You can view/edit your config parameters by using Parse RestAPI.
To view your config parameters:
curl -X GET \
-H "X-Parse-Application-Id: APP_ID_KEY" \
-H "X-Parse-REST-API-Key: REST_API_KEY" \
PATH_TO_PARSE_API + "/config"
Example:
curl -X GET \
-H "X-Parse-Application-Id: zirTLZa3h88Q6BBXj8WkkLxxxxxxxxxxxxxxxxxx" \
-H "X-Parse-REST-API-Key: ydcXhehk9wkz01bDBgpteVxxxxxxxxxxxxxxxxxx" \
https://api.parse.com/1/config
To edit your config parameters:
curl -X PUT \
-H "X-Parse-Application-Id: zirTLZa3h88Q6BBXj8WkkLxxxxxxxxxxxxxxxxxx" \
-H "X-Parse-Master-Key: OXuUPDbmAWOIcYZdzUXR0prxxxxxxxxxxxxxxxxxx" \
-d "{\"params\":{\"Test\":\"222\"}}" \
https://api.parse.com/1/config
NOTE:
To read the config you must to set the REST-API-KEY
To edit you must set your MASTER_KEY
Good Luck

Resources