can not use traefik with dynamic configuration file - go

I'm trying to learn and use traefik.
here is my docker-compose.yaml:
version: "3"
services:
traefik:
image: "traefik:v2.0"
container_name: "traefik"
ports:
- "80:80"
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
- ./traefik:/etc/traefik
- ./docker:/etc/docker
whoami:
image: "containous/whoami"
container_name: "whoami"
and here is my traefik.toml:
[entryPoints]
[entryPoints.web]
address = ":80"
[providers]
[providers.file]
filename = "/etc/docker/dynamic_conf.toml"
[providers.docker]
exposedByDefault = false
[api]
insecure = true
and this is my dynamic_conf.toml:
[http]
[http.routers]
[http.routers.whoami]
rule = "Host(`whoami.localhost`)"
entrypoints = "web"
service = "whoami"
but when i build the image and run it, I get an error:
Cannot start the provider *file.Provider: toml: cannot load TOML value of type string into a Go slice
Screenshot: traefik errors
I couldn't find out the reason, I searched and I changed
filename = "/etc/docker/dynamic_conf.toml"
to
filename = ["/etc/docker/dynamic_conf.toml"]

entryPoints is a slice, not a string.
I'm not sure if you need to change the capitalization, but you definitely need to change it to a slice, like this:
entryPoints = ["web"]
You can find an example for it on this page under Priority > Set priorities -- using the File Provider.
Also, the filename property is a string, so leave it as it was before. See this link:
filename = "/etc/docker/dynamic_conf.toml"

Related

Yaml - how to write mapping(s) in single line

I have the following yaml.
version: "0.1"
services:
svc:
image: test
networks:
- test_net_1
- test_net_2
- test_net_3
networkMapping:
test_net_1:
external: true
test_net_2:
external: true
test_net_3:
external: true
I would like to rewrite the networkMapping in a single line like the following
version: "0.2"
services:
svc:
image: test
networks: ['test_net_1', 'test_net_2', 'test_net_3']
networkMapping: {{'test_net_1': {'external': true}}, {'test_net_2': {'external': true}}, {'test_net_3': {'external': true}}}
but when on lint/parse it returns like this
version: "0.2"
services:
svc:
image: test
networks:
- test_net_1
- test_net_2
- test_net_3
networkMapping:
?
test_net_1':
external: true
: ~
?
test_net_2:
external: true
: ~
?
test_net_3:
external: true
: ~
and it cause error in app 'invalid map key: map[interface {}]interface {}{"test_net_1":map[interface {}]interface {}{"external":true}}'.
I checked with double instead of single quotes and without quotes too. But no luck :(.
We can change to to associate array by replacing first and last {} with [] but the app need it as mapping rather than associate array.
Just wondering if anyone had similar problem and any solution?
many thanks in advance.
You use too many {}. This is how you should write it:
networkMapping: {'test_net_1': {'external': true}, 'test_net_2': {'external': true}, 'test_net_3': {'external': true}}

Endpoint URL for DynamoDB inside localstack's Lambda function

I'm using localstack for local development. I have a DynamoDB table named readings and I'd like to insert items from a lambda function.
I have deployed simple lambda function in python runtime:
import os
import boto3
def lambda_handler(events, context):
DYNAMODB_ENDPOINT_URL = os.environ.get("DYNAMODB_ENDPOINT_URL")
dynamodb = boto3.resource("dynamodb", endpoint_url=DYNAMODB_ENDPOINT_URL)
readings_table = dynamodb.Table(DYNAMODB_READINGS_TABLE_NAME)
readings_table.put_item(Item={"reading_id": "10", "other": "test"})
But I'm getting error: [ERROR] EndpointConnectionError: Could not connect to the endpoint URL: "http://localstack:4569/"
I've tried combinations of localhost and localstack along with ports: 4566 and 4569. All of them fail.
Here's my docker-compse service that I use to start localstack
localstack:
image: localstack/localstack:0.11.2
ports:
- 4566:4566
- 8080:8080
environment:
SERVICES: "dynamodb,sqs,lambda,iam"
DATA_DIR: "/tmp/localstack/data"
PORT_WEB_UI: "8080"
LOCALSTACK_HOSTNAME: localstack
LAMBDA_EXECUTOR: docker
AWS_ACCESS_KEY_ID: "test"
AWS_SECRET_ACCESS_KEY: "test"
AWS_DEFAULT_REGION: "us-east-1"
volumes:
- localstack_volume:/tmp/localstack/data
- /var/run/docker.sock:/var/run/docker.sock
# When a container is started for the first time, it will execute files with extensions .sh that are found in /docker-entrypoint-initaws.d.
# Files will be executed in alphabetical order. You can easily create aws resources on localstack using `awslocal` (or `aws`) cli tool in the initialization scripts.
# source: https://github.com/localstack/localstack/pull/1018/files#diff-04c6e90faac2675aa89e2176d2eec7d8R185
- ./localstack-startup-scripts/:/docker-entrypoint-initaws.d/
What would be the correct endpoint url that I have to set inside in my lambda so that I can send requests to localstack's DynamoDB?
According to the docs LOCALSTACK_HOSTNAME is a read-only env var:
LOCALSTACK_HOSTNAME: Name of the host where LocalStack services are available. Use this hostname as endpoint (e.g., http://${LOCALSTACK_HOSTNAME}:4566) in order to access the services from within your Lambda functions (e.g., to store an item to DynamoDB or S3 from a Lambda).
try with
ports:
- "0.0.0.0:4566-4599:4566-4599"
Hope it helps
By following Robert Taylor's answer, I was able to get the following working in my Java Lambda (this should really be a comment in that answer, but code formatting for big snippet is not adequate in comment, so I'm sharing this as a separate answer to make people's life easier):
var url = System.getenv("LOCALSTACK_HOSTNAME");
var credentials = new BasicAWSCredentials("mock_access_key", "mock_secret_key");
var ep = new AwsClientBuilder.EndpointConfiguration(String.format("http://%s:4566",url), "us-east-1");
s3 = AmazonS3ClientBuilder.standard()
.withEndpointConfiguration(ep)
.withPathStyleAccessEnabled(true)
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.build();
dynamoDBMapper = new DynamoDBMapper(
AmazonDynamoDBClientBuilder.standard()
.withEndpointConfiguration(ep)
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.build()
);

Creating private gke cluster

Creating private gke cluster with yaml.
Currently looking into creating a private gke. tried adding private settings in yaml file but getting error
resources:
- name: myclus
type: gcp-types/container-v1:projects.locations.clusters
properties:
parent: projects/[PROJECT_ID]/locations/[REGION]
cluster:
name: my-clus
zone: [ZONE]
network: [NETWORK]
subnetwork: [SUBNETWORK] ### leave this field blank if using the default network###
initialClusterVersion: "1.13"
nodePools:
- name: my-clus-pool1
initialNodeCount: 1
autoscaling:
enabled: true
minNodeCount: 1
maxNodeCount: 12
management:
autoUpgrade: true
autoRepair: true
config:
machineType: n1-standard-1
diskSizeGb: 15
imageType: cos
diskType: pd-ssd
oauthScopes: ###Change scope to match needs###
- https://www.googleapis.com/auth/cloud-platform
preemptible: false
Looking for it to create a private cluster with no external IPs.
Did you ever had the chance to go over this documentation?
https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters
https://cloud.google.com/kubernetes-engine/docs/how-to/private-clusters#public_master
Well, I also found this other Official Google Document that can help you achieve what you want:
https://cloud.google.com/solutions/creating-kubernetes-engine-private-clusters-with-net-proxies
On the "Creating the Docker Image" section there's a Dockerfile example.
Best of Luck!

Updating the Yaml file, with new fields using ruamel

I am trying to update the yaml file using ruamel python.
proc=subprocess.Popen(['kubectl','get','pod','web3','-o','yaml','--export'], stdout=subprocess.PIPE)
rein=proc.stdout.read()
result, indent, block_seq_indent = ruamel.yaml.util.load_yaml_guess_indent(rein, preserve_quotes=True)
So far I have tried :
result['spec'].append('nodeSelector')
which gives ERROR :
result['spec'].append('nodeSelector')
AttributeError: 'CommentedMap' object has no attribute 'append'
Also tried like this :
result['spec']['nodeSelector']['kubernetes.io/hostname']='kubew1'
gives :
result['spec']['nodeSelector']['kubernetes.io/hostname']='kubew1'
File "/usr/local/lib/python3.6/dist-packages/ruamel/yaml/comments.py", line 752, in __getitem__
return ordereddict.__getitem__(self, key)
KeyError: 'nodeSelector'
My Intial Yaml File is :
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
app: demo
name: web
name: web3
selfLink: /api/v1/namespaces/default/pods/web3
spec:
containers:
- image: aexlab/flask-sample-one
imagePullPolicy: Always
name: web
ports:
- containerPort: 5000
name: http
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-7bcc9
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
And Expected fields I want to add inside 'spec' is :
nodeSelector:
kubernetes.io/hostname: kubew1
Any Ideas how to achieve this with ruamel library.
In your YAML file your root level collection is a mapping and the value for the key spec in that mapping is itself a mapping. Both of those mappings get loaded as dict-like objects using ruamel.yaml named CommentedMap.
As with normal dicts you can add key-value pairs, deleted keys (and their values), and update values for a key, but there is no .append() method, as there is with a list (i.e. appending an extra item to a list).
Your output is a bit terse, but of course you cannot just add nodeSelector to anything (list/sequence nor dict/mapping) and expect that to add kubernetes.io/hostname: kubew1 (a mapping in its own right) automatically.
Your try of:
result['spec']['nodeSelector']['kubernetes.io/hostname'] = 'kubew1'
cannot work because there is no dict result['spec']['nodeSelector'] where you can add the key kubernetes.io/hostname.
You would either first have to create a key with an emtpy dict as value:
result['spec']['nodeSelector'] = {}
result['spec']['nodeSelector']['kubernetes.io/hostname'] = 'kubew1'
or do
result['spec']['nodeSelector'] = {'kubernetes.io/hostname': 'kubew1'}
Please note that the above has nothing much to do with ruamel.yaml, that is just basic Python data structure manipulation. Also note that there are over 100 libraries in the ruamel namespace, out of which ruamel.yaml is just one of several published as open source, so using ruamel is not very clear statement, although of course the context often provides enough information on which library you actually use.

How to use the security group existing in horizon in heat template

I'm newbies on heat yaml template loaded by OpenStack
I've got this command which works fine :
openstack server create --image RHEL-7.4 --flavor std.cpu1ram1 --nic net-id=network-name.admin-network --security-group security-name.group-sec-default value instance-name
I tried to write this heat file with the command above :
heat_template_version: 2014-10-16
description: Simple template to deploy a single compute instance with an attached volume
resources:
my_instance:
type: OS::Nova::Server
properties:
name: instance-name
image: RHEL-7.4
flavor: std.cpu1ram1
networks:
- network: network-name.admin-network
security_group:
- security_group: security-name.group-sec-default
security-group:
type: OS::Neutron::SecurityGroup
properties:
rules: security-name.group-sec-default
my_volume:
type: OS::Cinder::Volume
properties:
size: 10
my_attachment:
type: OS::Cinder::VolumeAttachment
properties:
instance_uuid: { get_resource: my_instance }
volume_id: { get_resource: my_volume }
mountpoint: /dev/vdb
The stack creation failed with the following message error :
openstack stack create -t my_first.yaml First_stack
openstack stack show First_stack
.../...
| stack_status_reason | Resource CREATE failed: BadRequest: resources.my_instance: Unable to find security_group with name or id 'sec_group1' (HTTP 400) (Request-ID: req-1c5d041c-2254-4e43-8785-c421319060d0)
.../...
Thanks for helping,
According to the template guide it is expecting the rules type is of list.
So, change the content of template as below for security-group:
security-group:
type: OS::Neutron::SecurityGroup
properties:
rules: [security-name.group-sec-default]
OR
security-group:
type: OS::Neutron::SecurityGroup
properties:
rules:
- security-name.group-sec-default
After digging, I finally found what was wrong in my heat file. I had to declare my instance like this :
my_instance:
type: OS::Nova::Server
properties:
name: instance-name
image: RHEL-7.4
flavor: std.cpu1ram1
networks:
- network: network-name.admin-network
security_groups: [security-name.group-sec-default]
Thanks for your support

Resources