I am trying to setup a 3 node cluster (nimbus/supervisor/supervisor), but my conf/storm.yaml file cannot be parsed when running bin/storm nimbus.
conf/storm.yaml:
storm.zookeeper.servers:
- "192.168.1.202"
- "192.168.1.203"
- "192.168.1.204"
storm.local.dir: "/opt/storm-1.2.1”
nimbus.seeds: ["192.168.1.202"]
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
storm.health.check.dir: "healthchecks"
storm.health.check.timeout.ms: 5000
You have a special quotation mark at the end of
storm.local.dir: "/opt/storm-1.2.1”
You can validate your yaml with e.g. http://yaml-online-parser.appspot.com/
Related
in our cluster some apps are sending logs as multiline, and the problem is that the log structure is different from app to app.
How can we set up an 'if' condition that will include the
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after
In it?
Our code:
filebeatConfig:
filebeat.yml: |
filebeat.inputs:
- type: container
paths:
- /var/log/containers/*.log
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
- drop_event:
when:
contains:
container.image.name: "kibana"
output.logstash:
hosts: ["logstash-listener:5044"]
You need to use auto-discovery (either Docker or Kubernetes) with template conditions.
You will probably have at least two templates, one for capturing your containers that emit multiline messages and another for other containers.
filebeat.autodiscover:
providers:
- type: kubernetes
templates:
- condition: <--- your multiline condition goes here
contains:
kubernetes.namespace: xyz-namespace
config:
- type: container
paths:
- /var/lib/docker/containers/${data.docker.container.id}/*.log
multiline:
pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
negate: true
match: after
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
- drop_event:
when:
contains:
container.image.name: "kibana"
I'm a bit new to Grafana so this might be an easy one! I have a simple config-promtail.yaml file loading logs into Loki and everything is working, but I'd like to restrict the log rows passed to Loki to only those lines that include the word "error". Here is what I have:
server:
http_listen_port: <port #>
grpc_listen_port: <port #>
positions:
filename: /tmp/positions.yaml
clients:
- url: 'http://10.128.15.231:3100/loki/api/v1/push'
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: log_export
__path__: /path/to/log/file.log
host: host-name
pipeline_stages:
- match:
selector: '{host="host-name"} |= "error"'
action: keep
it works fine until I add the |= "error
I've also tried something like this:
pipeline_stages:
- match:
selector: '{host="host-name"}'
stages:
- regex:
expression: '.*error.*'
which also throws config errors. it seems like this should be relatively simple, but the documentation is really not clear...thanks in advance for any assistance!
I create a .golangci.yml in working directory, which have more linters than default, and run golangci-lint run -v but found linters are still default value, anyone encountered same issue? I have checked .golangci.yml path is correct. below is our configure value:
linters:
disable-all: true
enable:
- bodyclose
- deadcode
- depguard
- dogsled
- dupl
- errcheck
- exhaustive
- gochecknoinits
- goconst
- gocritic
- gofmt
- gomnd
- goprintffuncname
- gosec
- gosimple
- govet
- ineffassign
- interfacer
- lll
- misspell
- nakedret
- noctx
This yaml file isn't indented properly. try:
linters:
enable:
- bodyclose
- deadcode
- depguard
- dogsled
- dupl
- errcheck
- exhaustive
- gochecknoinits
- goconst
- gocritic
- gofmt
- gomnd
- goprintffuncname
- gosec
- gosimple
- govet
- ineffassign
- interfacer
- lll
- misspell
- nakedret
- noctx
I want to automatically renew SSL certificates provided by let's encrypt.
Here's my plan:
Use crontab to execute certbot renew everyday.
If there's any certificate renewed by certbot renew, use AWS CLI to upload the certificate to a load balancer. (AWS China doesn't have Certificate Manager yet, that's why I use let's encrypt.)
My problem is that I don't know how to detect if there's any certificate renewed by certbot renew command.
This command returns something like the following:
*** renew ***
Saving debug log to /var/log/letsencrypt/letsencrypt.log
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/ooo.com-0001.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not yet due for renewal
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/xxx.com-0002.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not yet due for renewal
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following certs are not due for renewal yet:
/etc/letsencrypt/live/ooo.com-0001/fullchain.pem expires on 2019-12-25 (skipped)
/etc/letsencrypt/live/xxx.com-0002/fullchain.pem expires on 2019-12-25 (skipped)
No renewals were attempted.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
same
I think it would be difficult to parse the string to decide if there's any renewed certificates.
I have Traefik configured to issue Let's Encrypt wildcard certificates with DNS-01 challenge.
I have the variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, AWS_HOSTED_ZONE_ID in the env file, for *.domain1.com (domain1.com). This AWS_HOSTED_ZONE_ID is related to domain1.com only.
I need to add new domain domain2.com also hosted in Route53, so Traefik can issue certificates for both *.domain1.com and *.domain2.com.
How have Traefik issue Letsencrypt certificates in multi Route53 domains?
Next is my treafik.yml file:
version: "3.6"
services:
traefik:
image: traefik
env_file: /mnt/ceph/traefik/env
command:
- "--debug=true"
- "--logLevel=DEBUG"
- "--api"
- "--entrypoints=Name:http Address::80 Redirect.EntryPoint:https"
- "--entrypoints=Name:https Address::443 Compress:true TLS"
- "--defaultentrypoints=http,https"
- "--acme"
- "--acme.storage=acme.json"
- "--acme.acmeLogging=true"
- "--acme.entryPoint=https"
- "--acme.email=email#domain1.com"
#- "--acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory"
- "--acme.caServer=https://acme-v02.api.letsencrypt.org/directory"
- "--acme.dnsChallenge.provider=route53"
- "--acme.dnsChallenge.delayBeforeCheck=0"
- "--acme.domains=*.domain1.com,domain1.com"
- "--docker"
- "--docker.domain=domain1.com"
- "--docker.exposedByDefault=false"
- "--docker.swarmMode"
- "--docker.watch"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /mnt/ceph/traefik/acme.json:/acme.json
networks:
- backend
- webgateway
ports:
- target: 80
published: 80
mode: host
- target: 443
published: 443
mode: host
- target: 8080
published: 8080
mode: host
deploy:
mode: global
placement:
constraints:
- node.role == manager
update_config:
parallelism: 2
failure_action: rollback
order: start-first
#delay: 5s
restart_policy:
condition: on-failure
labels:
- "traefik.enable=true"
- "traefik.backend=dashboard"
- "traefik.port=8080"
- "traefik.frontend.rule=Host:dashboard.domain1.com"
networks:
backend:
name: traefik_backend
driver: overlay
external: true
webgateway:
driver: overlay
Thank you in advance!!