Somehow i could manage to run the opendistro cluster on my local mac using docker with kibana dashboard and elasticsearch APIs working as expected. But getting the following error while communicating to elasticsearch from metricbeat.
"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Get https://localhost:9200: x509: certificate signed by unknown authority"
Is there any way to call elasticsearch with insecure flag from metricbeat?
Ok, I found a solution. Just add the below line in your metricbeat.yaml file under elasticsearch.output
ssl.verification_mode: none
Related
We are trying to get elastic-apm install (for now on our development systems).
According to Homebrew, we have the latest elasticsearch-oss (7.10.2), kibana-oss (7.10.2) and today installed apm-server-oss (which is at version 7.13.0).
Running a apm-server test output we get:
% apm-server test output
elasticsearch: http://localhost:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: ::1, 127.0.0.1
dial up... OK
TLS... WARN secure connection disabled
talk to server... ERROR Connection marked as failed because the onConnect callback failed: could not connect to a compatible version of Elasticsearch: 400 Bad Request: {"error":{"root_cause":[{"type":"invalid_index_name_exception","reason":"Invalid index name [_license], must not start with '_'.","index_uuid":"_na_","index":"_license"}],"type":"invalid_index_name_exception","reason":"Invalid index name [_license], must not start with '_'.","index_uuid":"_na_","index":"_license"},"status":400}
Because the documentation on getting APM going is somewhat obtuse, perhaps this is a configuration issue. But how to investigate further?
Is the solution to install an earlier version of apm-server? If so....how to actually do that with homebrew?
I had the same issue when using non-oss versions. Managed to fix this by upgrading ElasticSearch and Kibana to 7.13.2.
I have installed knative on microk8s using ubuntu (Ubuntu 20.04 LTS). I am trying basic knative example , but it's not working.
1. First I've tried with kn. (kn was not able to read configuration, so I've exported the configuration using microk8s.config > ~/kubeconfig
kn --kubeconfig ~/kubeconfig service create hello --image gcr.io/knative-samples/helloworld-go --env TARGET=Knative
Internal error occurred: failed calling webhook "webhook.serving.knative.dev": Post "https://webhook.knative-serving.svc:443/defaulting?timeout=30s": x509: certificate is not valid for any names, but wanted to match webhook.knative-serving.svc
I've tried to load using kubectl create, but getting the same error.
Any idea.
What version of Knative are you using?
It looks like your Knative webhook may be generating a certificate with an empty subject. Have you tried connecting to the webhook directly on the cluster via curl -kvv https://webhook.knative-serving.svc:443/defaulting? That should print out the certificate and ignore the validation so that you can test that the service is working.
My apm server can't connect to ES with the following log
2020-04-12T04:08:17.102Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://example.com:9200)): Connection marked
as failed because the onConnect callback failed: resource 'apm-7.4.2-span' exists, but it is not an alias
I tried to 'reset' the index by the following command, it won't work either
$ apm-server setup --index-management
Exiting: resource 'apm-7.4.2-span' exists, but it is not an alias
I tried to setup a policy where apm data is deleted after 3 month, and I think I messed up the index setup.. (I can't remember what I did exactly)
How do I reset the index and start using apm again?
(It's a plus if I can retain the data, but I can sacrifice it)
I am using Open Distro for Elasticsearch rpm. I have setup my LDAP server in config.yml. However when I start my elasticsearch cluster, it gives me this error "Empty file path for opendistro_security.ssl.transport.truststore_
filepath". I am using demo certificates which are given by open distro. Can someone please tell me how can I generate keystore and truststore for demo certificates in Opendistro for elasticsearch?
I tried using demo certificates, but it is not working.
The error I am getting is
Empty file path for opendistro_security.ssl.transport.truststore_filepath
I'm experiencing that the Kubernetes API server fails to start during cluster bootstrapping with the following error log, apparently due to being unable to initialize its "client CA configmap":
E1029 14:35:56.211083 5 client_ca_hook.go:78] Timeout: request did not complete within allowed duration
F1029 14:35:56.211121 5 hooks.go:126] PostStartHook “ca-registration” failed: unable to initialize client CA configmap: timed out waiting for the condition
It seems to happen here in the Kubernetes source code. What might cause this error?
See the full log here.
Update: It seems that my etcd cluster isn't accessible from master nodes, even though the same command works from etcd member machines:
$ sudo ETCDCTL_API=3 etcdctl --cacert=/opt/tectonic/tls/etcd-client-ca.crt \
--cert=/opt/tectonic/tls/etcd-client.crt --key=/opt/tectonic/tls/etcd-client.key \
--endpoints=https://coreos-testing-etcd-0.socialfoodie.club:2379 \
endpoint health
https://coreos-testing-etcd-0.socialfoodie.club:2379 is unhealthy: failed to connect: grpc: timed out when dialing
Error: unhealthy cluster
I found out that despite the cryptic error message in the API server, the cause is that it can't write to the etcd cluster. The reason was that the API server was configured with a different client certificate authority than what the etcd cluster was using, due to a timing issue wrt. copying certificates in my Terraform cluster setup. I figured out that the CA was the problem by using curl to contact the etcd cluster instead of etcdctl, as it gave a clear error message.
Thanks to #johnharris85 for suggesting etcd connectivity being an issue!