My apm server can't connect to ES with the following log
2020-04-12T04:08:17.102Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://example.com:9200)): Connection marked
as failed because the onConnect callback failed: resource 'apm-7.4.2-span' exists, but it is not an alias
I tried to 'reset' the index by the following command, it won't work either
$ apm-server setup --index-management
Exiting: resource 'apm-7.4.2-span' exists, but it is not an alias
I tried to setup a policy where apm data is deleted after 3 month, and I think I messed up the index setup.. (I can't remember what I did exactly)
How do I reset the index and start using apm again?
(It's a plus if I can retain the data, but I can sacrifice it)
Related
We are trying to get elastic-apm install (for now on our development systems).
According to Homebrew, we have the latest elasticsearch-oss (7.10.2), kibana-oss (7.10.2) and today installed apm-server-oss (which is at version 7.13.0).
Running a apm-server test output we get:
% apm-server test output
elasticsearch: http://localhost:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: ::1, 127.0.0.1
dial up... OK
TLS... WARN secure connection disabled
talk to server... ERROR Connection marked as failed because the onConnect callback failed: could not connect to a compatible version of Elasticsearch: 400 Bad Request: {"error":{"root_cause":[{"type":"invalid_index_name_exception","reason":"Invalid index name [_license], must not start with '_'.","index_uuid":"_na_","index":"_license"}],"type":"invalid_index_name_exception","reason":"Invalid index name [_license], must not start with '_'.","index_uuid":"_na_","index":"_license"},"status":400}
Because the documentation on getting APM going is somewhat obtuse, perhaps this is a configuration issue. But how to investigate further?
Is the solution to install an earlier version of apm-server? If so....how to actually do that with homebrew?
I had the same issue when using non-oss versions. Managed to fix this by upgrading ElasticSearch and Kibana to 7.13.2.
This is my first time using Mongodb, It connect normally using the IDE from Mongodb university basics m001 course but when I tried to connect from cmd it won't connect and gives me this error
*** You have failed to connect to a MongoDB Atlas cluster. Please ensure that your IP whitelist allows connections from your network.
Error: Authentication failed. :
connect#src/mongo/shell/mongo.js:374:17 #(connect):2:6 exception:
connect failed exiting with code 1
I added the path to my environment & checked the network access from my Atlas and I can connect to it from anywhere
Can anyone help me with this please??
Somehow i could manage to run the opendistro cluster on my local mac using docker with kibana dashboard and elasticsearch APIs working as expected. But getting the following error while communicating to elasticsearch from metricbeat.
"Failed to connect to backoff(elasticsearch(https://localhost:9200)): Get https://localhost:9200: x509: certificate signed by unknown authority"
Is there any way to call elasticsearch with insecure flag from metricbeat?
Ok, I found a solution. Just add the below line in your metricbeat.yaml file under elasticsearch.output
ssl.verification_mode: none
I'm experiencing that the Kubernetes API server fails to start during cluster bootstrapping with the following error log, apparently due to being unable to initialize its "client CA configmap":
E1029 14:35:56.211083 5 client_ca_hook.go:78] Timeout: request did not complete within allowed duration
F1029 14:35:56.211121 5 hooks.go:126] PostStartHook “ca-registration” failed: unable to initialize client CA configmap: timed out waiting for the condition
It seems to happen here in the Kubernetes source code. What might cause this error?
See the full log here.
Update: It seems that my etcd cluster isn't accessible from master nodes, even though the same command works from etcd member machines:
$ sudo ETCDCTL_API=3 etcdctl --cacert=/opt/tectonic/tls/etcd-client-ca.crt \
--cert=/opt/tectonic/tls/etcd-client.crt --key=/opt/tectonic/tls/etcd-client.key \
--endpoints=https://coreos-testing-etcd-0.socialfoodie.club:2379 \
endpoint health
https://coreos-testing-etcd-0.socialfoodie.club:2379 is unhealthy: failed to connect: grpc: timed out when dialing
Error: unhealthy cluster
I found out that despite the cryptic error message in the API server, the cause is that it can't write to the etcd cluster. The reason was that the API server was configured with a different client certificate authority than what the etcd cluster was using, due to a timing issue wrt. copying certificates in my Terraform cluster setup. I figured out that the CA was the problem by using curl to contact the etcd cluster instead of etcdctl, as it gave a clear error message.
Thanks to #johnharris85 for suggesting etcd connectivity being an issue!
I am trying to use mongodb as a service and started with heroku services ie dashboard.heroku.com/apps/scaleqamongo/resources. I tried multiple addons which they call like mongolab, mongohq. I create databases and then try to add that service address into my app as
db: 'mongodb://username:password#candidate.19.mongolayer.com:10292,candidate.18.mongolayer.com:10292/app25406086',
but it gives me error
Error: Cannot determine state of server
Error: No replica set primary available for query with ReadPreference PRIMARY
at ReplSet.checkoutReader (/prod/app/dashboard/node_modules/mongoose/node_modules/mongodb/lib/mongodb/connection/repl_set/repl_set.js:585:14)
at Cursor.nextObject (/prod/app/dashboard/node_modules/mongoose/node_modules/mongodb/lib/mongodb/cursor.js:723:48)
at Cursor.nextObject (/prod/app/dashboard/node_modules/mongoose/node_modules/mongodb/lib/mongodb/scope.js:22:20)
at Collection.findOne (/prod/app/dashboard/node_modules/mongoose/node_modules/mongodb/lib/mongodb/collection/query.js:157:10)
at /prod/app/dashboard/node_modules/connect-mongo/lib/connect-mongo.js:214:18
at MongoStore._get_collection (/prod/app/dashboard/node_modules/connect-mongo/lib/connect-mongo.js:148:21)
at MongoStore.get (/prod/app/dashboard/node_modules/connect-mongo/lib/connect-mongo.js:213:10)
at Object.session [as handle] (/prod/app/dashboard/node_modules/express/node_modules/connect/lib/middleware/session.js:311:11)
at next (/prod/app/dashboard/node_modules/express/node_modules/connect/lib/proto.js:193:15)
at Object.methodOverride [as handle] (/prod/app/dashboard/node_modules/express/node_modules/connect/lib/middleware/methodOverride.js:48:5)
While I am able to connect to the database using the shell command
mongo candidate.19.mongolayer.com:10292/app25406086 -u username -p password
What is the problem? Why am I not able to connect?
This is an intermittent error which got appeared for some time after I started the server but disappeared later on.