As suggested by https://crate.io/docs/crate/guide/en/latest/deployment/cloud/aws/ec2-setup.html. I've been trying to setup a CrateDb (3.0.2) cluster. However setting network.host: _site_ prevents the node from starting. Setting everything back to default settings of course starts the instance but this is bound to localhost.
All I'm trying to do is to evaluate and set up CrateDB cluster. Sadly the error received isn't useful as it seems to complain about heap size (on a fresh Windows install with 8GB ram and empty database). Does anyone know how to set up a production instance ?
are you installing it on amazon web services or on windows? the documentation above is for AWS. here's the correct documentation: https://crate.io/docs/crate/guide/en/latest/scaling/multi-node-setup.html
Related
Using Elasticsearch sink connector to insert data to ES 7.2 instance hosted on VM.
Getting this : Elastic search max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
Is it possible to ignore bootstrap checks ?
How can I increase the virtual memory for Elasticsearch in the docker container ?
Bootstrap checks inspect a variety of Elasticsearch and system settings. If youre in development mode, any bootstrap checks that fail appear as warnings in the Elasticsearch log. If youre in production mode failed bootstrap will cause Elasticsearch to refuse to start.
Elasticsearch work mode is configured implicit. As soon as you configure a network setting like network.host,Elasticsearch assumes that you are moving to production and will upgrade the above warnings to exceptions.
Regarding your specific case you need to increase it on the host machine not the docker by running this command:
sudo sysctl -w vm.max_map_count=262144 . and then restart your docker-containers.
BTW not recommended but if you are running a single node you can skip the bootstrap checks by not binding transport to an external interface or by binding transport to an external interface and setting the discovery type to single-node.
I have an ARM template to deploy all of my resources as the production setup. Initially when I used to deploy the resources through the ARM template, everything was fine and our setup is up and running. We did started a MemSQL cluster into the setup. We have an application running using the MemSQL cluster. Now, we have some changes to the setup, specifically to the Ubuntu VMs, where MemSQL is running, like adding disks, assigning private IP etc.
The question what I have here is, does any changes made through the ARM deployment, effects the, in memory data for the running application on the VMs, specifically the MemSQL data?
the answer is - it depends. it wouldn't delete any data (unless you are using complete deployment mode). but it might (depending on what you are doing) detach data disks, or reboot the vms (if you change the sku, for example), if you change private ip addresses cluster might fall apart, as the nodes won't be able to talk to each other, etc.
I'd recommend having a dev cluster where you test your ARM templates and then you can apply the changes to the prod cluster
We are working on Apache storm with kafka in AWS cluster.
As a developer i want to have a local environment setup to debug the code.
But i checked the prerequisites of Horton sandbox,it requires above 8GB RAM.So we thought to have debugging environment using AWS itself.
Let me know how to do debugging in AWS ot is there any best way to have sandbox with less ram or any another best procedure to follow.
Thanks for your help in advance.
Use Vagrant to manage your EC2 instances, so you can always get the nominated environment for developing
Here is the open source code:
https://github.com/mitchellh/vagrant-aws
The README gives the detail on how to set it with Vagrant.
I successfully deployed Kubernetes on AWS using "getting started on AWS ec2 guide" (http://kubernetes.io/v1.0/docs/getting-started-guides/aws.html), but the disk size of all the minions (kubernetes hosts) is 8gb. I would like to increase the disk size, but I haven't found a way to do it.
I can change the VM size by setting MINION_SIZE (e.g. export MINION_SIZE=m3.medium) prior to installing, but the disk size is still 8gb.
From the Kubernetes install instructions for other cloud providers there's an option to set MINION_DISK_SIZE to set the disk size. I tried that with AWS ec2 installation, and the variable is ignored.
I also poked around the config files, but I didn't see anything obvious.
Any suggestions on how to set the disk size for minions when installing Kubernetes on AWS ec2?
I recently stumbled upon the same issue. Have a look at BLOCK_DEVICE_MAPPINGS in kubernetes/cluster/aws/util.sh. You can modify it to have something more appropriate for a EBS-only minion.
For example:
[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":80}}]
AWS docs: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html
I have faced this very issue and tried the current accepted answer but it looks like Kubernetes is changing quite fast what may make this answer also outdated soon.
To this date, I've tested the solution below that might become or not a definitive solution in the future:
There is this PR on Kubernetes' github project that implements an easy way to ignore the SSD storage by setting KUBE_AWS_STORAGE=ebs before running kubernetes/cluster/kube-up.sh.
Hope it is helpful!
It's a known fact that it is not possible to create a cluster in a single machine by changing ports. The workaround is to add virtual Ethernet devices to our machine and use these to configure the cluster.
I want to deploy a cluster of , let's say 6 nodes, on two ec2 instances. That means, 3 nodes on each machine. Is it possible? What should be the seed nodes address, if it's possible?
Is it a good idea for production?
You can use Datastax AMI on AWS. Datastax Enterprise is a suitable solution for production.
I am not sure about your cluster, because each node need its own config files and it is default. I have no idea how to change it.
There are simple instructions here. When you configure instances settings, you have to write advanced settings for cluster, like --clustername yourCluster --totalnodes 6 --version community etc. You also can install Cassandra manually by installing latest version java and cassandra.
You can build cluster by modifying /etc/cassandra/cassandra.yaml (Ubuntu 12.04) fields like cluster_name, seeds, listener_address, rpc_broadcast and token. Cluster_name have to be same for whole cluster. Seed is master node, which IP you should add for every node. I am confused about tokens