Nomad Job - Failed to place all allocations - nomad

I’m trying to deploy an AWS EBS volume via nomad but I’m this below error. How do I resolve it?
$ nomad job plan -var-file bambootest.vars bamboo2.nomad
+/- Job: “bamboo2”
+/- Stop: “true” => “false”
+/- Task Group: “main” (1 create)
Volume {
AccessMode: “single-node-writer”
AttachmentMode: “file-system”
Name: “bambootest”
PerAlloc: “false”
ReadOnly: “false”
Source: “bambootest”
Type: “csi”
}
Task: “web”
Scheduler dry-run:
WARNING: Failed to place all allocations.
Task Group “main” (failed to place 1 allocation):
Class “system”: 3 nodes excluded by filter
Class “svt”: 2 nodes excluded by filter
Class “devtools”: 2 nodes excluded by filter
Class “bambootest”: 2 nodes excluded by filter
Class “ambt”: 2 nodes excluded by filter
Constraint “${meta.namespace} = bambootest”: 9 nodes excluded by filter
Constraint “missing CSI Volume bambootest”: 2 nodes excluded by filter
Below is an excerpt of the volume block that seems to be the problem.
group main {
count = 1
volume "bambootest" {
type = "csi"
source = "bambootest"
read_only = false
access_mode = "single-node-writer"
attachment_mode = "file-system"
}
task web {
driver = "docker"

Related

Elastic search - No node attributes configured in elasticsearch.yml You can't control shard allocation without node attributes

When setting up index cold/warm phases in Kibana, this error appears:
No node attributes configured in elasticsearch.yml
You can't control shard allocation without node attributes.
Learn about shard allocation. https://www.elastic.co/guide/en/elasticsearch/reference/7.3/shards-allocation.html
I have added the shard allocation configuration (detailed in the above link) to elasticsearch.yml on all 3 nodes, but still get the same error. What am I missing?
#SHARD ALLOCATION
cluster.routing.allocation.enable: All
cluster.routing.allocation.node_concurrent_incoming_recoveries: 2
cluster.routing.allocation.node_concurrent_outgoing_recoveries: 2
cluster.routing.allocation.node_concurrent_recoveries: 2
cluster.routing.allocation.node_initial_primaries_recoveries: 4
cluster.routing.allocation.same_shard.host: false
#SHARD BALANCING
cluster.routing.rebalance.enable: all
cluster.routing.allocation.allow_rebalance: indices_all_active
cluster.routing.allocation.cluster_concurrent_rebalance: 2
cluster.routing.allocation.balance.shard: 0.45f
cluster.routing.allocation.balance.index: 0.55f
cluster.routing.allocation.balance.threshold: 1.0f
#DISK SPACE MANAGEMENT
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 85%
cluster.routing.allocation.disk.watermark.high: 90%
cluster.routing.allocation.disk.watermark.flood_stage: 95%
cluster.info.update.interval: 240s
cluster.routing.allocation.disk.include_relocations: true
To define a node attribute, you should add a setting like:
node.attr.[some attribute]: value
like
node.attr.box_type: hot
and later you can change an index setting and make it to be on a so cold hot node:
PUT /logs_2016-12-26
{
"settings": {
"index.routing.allocation.require.box_type": "hot"
}
}
box_type is an arbitrary attribute here.
For more details you can check here and here

Terraform: Resource 'aws_instance' not found for variable 'aws_instance.id'

I have started using terraform to automate AWS resource provisioning for setting up k8s cluster. I am facing an issue when trying to refer aws_instance.id from aws_eip. Here are the useful details:
aditya#aditya-VirtualBox:~/Desktop/terraform-states$ terraform -v
Terraform v0.11.11
+ provider.aws v1.54.0
1) aws-eip.tf
resource "aws_eip" "nat" {
instance = "${aws_instance.xenial.id}"
vpc = true
depends_on = ["aws_internet_gateway.esya_igw"]
}
2) aws_inst.tf:
resource "aws_instance" "xenial" {
ami = "${var.aws_ami}"
instance_type = "t3.large"
ebs_optimized = true
monitoring = true
count = "8"
key_name = "${var.aws_key_name}"
tags{
Name = "KubeVMCluster${count.index + 1}"
}
}
Expected Behavior: AWS EIP must be able to refer to AWS Instance.
Current Behavior: We are getting this error:
aditya#aditya-VirtualBox:~/Desktop/terraform-states$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
Error: Error running plan: 1 error(s) occurred:
* aws_eip.nat: 1 error(s) occurred:
* aws_eip.nat: Resource 'aws_instance.xenial' not found for variable 'aws_instance.xenial.id'
I have tried to find a solution by referring to similar kind of issues in Github and elsewhere, but to no avail. According to me, I don't find anything problematic with the declarative code.
I need help in resolving this issue.
Regards
Aditya

How do i know the service status in ansible?

In my ansible coding i want to know the status of the service like service httpd status (service is runngin or not) the result would be store in to variable. Using that status i will use some other code in ansible.
I am using ansible service module there is no option for status. If i use the shell module i got this warning
[WARNING]: Consider using service module rather than running service
so is it any other module doing to get service status?
No, there is no standard module to get services' statuses.
But you can suppress warning for specific command task if you know what are you doing:
- command: service httpd status
args:
warn: false
I've posted a quick note about this trick a while ago.
You can use the service_facts module.
For example, say I want to see the status of Apache.
- name: Check for apache status
service_facts:
- debug:
var: ansible_facts.services.apache2.state
The output is:
ok: [192.168.blah.blah] => {
"ansible_facts.services.apache2.state": "running"
}
If you would like to see all of them, you can do that by just going two levels up in the array:
var: ansible_facts.services
The output will list all the services, and will look like this (truncated for the sake of brevity):
"apache2": {
"name": "apache2",
"source": "sysv",
"state": "running"
},
"apache2.service": {
"name": "apache2.service",
"source": "systemd",
"state": "running"
},
"apparmor": {
"name": "apparmor",
"source": "sysv",
"state": "running"
},
etc,
etc
I am using Ansible 2.7. Here are the docs for that module: Click here
here is an example of starting a service and then checking status using service facts, in my example you have to register the variable then output it using debug var and pointing to the correct format in the json chain resulting output:
## perform start service for alertmanager
- name: Start service alertmanager if not started
become: yes
service:
name: alertmanager
state: started
## check to see the state of the alertmanager service status
- name: Check status of alertmanager service
service_facts:
register: service_state
- debug:
var: service_state.ansible_facts.services["alertmanager.service"].state
Hopefully service: allow user to query service status #3316 will be merged into the core module soon.
You can patch it by hand using this diff to system/service.py
Here's my diff using ansible 2.2.0.0. I've run this on my mac/homebrew install and it works for me.
This is the file that I edited: /usr/local/Cellar/ansible/2.2.0.0_2/libexec/lib/python2.7/site-packages/ansible/modules/core/system/service.py
## -36,11 +36,12 ##
- Name of the service.
state:
required: false
- choices: [ started, stopped, restarted, reloaded ]
+ choices: [ started, stopped, status, restarted, reloaded ]
description:
- C(started)/C(stopped) are idempotent actions that will not run
- commands unless necessary. C(restarted) will always bounce the
- service. C(reloaded) will always reload. B(At least one of state
+ commands unless necessary. C(status) would report the status of
+ the service C(restarted) will always bounce the service.
+ C(reloaded) will always reload. B(At least one of state
and enabled are required.)
sleep:
required: false
## -1455,7 +1456,7 ##
module = AnsibleModule(
argument_spec = dict(
name = dict(required=True),
- state = dict(choices=['running', 'started', 'stopped', 'restarted', 'reloaded']),
+ state = dict(choices=['running', 'started', 'stopped', 'status', 'restarted', 'reloaded']),
sleep = dict(required=False, type='int', default=None),
pattern = dict(required=False, default=None),
enabled = dict(type='bool'),
## -1501,6 +1502,9 ##
else:
service.get_service_status()
+ if module.params['state'] == 'status':
+ module.exit_json(state=service.running)
+
# Calculate if request will change service state
service.check_service_changed()

Indexing tuples from storm to elasticsearch with elasticsearch-hadoop library does not work

I want to index documents into Elasticsearch from Storm, but I couldn't get any document to be indexed into Elasticsearch.
In my topology I have a KafkaSpout that emits a json like this { “tweetId”: 1, “text”: “hello” } to a EsBolt that is a native bolt from elasticsearch-hadoop library that writes the Storm Tuples to Elasticsearch (doc is here: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/storm.html).
These are the configs for my EsBolt:
Map conf = new HashMap();
conf.put("es.nodes","127.0.0.1");
conf.put("es.port","9200");
conf.put("es.resource","twitter/tweet");
conf.put("es.index.auto.create","no");
conf.put("es.input.json", "true");
conf.put("es.mapping.id", "tweetId");
EsBolt elasticsearchBolt = new EsBolt("twitter/tweet", conf);
The first two configurations have these values by default, but I chose to set them explicitly. I have also tried without them, getting the same result.
And this is how I build my topology:
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout(TWEETS_DATA_KAFKA_SPOUT_ID, kafkaSpout, kafkaSpoutParallelism)
.setNumTasks(kafkaSpoutNumberOfTasks);
builder.setBolt(ELASTICSEARCH_BOLT_ID, elasticsearchBolt, elasticsearchBoltParallelism)
.setNumTasks(elasticsearchBoltNumberOfTasks)
.shuffleGrouping(TWEETS_DATA_KAFKA_SPOUT_ID);
return builder.createTopology();
Before I run the topology locally I create the "twitter" index in Elasticsearch and a mapping "tweet" for this index.
This is what I get if I retrieve the mapping for my newly created type (curl -XGET 'http://localhost:9200/twitter/_mapping/tweet'):
{
"twitter": {
"mappings": {
"tweet": {
"properties": {
"text": {
"type": "string"
},
"tweetId": {
"type": "string"
}
}
}
}
}
}
I run the topology locally and this is what I get in my console when processing a tuple:
Processing received message FOR 6 TUPLE: source: tweets-data-kafka-spout:9, stream: default, id: {-8010897758788654352=-6240339405307942979}, [{"tweetId":"1","text":"hello"}]
Emitting: elasticsearch-bolt __ack_ack [-8010897758788654352 -6240339405307942979]
TRANSFERING tuple TASK: 2 TUPLE: source: elasticsearch-bolt:6, stream: __ack_ack, id: {}, [-8010897758788654352 -6240339405307942979]
BOLT ack TASK: 6 TIME: TUPLE: source: tweets-data-kafka-spout:9, stream: default, id: {-8010897758788654352=-6240339405307942979}, [{"tweetId":"1","text":"hello"}]
Execute done TUPLE source: tweets-data-kafka-spout:9, stream: default, id: {-8010897758788654352=-6240339405307942979}, [{"tweetId":"1","text":"hello"}] TASK: 6 DELTA:
So the tuples seems to be processed. However I don't have any document indexed in Elasticsearch.
I suppose I am doing something wrong when I set the configurations for EsBolt, maybe missing a configuration or something.
Documents will only be indexed once you reach the flush size, specified by es.storm.bolt.flush.entries.size
Alternately, you may set a TICK frequency that triggers a queue flush.
config.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 5);
By default, es-hadoop flushes on tick, as per the es.storm.bolt.tick.tuple.flush parameter.
I have also got the same issue, but when I looking for the es-Hadoop documents, I find because I was miss set the frequency that triggers a queue flush.Then I add a configurations to my store topology (es.storm.bolt.flush.entries.size ), it's fine.but when we setting the value for Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS .it's throw an exception :java.lang.RuntimeException:java.lang.NullPointerException in bolt execute function. then we use debug mode to test my topology, I find the input tuple in bolt execute don't contain any entries, but this empty tuple is been triggered.
That's what I feel confusion. Don't the tuple will be emitted according to the setting time, Even though this tuple is empty after we set Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS.i think which is a bug.
enter image description here
enter image description here
more information you can see:https://www.elastic.co/guide/en/elasticsearch/hadoop/current/storm.html

MapR installation failing for single node cluster

I was referring quick installation guide for single node cluster. For this i used 20GB storage file for MaprFS but while on installation , it is giving ' Unable to find disks: /maprfs/storagefile' .
Here is my configuration file.
# Each Node section can specify nodes in the following format
# Hostname: disk1, disk2, disk3
# Specifying disks is optional. If not provided, the installer will use the values of 'disks' from the Defaults section
[Control_Nodes]
maprlocal.td.td.com: /maprfs/storagefile
#control-node2.mydomain: /dev/disk3, /dev/disk9
#control-node3.mydomain: /dev/sdb, /dev/sdc, /dev/sdd
[Data_Nodes]
#data-node1.mydomain
#data-node2.mydomain: /dev/sdb, /dev/sdc, /dev/sdd
#data-node3.mydomain: /dev/sdd
#data-node4.mydomain: /dev/sdb, /dev/sdd
[Client_Nodes]
#client1.mydomain
#client2.mydomain
#client3.mydomain
[Options]
MapReduce1 = true
YARN = true
HBase = true
MapR-DB = true
ControlNodesAsDataNodes = true
WirelevelSecurity = false
LocalRepo = false
[Defaults]
ClusterName = my.cluster.com
User = mapr
Group = mapr
Password = mapr
UID = 2000
GID = 2000
Disks = /maprfs/storagefile
StripeWidth = 3
ForceFormat = false
CoreRepoURL = http://package.mapr.com/releases
EcoRepoURL = http://package.mapr.com/releases/ecosystem-4.x
Version = 4.0.2
MetricsDBHost =
MetricsDBUser =
MetricsDBPassword =
MetricsDBSchema =
Below is the error that i am getting.
2015-04-16 08:18:03,659 callbacks 42 [INFO]: Running task: [Verify Pre-Requisites]
2015-04-16 08:18:03,661 callbacks 87 [ERROR]: maprlocal.td.td.com: Unable to find disks: /maprfs/storagefile from /maprfs/storagefile remove disks: /dev/sda,/dev/sda1,/dev/sda2,/dev/sda3 and retry
2015-04-16 08:18:03,662 callbacks 91 [ERROR]: failed: [maprlocal.td.td.com] => {"failed": true}
2015-04-16 08:18:03,667 installrunner 199 [ERROR]: Host: maprlocal.td.td.com has 1 failures
2015-04-16 08:18:03,668 common 203 [ERROR]: Control Nodes have failures. Please fix the failures and re-run the installation. For more information refer to the installer log at /opt/mapr-installer/var/mapr-installer.log
Please help me here.
Thanks
Shashi
Error is resolved by adding skip-check new option after install
/opt/mapr-installer/bin/install --skip-checks new

Resources