Why my Yarn shows less memory (12 GB) than host machine (32) GB - hadoop

Question

I am using EMR and I have task nodes with 32 GB of memory. However when I login to my YARN UI. it says it has only 12 GB of memoery.
Yes, I understand some memory should be used by OS and other services running. However, 20 GB is too much.
On host machine
free -h
total used free shared buffers cached
Mem: 30G 18G 12G 88K 128M 12G
-/+ buffers/cache: 5.6G 25G
Swap: 0B 0B 0B
on other machine.
free -h
total used free shared buffers cached
Mem: 30G 11G 18G 88K 75M 8.5G
-/+ buffers/cache: 3.4G 27G
Swap: 0B 0B 0B
so even after having 18 gb free, why Yarn shows only 12 GB available?

Answer 1

After doing some search on google that it was being restricted by yarn setting. Same thing has been suggested by pradeep in his comment.
I did the following thing the setting on the cluster
Find out the instance group ID for your task and core nodes. Use following command for that.
aws emr describe-cluster --cluster-id j-xxxxxx | jq -r '.Cluster.InstanceGroups[] | select(.InstanceGroupType == "CORE").Id'
aws emr describe-cluster --cluster-id j-xxxxxx | jq -r '.Cluster.InstanceGroups[] | select(.InstanceGroupType == "TASK").Id'
Create JSON configuration to update your emr. You will have to create two configuration file one for TASK group and another for Core group. Or you can simple replace value of InstanceGroupId after each update.
[
{
"InstanceGroupId":"<output_of_above_command>",
"Configurations":[
{
"Classification":"yarn-site",
"Properties":{
"yarn.nodemanager.resource.memory-mb":"32768"
},
"Configurations":[]
}
]
}
]
Finally run the command to update the instance group.
aws emr modify-instance-groups --cluster-id j-******* --instance-groups file://instaceGroup.json