Minikube - Not able to get any result from elastic search to if it uses existing indices - elasticsearch

Question

I am trying to load my existing local elastic search indices into kubernetes (version - minikube v1.9.2) elastic search pod.
What I finally understood is I have to use mountpath and hostpath combination to do that. Additionlay if I want to provide a custom index file (not the default one), then I have to use a configMap to override path.data of config/elasticsearch.yml
I did those as below and it created a directory in mount path and update config/elasticsearch.yml file but a mount path directory does not contain the host path directory’s content.
I could not figure out the reason behind it. Could some one let me know what am I doing wrong here?
Then went I head and manually copied indexes from local host to kubernetes pod using
kubectl cp localelasticsearhindexdirectory podname:/data/elk/
But then I tried do a elastic search and it gives me a empty result ( even though index manually copied).
If I use the same index with a local elastic search ( not on kubernetes) then I can get the result.
Could someone please give some advice to diagnose following issues
Why mount path does not have the hostpjths content
How to debug / What steps should I follow understand why it’s not able to get the result with the elasticsearch on pod?
kind: Deployment
metadata:
name: elasticsearch
spec:
selector:
matchLabels:
run: elasticsearch
replicas: 1
template:
metadata:
labels:
run: elasticsearch
spec:
containers:
- image: docker.elastic.co/elasticsearch/elasticsearch:6.6.1
name: elasticsearch
imagePullPolicy: IfNotPresent
env:
- name: discovery.type
value: single-node
- name: cluster.name
value: elasticsearch
ports:
- containerPort: 9300
name: nodes
- containerPort: 9200
name: client
volumeMounts:
- name: storage
mountPath: /data/elk
- name: config-volume
mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
subPath: elasticsearch.yml
volumes:
- name: config-volume
configMap:
name: elasticsearch-config
- name: storage
hostPath:
path: ~/elasticsearch-6.6.1/data
---
apiVersion: v1
kind: ConfigMap
metadata:
name: elasticsearch-config
data:
elasticsearch.yml: |
cluster:
name: ${CLUSTER_NAME:elasticsearch-default}
node:
master: ${NODE_MASTER:true}
data: ${NODE_DATA:true}
name: ${NODE_NAME:node-1}
ingest: ${NODE_INGEST:true}
max_local_storage_nodes: ${MAX_LOCAL_STORAGE_NODES:1}
processors: ${PROCESSORS:1}
network.host: ${NETWORK_HOST:_site_}
path:
data: ${DATA_PATH:"/data/elk"}
repo: ${REPO_LOCATIONS:[]}
bootstrap:
memory_lock: ${MEMORY_LOCK:false}
http:
enabled: ${HTTP_ENABLE:true}
compression: true
cors:
enabled: true
allow-origin: "*"
discovery:
zen:
ping.unicast.hosts: ${DISCOVERY_SERVICE:elasticsearch-discovery}
minimum_master_nodes: ${NUMBER_OF_MASTERS:1}
xpack:
license.self_generated.type: basic ```
**service.yaml**
```apiVersion: v1
kind: Service
metadata:
name: elasticsearch
labels:
service: elasticsearch
spec:
ports:
- name: client
port: 9200
protocol: TCP
targetPort: 9200
- name: nodes
port: 9300
protocol: TCP
targetPort: 9300
type: NodePort
selector:
run: elasticsearch```

Answer 1

Solution in HostPath with minikube - Kubernetes worked for me.
To mount a local directory into a pod in minikube (version - v1.9.2), you have to mount that local directory into minikube then use minikube mounted path in hostpath
(https://minikube.sigs.k8s.io/docs/handbook/mount/).
minikube mount ~/esData:/indexdata
📁 Mounting host path /esData into VM as /indexdata ...
▪ Mount type: <no value>
▪ User ID: docker
▪ Group ID: docker
▪ Version: 9p2000.L
▪ Message Size: 262144
▪ Permissions: 755 (-rwxr-xr-x)
▪ Options: map[]
▪ Bind Address: 192.168.5.6:55230
🚀 Userspace file server: ufs starting
✅ Successfully mounted ~/esData to /indexdata
📌 NOTE: This process must stay alive for the mount to be accessible ...
You have to run minikube mount in a separate terminal because it starts a process and stays there until you unmount.
Instead of doing it as Deployment as in the original question, now I am doing it as Statefulset but the same solution will work for Deployment also.
Another issue which I faced during mounting was elastic search server pod was throwing java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes . Then I saw here that I have to use initContainers to set full permission in /usr/share/elasticsearch/data/nodes.
Please see my final yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: elasticsearch
spec:
serviceName: "elasticsearch"
replicas: 1
selector:
matchLabels:
app: elasticsearch
template:
metadata:
labels:
app: elasticsearch
spec:
initContainers:
- name: set-permissions
image: registry.hub.docker.com/library/busybox:latest
command: ['sh', '-c', 'mkdir -p /usr/share/elasticsearch/data && chown 1000:1000 /usr/share/elasticsearch/data' ]
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
containers:
- name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:6.6.1
env:
- name: discovery.type
value: single-node
ports:
- containerPort: 9200
name: client
- containerPort: 9300
name: nodes
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
volumes:
- name: data
hostPath:
path: /indexdata
---
apiVersion: v1
kind: Service
metadata:
name: elasticsearch
labels:
service: elasticsearch
spec:
ports:
- port: 9200
name: client
- port: 9300
name: nodes
type: NodePort
selector:
app: elasticsearch