There is a file named "elasticsearch.yml". I have following questions about the file:-
Is it mandatory to name the file as elastic-search?
There is a property named cluster.name in the file, what is the use? If we don't mention any name, will it use any default name?
I am confused because i removed the name from the YML file but the program still worked.
Elasticsearch.yml is a configuration file. it contains various configurations related to cluster, node.
cluster.name is the property to set the name of your cluster. Default name of your cluster is elasticsearch. You can change it to any name you want.
If you remove cluster.name , it won't affect your program. Default name would be taken.
You can find answers in-line here:
Is it mandatory to name the file as elastic-search?
Yes, it is mandatory to not rename elasticsearch.yml. You can find more info here.
There is a property named cluster.name in the file, what is the use? If we don't mention any name, will it use any default name?
cluster.name is used to name your elasticsearch cluster. It has default value elasticsearch . But it is highly recommended to change this parameter as this is used to discover and auto-join other nodes. You can read more about this parameter here.
I am confused because i removed the name from the YML file but the program still worked.
Here there is little confusion that you changed cluster.name parameter value or completely removed that parameter from elasticsearch.yml . If you completely removed that then it still holds default value i.e elasticsearch or if you changed it's value then it depends on your program that how it identifying elasticsearch cluster.
Related
Looking at this documentation on adding fields, I see that filebeat can add any custom field by name and value that will be appended to every documented pushed to Elasticsearch by Filebeat.
This is defined in filebeat.yml:
processors:
- add_fields:
target: project
fields:
name: myproject
id: '574734885120952459'
Is there a way, strictly from filebeat.yml, to give fields added here a "type" as well? For example, can I assign "name" to type "keyword" ?
The only way I've seen to accomplish does not involve filebeat.yml, but creating a custom fields.yml file in the filebeat directory (this all applies for any beat) with the field and type specified under the "beat" key.
Example, if field "id" above was declared in filebeat.yml, and we wanted it to be a custom field of type "keyword," we would do the following:
copy fields.yml to my_filebeat_fields.yml file in the filebeat directory.
In my_filebeat_fields, add in this section:
- key: beat
anchor: beat-common
title: Beat
description: >
Contains common beat fields available in all event types.
fields:
# your customization begins here:
- name: id
- type: keyword
Then do the following to use this new custom fields file:
Modify this portion of filebeat.yml to include:
#==================== Elasticsearch template setting ==========================
setup.template.name: "filebeat-*"
setup.template.fields: "my_filebeat_fields.yml"
setup.template.overwrite: true
then load the template into elastic in whichever method is appropriate for your environment, following this guide.
(Assuming version 7.x for everything)
EDIT:
Apparently setting "setup.template.append_fields" option in the filebeat.yml file could also work, but I have not explored that.
https://www.elastic.co/guide/en/beats/filebeat/current/configuration-template.html
I was using netflix's nebula. Looking here, I saw this line:
fileType [org.freecompany.redline.payload.Directive] - Default for types, e.g. CONFIG, DOC, NOREPLACE, LICENSE
I didn't find any doc about the actual meaning of this enum, but I've found the original code.
Now I want an actual description of this enum. I thought NOREPLACE is releated to being not allowed to replace the file. But I want to be sure and don't rely on assumptions.
I have only seen noreplace as an additional attribute on a config file, e.g. %config(noreplace). It means that if the user has edited the file, the installer should put its new version as filename.rpmnew; by default %config files are replaced with the user one put as filename.rpmold .
I want to set ‘http.max_content_length’ at runtime. Is it possible and how one can do it? And if one can do it at runtime can he also change publishing_port/host?
No. You can't. As per elasticsearch documentation, http.max_content_length is not dynamically updatable. For more details refer this link..
Changes we do in YAML file will get reflected in node after once you restart the node.
I have created a new mapping in Informatica and I defined a mapping parameter $$LAST_EXTRACT_DATE (this is already defined in DAC(as it is working for out of box mappings)). However, When DAC is executing, this variable is not getting its values in parameter file of mapping. Other mappings are working fine. It's just this mapping which has an issue. I am trying to make this mapping to run in incremental.
Any suggestions please?
Yogesh,
I assumed that your mapping run with no issue and parameter $$LAST_EXTRACT_DATE are being replaced with a correct date through the DAC. You don't parameter file if you using DAC to run mapping. A parameter file is needed when you start the task from Informatica.
Make sure that the same file is used while running through DAC, as it's possible to run workflows in two ways:
just starting a workflow will make it read parameter file specified in the properties
starting workflow with indicated parameter file will make it read the given file INSTEAD OF the one specified in the properties.
According to https://www.consul.io/docs/agent/options.html#_node_meta
I can associate with a consul node any metadata key/value pair.
Can prometheus read this metadata ?
I understand that only the following meta labels are available for prometheus:
__meta_consul_address: the address of the target
__meta_consul_node: the node name defined for the target
__meta_consul_tags: the list of tags of the target joined by the tag separator
__meta_consul_service: the name of the service the target belongs to
__meta_consul_service_address: the service address of the target
__meta_consul_service_port: the service port of the target
__meta_consul_service_id: the service ID of the target
__meta_consul_dc: the datacenter name for the target
But I would like to be absolutely sure that I miss nothing or there is no a trick to do it.
Thank you
That's not supported as the feature was only released a month ago, but feel free to send a pull request.
Yes. This was introduced into Prometheus 1.8
You can now simply reference __meta_consul_metadata_$KEYNAME
The following shows a prometheus label rewrite which filters the nodes 'location' metadata to a ldn fact which we've added to Consul agents running in London.
- source_labels: [__meta_consul_metadata_location]
separator: ;
regex: ldn
replacement: $1
action: keep