i've been working on some prometheus templating and whenever i look at other examples of templating i keep encountering this command:
$jobs = label_values(job)
$instance = up{job=~"$jobs"}
i understand that $jobs is a variable being created, but i have next to no clue what the up command is doing. I've looked online and i can't really narrow down the search enough for a generic word like 'up' haha
my best guess is that it makes the $instance variable equal only to cases where job is similar to jobs? i'm really not sure
any help would clarify a bunch. thanks!
According to the Jobs and instances Prometheus documentation:
When Prometheus scrapes a target, it attaches some labels
automatically to the scraped time series which serve to identify the
scraped target:
job: The configured job name that the target belongs to.
instance: The <host>:<port> part of the target's URL that was scraped.
If either of these labels are already present in the scraped data, the
behavior depends on the honor_labels configuration option.
For each instance scrape, Prometheus stores a sample in the following
time series:
up{job="<job-name>", instance="<instance-id>"}: 1 if the instance is healthy, i.e. reachable, or 0 if the scrape failed.
The up time series is useful for instance availability monitoring.
Related
I'm a prometheus newbie and have been trying to figure out the right query to get the last continuous uptime for my service.
For example, if the present time is 0:01:20 my service was up at 0:00:00, went down at 0:01:01 and went up again at 0:01:10, I'd like to see the uptime of "10 seconds".
I'm mainly looking at the "up{}" metric and possibly combine it with the functions (changes(), rate(), etc.) but no luck so far. I don't see any other prometheus metric similar to "up" either.
The problem is that you need something which tells when your service was actually up vs. whether the node was up :)
We use the following (I hope one will help or the general idea of each):
1. When we look at a host we use node_time{...} - node_boot_time{...}
2. When we look at a specific process / container (docker via cadvisor in our case) we use node_time{...} - on(instance) group_right container_start_time_seconds{name=~"..."}) by(name,instance)
The following PromQL query must be used for calculating the application uptime in seconds:
time() - process_start_time_seconds
This query works for all the applications written in Go, which use either github.com/prometheus/client_golang or github.com/VictoriaMetrics/metrics client libraries, which expose the process_start_time_seconds metric by default. This metric contains unix timestamp for the application start time.
Kubernetes exposes the container_start_time_seconds metric for each started container by default. So the following query can be used for tracking uptimes for containers in Kubernetes:
time() - container_start_time_seconds{container!~"POD|"}
The container!~"POD|" filter is needed in order to filter aux time series:
Time series with container="POD" label reflect e.g. pause containers - see this answer for details.
Time series without container label correspond to e.g. cgroups hierarchy. See this answer for details.
If you need to calculate the overall per-target uptime over the given time range, then it is possible to estimate it with up metric. Prometheus automatically generates up metric per each scrape target. It sets it to 1 per each successful scrape and sets it to 0 otherwise. See these docs for details. So the following query can be used for estimating the total uptime in seconds per each scrape target during the last 24 hours:
avg_over_time(up[24h]) * (24*3600)
See avg_over_time docs for details.
I am new in niffi i am using getMongo to extract document from mongodb but same result is coming again and again but the result of query is only 2 document the query is {"qty":{$gt:10}}
There is a similar question regarding this. Let me quote what I had said there:
"GetMongo will continue to pull data from MongoDB based on the provided properties such as Query, Projection, Limit. It has no way of tracking the execution process, at least for now. What you can do, however, is changing the Run Schedule and/or Scheduling Strategy. You can find them by right clicking on the processor and clicking Configure. By default, Run Schedule will be 0 sec which means running continuously. Changing it to, say, 60 min will make the processor run every one hour. This will still read the same documents from MongoDB again every one hour but since you have mentioned that you just want to run it only once, I'm suggesting this approach."
The question can be found here.
I want to run only one pod of my kubernetes app at a time(relaunch in case of failure), I am using job controller.
But as per documentations, kubernetes may launch more than one pods and will eventually achieve specified replicas. Is there any way to achieve exactly one pod at a time or any recommended design pattern for such use cases.
My app is reading data from HDFS and writing it to a message queue. It exits after processing all the files. I want to minimize possibility of writing duplicate records.
I suggest you use replicasets for this. Set the number of replica to 1. More here https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/#when-to-use-a-replicaset
In principle, in the case of Jobs with no paralellism implied, there shouldn't be this kind of "race condition" ("should be 1" according to the documentation [1]). The job would be rescheduled only if an attempt fails. Did you come across the situation where 2 pods from the same job were being executed at the same time?
In any case, if you want to be completely sure, you may want to implement an extra coordination method or external solution.
[1] https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/
If I understand your question correctly I think you are looking for: .spec.strategy.rollingUpdate.maxSurge
If you set this to 0 then the existing pods will be killed before starting an new one.
I have just added the rufus scheduler gem to my application and ran it for a few minutes in development mode to find that it works.
But of course I'd like to write a spec that ensures the schedules are set up correctly. For example, typos could slip into the interval strings or some other gremlin might prevent.
My initial idea was to look at Scheduler#jobs but that can become tricky quite quickly: if there are, for example, two jobs with the same interval, I cannot see a straightforward way to identify the one to test.
Apart from that, it should be possible to set up some expectations, run the block and check whether the expected methods were called.
Do you have recommendations on how to test for correctness of job schedules at a given point in the application lifecycle?
You can place tags on jobs:
https://github.com/jmettraux/rufus-scheduler#tags
It helps identifying them. It's also useful to look them up:
https://github.com/jmettraux/rufus-scheduler#schedulerjobstag--tags--x
What is purpose to use setup threadgroup and teardown threagroup in Jmeter? Please explain same with example.
I know why we use thread groups and also aware of fact that setup is for pre activities like creating user and monitoring purpose but not sure with an incident where can i use it. same with tear down.
It sounds like you have pretty much figured it out already, but let me give you a few examples of when I've used it.
setup:
Get a large data set from a database into a jmeter variable for use during the test.
Get and log the version number from the system under test:s version number.
Run a javascript to set jmeter properties based on more simple input parameters/properties. Lets say you want to configure the selection of target host a simple true/false value, but in your test you need to expand it to different strings, and you dont want to have logic spread out all over your test plan.
teardown:
Never used it, but I guess it is mainly useful for cleaning up your system (e.g. deleting users that were created during the test)
correct me as i'm probably wrong, but a setUp thread cannot be used to store variables for use on the test threads (that i can see). any variables that i use in the setUp are never available. however, i found that if i use a beanshell and convert the variable in the setUp thread to a property like this
${__setProperty(userToken, ${userToken})};
then on each test thread i either use the property directly like:
${__property(userToken)}
or at the top of my thread i convert the property back into a variable like:
vars.put("userToken","${__property(userToken)}")
however, this seems a bit long winded and it would be great if there was a way to set up variables in the setUP to be used on every thread.
A special type of ThreadGroup that can be utilized to perform Pre-Test/ Post-Test Actions.The behavior of these threads is exactly like a normal Thread Group element.
The difference is that these type of threads execute before/after the test has finished executing its regular Thread Groups.enter image description here