Is there a way to Relabel a detected field value contained that is part of JSON with Loki? - grafana-loki

I haven't been able to find this in the Loki documentation.
Currently, my logs contain a Label resource that has information about the running service. Ideally, I would like to extract some of these values to make the logs very easy to filter by.
This is what the label value looks like:
{"labels":{"configuration_name":"myservicename","location":"region","service_name":"myservicename},"type":""}
I'd like to be able to remap the service_name. I'm using the Helm chart, and setting the values for scrapeConfigs.relabel_configs - I have this working for some basic remaps, however, I haven't been able to remap the JSON values, or even confirm whether that is possible.
Any help on this would be greatly appreciated!

This is for the Node.JS integration of Granfana
To filter an array, you can use array.filter.
For example, if you have an array like so
var names = [
{ "name":"program" },
{ "name":"application"}
]
You can filter my the name like so
var filtered = name.filter(n => n.name == "program")
// filtered = [{"name":"program"}]
Here is a full example:
var logs = [
{
"labels": {
"configuration_name":"serviceA",
"location":"region",
"service_name":"serviceA"
},
"type":""
},
{
"labels": {
"configuration_name":"programB",
"location":"region",
"service_name":"programB"
},
"type":""
},
{
"labels": {
"configuration_name":"systemC",
"location":"region",
"service_name":"systemC"
},
"type":""
}
]
var filteredLogs = logs.filter(l => l.labels.configuration_name == "systemC")
console.log(filteredLogs[0])

Related

Match keys with sibling object JSONATA

I have an JSON object with the structure below. When looping over key_two I want to create a new object that I will return. The returned object should contain a title with the value from key_one's name where the id of key_one matches the current looped over node from key_two.
Both objects contain other keys that also will be included but the first step I can't figure out is how to grab data from a sibling object while looping and match it to the current value.
{
"key_one": [
{
"name": "some_cool_title",
"id": "value_one",
...
}
],
"key_two": [
{
"node": "value_one",
...
}
],
}
This is a good example of a 'join' operation (in SQL terms). JSONata supports this in a path expression. See https://docs.jsonata.org/path-operators#-context-variable-binding
So in your example, you could write:
key_one#$k1.key_two[node = $k1.id].{
"title": $k1.name
}
You can then add extra fields into the resulting object by referencing items from either of the original objects. E.g.:
key_one#$k1.key_two[node = $k1.id].{
"title": $k1.name,
"other_one": $k1.other_data,
"other_two": other_data
}
See https://try.jsonata.org/--2aRZvSL
I seem to have found a solution for this.
[key_two].$filter($$.key_one, function($v, $k){
$v.id = node
}).{"title": name ? name : id}
Gives:
[
{
"title": "value_one"
},
{
"title": "value_two"
},
{
"title": "value_three"
}
]
Leaving this here if someone have a similar issue in the future.

Is there any way to query products using dynamic filters in Magento 2.3 graphql endpoint?

I was looking at the new graphql endpoint exposed by Magento 2.3.1 and above at the query for listing products:
query products(
pageSize:6,
currentPage:1,
filter:{
category_id: { eq: "3" }
}
) {
filters {
name
request_var
filter_items_count
filter_items {
label value_string
}
}
items {
id
name
small_image {
url
}
# ...
}
# ...
}
The response body yields products in the items property just as expected and a bunch of custom filters in an array in the filters property which look like this:
"filters": [
{
"name": "Activity",
"request_var": "activity",
"filter_items_count": 12,
"filter_items": [
{
"label": "Outdoor",
"value_string": "5"
},
{
"label": "Yoga",
"value_string": "8"
},
{
"label": "Recreation",
"value_string": "9"
},
// rest of filter values
]
},
// rest of filters
]
Given the fact that those filters are dynamic and user defined is there a way of sending them back with a list products query in graphql? I would expect to have a property somewhere under the products query that could be an array of aforementioned filter objects but so far I haven't found anything neither in the schema nor in the official documentation.
Did anyone have any similar experience with this?
No. Theres no way of getting the filtered options sent back. There are options where you can return the additional filtering abilities (similar to when you are viewing a category page) to return list of ways to filter but nothing that returns the current active filters in place.
{
products(filter: {sku: {like: "%"} } pageSize: 500) {
filters{
request_var
name
filter_items{
label
value_string
}
}
items {
id
sku
name
...
}
}
}

GraphQL: Explore API without a wildcard (*)?

I am new to GraphQL and I wonder how I can explore an API without a possible wildcard (*) (https://github.com/graphql/graphql-spec/issues/127).
I am currently setting up a headless Craft CMS with GraphQL and I don't really know how my data is nested.
Event with the REST API I have no chance of just getting all the data, because I have to setup all the endpoints and therefore I have to know all field names as well.
So how could I easily explore my CraftCMS data structure?
Thanks for any hints on this.
Cheers
merc
------ Edit -------
If I use #simonpedro s suggestion:
{
__schema {
types {
name
kind
fields {
name
}
}
}
}
I can see a lot of types (?)/fields (?)...
For example I see:
{
"name": "FlexibleContentTeaser",
"kind": "OBJECT",
"fields": [
{
"name": "id"
},
{
"name": "enabled"
},
{
"name": "teaserTitle"
},
{
"name": "text"
},
{
"name": "teaserLink"
},
{
"name": "teaserLinkConnection"
}
]
But now I would like to know how a teaserLink ist structured.
I somehow found out that the teaserLink (it is a field with the type Entries, where I can link to another page) has the properties url & title.
But how would I set up query to explore the properties available within teaserLink?
I tried all sorts of queries, but I am always confrontend with messages like this:
I would be really glad if somebody could give me another pointer how I can find out which properties I can actually query...
Thank you
As far as I'm concerned currently there is no graphql implementation with that capability. However, if what you want to do is to explore the "data structure", i.e, the schema, you should use schema instrospection, which was thought for that (explore the graphql schema). For example, a simple graphql instrospection query would be like this:
{
__schema {
types {
name
kind
fields {
name
}
}
}
}
References:
- https://graphql.org/learn/introspection/
UPDATE for edit:
What you want to do I think is the following:
Make a query like this
{
__schema {
types {
name
kind
fields {
name
type {
fields {
name
}
}
}
}
}
}
And then find the wished type field to grab more information (the fields) from it. Something like this (I don't know if this works, just an idea):
const typeFlexibleContentTeaser = data.__schema.types.find(t => t === "FlexibleContentTeaser")
const teaserLinkField = typeFlexibleContentTeaser.fields.find(f => f.name === "teaserLink")
const teaserLinkField = teaserLinkField.type.fields;
i.e, you have to transverse recursively through the type field.

Couchbase full-text search and compound keys

I have the following data in Couchbase:
Document 06001:
{
"type": "box",
"name": "lxpag",
"number": "06001",
"materials": [
{
"type": "material",
"number": "070006",
"name": "hosepipe"
},
{
"type": "material",
"number": "080006",
"name": "Philips screw 4mm"
},
}
Document 12345:
{
"type": "material",
"number": "12345",
"name": "Another screw"
}
Now I want to be able to query by type and name or number: for a given query type only the documents with the respective type property shall be returned. Furthermore, a second query string specifies which kinds of materials should be searched for. If a material's id or name contains (not starts with) the search term, it shall be included. If one of the materials inside a box matches the term accordingly, the whole box shall be included.
What I have come up with is:
function (doc, meta) {
if (doc.type === 'box' && Array.isArray(doc.materials)) {
var queryString = "";
for (i = 0; i < doc.materials.length; ++i) {
var material = doc.materials[i];
if (material.name && material.number) {
queryString += " " + material.name + " " + material.number;
}
}
emit([doc.type, queryString], doc);
} else if (doc.type === 'material') {
var queryString = doc.name + " " + doc.number;
emit([doc.type, queryString], doc);
}
}
I see that this view might not be fit for substring searches (Do I need ElasticSearch for this?). Nevertheless, when I use the following query parameters:
startKey=["box","pag"]&endKey=["box\u02ad","pag\u02ad"]
...not only do I get the box but also all other documents that are returned by the view. Thus, with these keys, nothing is filtered. On the other hand, searching by key works.
How is this possible?
There is no good way of doing substring search with view keys. Your options are either integrating with ElasticSearch, or using N1QL, which lets you do wildcard string matches: "SELECT * FROM bucket WHERE type = 'material' and name LIKE '%screw%'"
I just saw the flaw in the queries: the parameters must be written in lowercase, otherwise they are not recognized by Couchbase and ignored (it would be really helpful if I got an error message here instead of the usual result list...). So instead, I have to query with
startKey=["box","pag"]&endKey=["box\u02ad","pag\u02ad"]
What I have not precisely found out so far is how to manage the substring search. Since pag is a substring of lxpag above query would not return any results. Any ideas no this matter?

How can I sort GeoJson file showing defined tags?

Good morning to all and thank you for your help.
I'm working in a map page (map.html) create by leaflet library that take data from a external geojson file called water_well.js. This file, previously generated by overpass service is just a list of markers. every Marker have some proprerties. Follow an exemple:
"properties": {
"operator:type": "international",
"is_in:district": "west_mamprusi",
"is_in:region": "northern",
"source:date": "2012-02-11",
"source:ele": "gps",
"water_wells:source_type": "borehole"
},
The main page extract those data from the file before with this javascript:
var wwMarker = L.geoJson(water_well, {
pointToLayer : function (feature, latlng) {
lat = feature.geometry.coordinates[0];
lng = feature.geometry.coordinates[1];
//following code that make error
op_type = feature.properties.operator_type;
district = feature.properties.is_in:district;
region = feature.properties.is_in:region;
source_date = feature.properties.source:date;
source_ele = feature.properties.source:ele;
source_type = feature.properties.water_wells:source_type;
.....
I'm sure the problem is my Zero javascript knowledge, but I'm not a programmer and I do this map for my NGO engaged in water wells in Burkina Faso.
The script for extraction of the data don't work in this point:
op_type = feature.properties.operator:type;
The problem is ":" because is invalid character.
The second question is that not all markers in the first file called water_well.js have the same "properties" filled ad actually it is possible that someone have different group of "properties like those two:
{
"type": "Feature",
"id": "node/1606958159",
"properties": {
"#id": "node/1606958159",
"amenity": "drinking_water",
"man_made": "water_well",
"name": "puits 4"
},
"geometry": {
"type": "Point",
"coordinates": [
-3.6235696,
12.02171
]
}
},
{
"type": "Feature",
"id": "node/1913126817",
"properties": {
"#id": "node/1913126817",
"ele": "170.8000030517578",
"grid_proximity": "grid_further_500_m",
"is_in:district": "builsa",
"is_in:region": "upper_east",
"man_made": "water_well",
"operational_status": "open",
"operator:type": "individual",
"pipe_connection": "no",
"pump": "manual",
"seasonal": "another_pattern",
"source": "MVP,Columbia University",
"source:date": "2012-02-14",
"source:ele": "gps",
"water_wells:source_type": "unprotected_well"
},
"geometry": {
"type": "Point",
"coordinates": [
-1.2430456,
10.3233693
]
}
},
maybe it is possible to extract all properties of each item independently from which one is present or not..... This can be de better way to solve the problem but I've no idea how to do that.
This is what I do (ckick the water tap to see pop-up): www.h2openmap.org/map
This is almost what I would like to do (ckick the water tap to see pop-up): overpass-turbo.eu/s/7Ov
Thank you for spending your time reading my question.
Have a nice day everyone, Francesco
You can access those properties using the bracketnotation, instead of using:
district = feature.properties.is_in:district;
Use bracketnotation:
district = feature.properties['is_in:district'];
Reference on property-accessors: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Property_Accessors
Now if you want to do something based on if a property exists there is a hasOwnProperty method available on objects. Since feature.properties is an object you can use that in a condition:
if (features.properties.hasOwnProperty('is_in:district')) {
// Property exists, do stuff
}
// or
if (!features.properties.hasOwnProperty('is_in:district')) {
// Property does not exist, do stuff
}
If you want to do something base on wether multiple properties exist you can use the && (and) operator:
if (features.properties.hasOwnProperty('is_in:district') &&
features.properties.hasOwnProperty('source:data')) {
// Properties exist, do stuff
}
// Or
if (!features.properties.hasOwnProperty('is_in:district') &&
!features.properties.hasOwnProperty('source:data')) {
// Properties do not exist, do stuff
}
You could use the || (or) operator to see if at least one of the conditions matches:
if (features.properties.hasOwnProperty('is_in:district') ||
features.properties.hasOwnProperty('source:data')) {
// At least one of the properties exist, do stuff
}
// Or
if (!features.properties.hasOwnProperty('is_in:district') ||
!features.properties.hasOwnProperty('source:data')) {
// At least one of the properties does not exist, do stuff
}
Reference for this can be found here under "Logical operators": https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Expressions_and_Operators
You can use something like to build (or don't build) the data object that you need for your popup. Hope that helps.

Resources