I am running "docker.elastic.co/beats/filebeat:7.9.1"
In filebeat.yml I configured to append some fields to the template
setup.template.overwrite: true
setup.template.append_fields:
- name: level
type: text
- name: msg
type: text
- name: stacktrace
type: text
- name: uri
type: text
- name: headers
type: text
This works fine. But as soon as I add the field:
- name: hash
type: keyword
I get the following error:
INFO template/load.go:169 Existing template will be overwritten, as overwrite is enabled.
ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(https://whatever.eu-central-1.aws.cloud.es.io:443)): Connection marked as failed because the onConnect call back failed: error loading template: error creating template: 1 error: fields contain key <hash>
The Elasticsearch version is v7.10.0.
What is going on?
The problem is that a field named "hash" is already defined in the "Elastic Common Schema (ECS)" fields in the default "fields.yml" file.
(see: https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-ecs.html)
Related
The following Parameter block is throwing me Remove prohibited property Warning and in turn the ci make validate command is throwing an error: "Template format error: YAML not well-formed."
Parameters:
Configuration:
Description: The environment/configuration name
Type: String
AccountName:
Description: AWS account name
Type: String
ServiceName:
Description: Service name
Type: String
Default: kada
S3BucketName:
Description: S3 bucket name
Type: String
user:
Description: Username
Type: String
client_id:
Description: Client id
Type: String
token_url:
Description: URL to fetch token
Type: String
post_url:
Description: URL where json is posted
Type: String
domain:
Description: KADA Domain
Type: String
post_token_url:
Description: url to fetch token
Type: String
The error is from the client id Parameter and continued for all other parameters saying remove prohibited property. I am new to cloud formation. Please help.
Got the fix! When declaring un Yaml, we cannot use _ or white space in the variables, hence I used camelcase and got it fixed.
Since there isn't currently a native databricks powerapps connector, I set up a custom connector that calls the jobs runnow api using an AD backed service principle bearer token. I then shared the custom connector and the connection with users for them to run. While this is working fine for everyone on my team, the actual end users are getting a 401 error response. Since everyone on my team are admins in Azure and Databricks, it's likely we have permissions that the end users don't. However, we're having a hard time pinpointing where that might be.
During PowerApps monitoring sessions, we did notice that the Request URL is different for the users getting successful 200 responses, versus those getting 401. The odd thing is, neither matches the specified host in the custom connector. Can anyone tell me how this is generated?
Here is the custom connector code we're using:
swagger: '2.0'
info: {title: Databricks Jobs, description: Call Databricks Jobs API,
version: '1.0'}
host: <databricksInstance>.azuredatabricks.net
basePath: /api/2.1/jobs/
schemes: [https]
consumes: []
produces: []
paths:
/run-now:
post:
responses:
default:
description: default
schema:
type: object
properties:
run_id: {type: integer, format: int32, description: run_id}
number_in_job: {type: integer, format: int32, description: number_in_job}
summary: Start Databricks Notebook as a Job
description: Start Databricks Notebook as a Job
operationId: run-now
x-ms-visibility: important
parameters:
- {name: Content-Type, in: header, required: true, type: string, default: application/json,
x-ms-visibility: internal}
- {name: Accept, in: header, required: true, type: string, default: application/json,
x-ms-visibility: internal}
- name: body
in: body
required: true
schema:
type: object
properties:
job_id: {type: integer, format: int64, description: job_id, title: ''}
required: [job_id]
definitions: {}
parameters: {}
responses: {}
securityDefinitions:
API Key: {type: apiKey, in: header, name: Authorization}
security:
- API Key: []
tags: []
Here is the actual request as seen in PowerApps monitor that returns a successful 200 code (same url for everyone getting a success):
And here is the actual request sent that returns the 401 error:
As you can see, the request url is the same for both except for that last highlighted string. Can anyone tell me how that is generated from the specified host in the API call definition? Any idea what that may signify?
I'm getting stuck on this issue. Essentially I want to output the list of 'Namespace' key/values like below:
MyCloudWatchMetricStream:
Type: AWS::CloudWatch::MetricStream
Properties:
IncludeFilters:
- Namespace: AWS/ELB
- Namespace: AWS/Events
- Namespace: AWS/Logs
When it's declared like this:
MyCWMIncludeFilters:
Type: String
Default: 'Namespace: AWS/ELB,Namespace: AWS/Events,Namespace: AWS/Logs'
I've tried this to no avail:
MyCloudWatchMetricStream:
Type: AWS::CloudWatch::MetricStream
Properties:
IncludeFilters: !Split [",", !Ref MyCWMIncludeFilters]
and getting this error on stack creation:
Embedded stack arn:aws:cloudformation:<redacted> was not successfully created:
The following resource(s) failed to create: [MyCloudWatchMetricStream].
Any ideas of what I can do? Alternatives? Best practices?
Thanks in advance!
I am trying to define my APIs using openapi version 3.0.0. I've generated following YAML file:
openapi: 3.0.0
info:
title: My YAML
description: My YAML
version: 1.0.0
servers:
- url: http://my.server.url
paths:
/v1/service/uri:
post:
summary: Getting some information.
parameters:
- name: Content-Type
in: header
description: Content type of request body.
required: true
style: simple
explode: false
schema:
type: string
- name: Host
in: header
description: Originate host which returns the response.
required: false
style: simple
explode: false
schema:
type: string
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/MyPostRequest'
example:
my_name: "zizi"
my_age: 29
required: true
responses:
200:
description: OK
content:
application/json:
schema:
$ref: '#/components/schemas/MyPostResponse'
components:
schemas:
MyPostRequest:
type: object
properties:
my_name:
type: string
my_age:
type: integer
format: int32
MyPostResponse:
type: object
properties:
status:
type: integer
format: int32
When I copy/paste these lines into Swagger Editor, it gives me duplicated mapping key error on line 19; it is for section description of parameter Content-Type.
I have studied openapi documentation, but I didn't see anything wrong with my YAML file.
I am not sure why you get that error, I tried to find out in which language Swagger is written, as there are several YAML parsers out there that are known to have problems, but couldn't easily determine that using google or wikipedia.
You don't have any duplicate keys in your file, but it is invalid (i.e. not valid YAML) because of the indentation of the key content (the second occurrence, the one under paths →
/v1/service/uri → post → responses → 200), that should be aligned with
description as that key cannot have a value node that is both a scalar (OK) as well as a mapping node content: ....
Cannot create a google deployment manager runtime config variable
resources:
- name: star-config
type: runtimeconfig.v1beta1.config
properties:
name: star-config
- name: igurl_variable
type: runtimeconfig.v1beta1.variable
properties:
name: igurl_variable
value: 'trek'
parent: $(ref.star-config.name)
I checked the logs and I see that the status is set to bad_request when I create the above deployment.
Audit log
status: {
message: "BAD_REQUEST"
}
What could be the reason for the error ?
You should try the with the properties fields as in the official documentation for both the config and variable resources.
The resource file should be something like:
resources:
- name: star-config
type: runtimeconfig.v1beta1.config
properties:
config: star-config
- name: igurl_variable
type: runtimeconfig.v1beta1.variable
properties:
variable: igurl_variable
text: 'trek'
parent: $(ref.star-config.name)