I want to passthrough a file in logstash for debugging purposes. Is it possible somehow?
If i use following config to stdout stdin input, it works fine:
input { stdin { } }
output { stdout { } }
But when i want to do the same for file, it does not work:
input {
file {
path => ["/home/logstash/xunit.json"]
}
}
output {
stdout { }
}
I only see following warning, and nothing more:
Using milestone 2 input plugin 'file'. This plugin should be stable, but if you see strange behavior, please let us know! For more information on plugin milestones, see http://logstash.net/docs/1.4.2/plugin-milestones {:level=>:warn}
What i am doing wrong?
P.S. If using verbose mode, i see following lines, then everything hangs:
Registering file input {:path=>["/home/logstash/xunit.json"], :level=>:info}
Pipeline started {:level=>:info}
P.P.S. User has access to file.
Related
I have the following pipeline:
pipeline {
agent any
stages {
stage('CI') {
parallel {
stage('גמלאות') {
steps {
input message: "Pass Sanity?"
}
}
stage('וועדות') {
steps {
input message: "Pass Sanity?"
}
}
stage('מבוטח') {
steps {
input message: "Pass Sanity?"
}
}
}
}
}
}
I'm expecting to see in the UI 3 option to select in each stage. but getting only one instead.
If I will look at tradinall console then I will be able to select if to proceed or to abort.
Is there a way to get this function thru the UI?
Here is an example of the UI:
It seems that the code is correct. The problem is with the Jenkins plugin.
I mooved to work with Blueocean plugin, and now it seems to work fine:
everyone!
I have logstash config, which forwards logs from RabbitMQ to elasticSearch. Something like this:
input {
rabbitmq {
...
}
}
filter {
if [type] == "rabbitmq" {
json {
source => "message"
target => "message"
}
}
}
output {
elasticsearch {
hosts => ["${ES_HOST}"]
user => "${ES_USERNAME}"
password => "${ES_PASSWORD}"
sniffing => false
index => "kit_events-%{[message][elasticsearch][index]}"
}
}
And we were forced to compress logs on a fly, because they are spending too much traffic.
Logs were moved into array and gzipped.
What is the correct way of configuring un-gzipping and splitting array back into objects?
I did some research and found out that there is gzip_lines plugin and something on Ruby(?) to parse array, but I failed to implement it. Did anyone make something like this before?
UPD:
Added this filter
filter {
if [type] == "kitlog-rabbitmq" {
ruby {
init => "
require 'zlib'
require 'stringio'
"
code => "
body = event.get('[http][response][body]').to_s
sio = StringIO.new(body)
gz = Zlib::GzipReader.new(sio)
result = gz.read.to_s
event.set('[http][response][body]', result)
"
}
}
}
And now catching an error
ERROR][logstash.filters.ruby ] Ruby exception occurred: not in gzip format
[DEBUG][logstash.pipeline ] output received {"event"=>{"#timestamp"=>2018-11-30T09:16:19.127Z, "tags"=>["_rubyexception"], "#version"=>"1", "message"=>"x^\\x8B\\xAEV*\\xCE\\xCE\\xCC\\xC9)V\\xB2R\\x88V\\xD26T07\\xB7\\xB0\\xB4\\xB44000W\\x8A\\xD5QPJ\\xCE\\xCF+IL.\\u0001\\xCA*)\\u0001\\xB9\\xA9\\xB9\\x89\\x999 N\\x96C\\x96^r~.X,\\xA5\\u0014(R\\xADT\\x9A\\u000E6#\\xA0\\xB2$#?\\u000F\\xAC\\xB9\\u0000\\\"\\xE2\\u001C\\xAC\\u0014[\\v\\xE4\\xE6%概\\xF4z\\u0001\\xE9b%\\xA0\\xC8\\xC0\\xD9\\u001D\\v\\u0000\\u0003\\x9ADk", "type"=>"kitlog-rabbitmq"}}
Was trying different gzipping methods, but result is still the same. Also tried changing input codecs (plain - utf-8, plain - binary)
So the content in rabbitmq is gzipped?
In the best of all possible worlds, logstash would see the content-encoding header and unzip it for you, but the plugin doesn't seem to do anything with that knowledge. You might request the feature.
The plugin does let you access the header, so you could do the gzip yourself. Something like this:
filter {
if [#metadata][rabbitmq_properties][content-encoding] == "gzip" {
ruby {
...
}
}
}
Examples of unzipping a string with ruby exist elsewhere. Hopefully the 'zip' gem is available in logstash.
I am using Gradle-2.11 and I am unable to find a way to create log files that logs debug level information. I don't want to do it through command line by redirecting the logs to the log file. I want Gradle code just like Apache Ant's 'record' task so that I can put that code in my build.gradle file wherever I want to create logs.
For ex: If I want to convert this ant task to gradle, then what would be the code:
<record name="${BuildLogPath}/${BuildLogFile}" append="no" loglevel="verbose" action="start"/>
Gradle integrates really nicely with Ant (https://docs.gradle.org/2.11/userguide/ant.html)
It doesn't automatically record each step. I didn't realize that is what you were asking. The updated below will produce the output and you can manually log.
ant.record(name: "${BuildLogPath}/${BuildLogFile}", append:false, loglevel: "verbose", action: "start")
ant.echo("start logging")
//... do stuff here
ant.echo(message: "end logging")
ant.record(name: "${BuildLogPath}/${BuildLogFile}", append:false, loglevel: "verbose", action: "stop")
This may do more of what you are asking. Note: This is something I adapted slightly from this excellent example:
http://themrsion.blogspot.com/2013/10/gradle-logging-writing-to-log-to-file.html
import org.gradle.logging.internal.*
String currentDate = new Date().format('yyyy-MMM-dd_HH-mm-ss-S')
String loggingDirectory = "${rootDir}/build/logs"
mkdir("${loggingDirectory}")
File gradleBuildLog = new File("${loggingDirectory}/${currentDate}_gradleBuild.log")
gradle.services.get(LoggingOutputInternal).addStandardOutputListener (new StandardOutputListener () {
void onOutput(CharSequence output) {
gradleBuildLog << output
}
})
gradle.services.get(LoggingOutputInternal).addStandardErrorListener (new StandardOutputListener () {
void onOutput(CharSequence output) {
gradleBuildLog << output
}
})
I have the following in my logstash configuration file:
input {
file {
path => "C:/myfile.txt"
}
}
output {
exec {
command => 'mytest.bat %message% %path%'
interval => 0
}
}
the %message% and %path% parameters are being passed to the batch file.
I am expecting to see:
message contain the line of the input file currently being parsed
path contain C:/myfile.txt
However, this is what the batch file recieves:
message "%message%"
path "C:/logstash-1.5.0/vendor/bundle/jruby/1.9/bin"
What is correct way to define the placeholders for:
the current line to be output
the name of file being parsed
Thanks
Please modify your config to
input {
file {
path => "C:/myfile.txt"
}
}
output {
exec {
command => "mytest.bat %{message} %{path}"
interval => 0
}
}
If you want to get the field in the event, please use %{message} ,not %message%
I'm trying to get this Grunt plugin to work:
https://npmjs.org/package/dss
This documentation plugin ironically seems to be lacking proper documentation. Can someone help me out by giving me an example of something that worked for them. The main thing that's screwing me up is the "your_target" property. Not sure what's suppose to go in there.
Say I have a SASS file in the following path from the root directory:
sass/main.scss
What would my ouput be by default? And where would it output to? Also what format does it ouput to?
grunt.initConfig({
DSS: {
options: {
// Task-specific options go here.
},
your_target: {
// Target-specific file lists and/or options go here.
},
},
})
Is "your_target" property the path to my sass file or the path to the documentation file I'm trying to create? Would it be defined like this?
...
your_target: {
// Target-specific file lists and/or options go here.
sass: "sass/main.scss"
},
...
I don't know what the property name should be. :(
dss: {
docs: {
options: {
},
files: {
'api/': 'css/**/*.{css,scss,sass,less,styl}'
}
}
}