I am getting 2014-06-23 02:23:14] log.INFO: General information log [] [] in my log.txt while calling $monolog->info('General information log'); I don't understand the need of [] [] hence I want to get rid of these square brackets from logged message. But I don't have any idea how...!
Well I did this via code hack
1st:
E:\xampp\htdocs\my_site\vendor\monolog\monolog\src\Monolog\Formatter\LineFormatter.php
2nd: Find this line
const SIMPLE_FORMAT = "[%datetime%] %channel%.%level_name%: %message% %context% %extra%\n";
3rd: and replace
const SIMPLE_FORMAT = "[%datetime%] %channel%.%level_name%: %message% \n";
[2014-06-23 13:23:33] production.INFO: yada ya da error
This is my log file
It will remove the square brackets I have tested.
Related
I am trying to send logs to DD (datadog) in such a way that the logs are being received as json and therefore shown properly in the portal through attributes.
My logger is a simple Logger.new(STDOUT, level: Logger::INFO).
If I stick to its standard output, it will in the form
I, [2022-07-30T22:43:35.216846 #1] INFO -- my-app: {"user":"1234"}
which is not really parsable by DD since not a proper JSON. In this case however all the logs appear at least on the DD portal.
Now.. I am trying to format the logs in a JSON manner in this way:
def self.logger
#logger ||= Logger.new(STDOUT, level: Logger::INFO)
#logger.progname = 'my-app'
#logger.formatter = proc do |severity, datetime, progname, msg|
{timestamp: datetime.to_s, progname: progname, severity: severity, correlation: Datadog::Tracing.log_correlation, message: msg}.to_json
end
#logger
end
This is my logger and thanks to this logs are seen properly in DD and parsed correctly because formatted in my app in a proper JSON.
The problem with this approach though seems to be that the logs are sent in 1 full block. Meaning that only the very first log is being visible. Let's say that I want to log this:
my_hash = {"message" => '1', "prop" => '1234'}.to_json
logger.info(my_hash)
my_hash = {"message" => '2', "prop" => '12345'}.to_json
logger.info(my_hash)
only the first log will be shown correctly on the DD portal. Parsed correctly with its message and prop attributes, but nothing about the second log.
Here is the thing, if I see the output of my app locally in the console I see this:
{"timestamp":"2022-07-31 01:15:39 +0200","progname":"my-app","severity":"INFO","correlation":"dd.service=my-app dd.trace_id=2976451780376429536 dd.span_id=0","message":"{"message":"1","prop":"1234"}"}{"timestamp":"2022-07-31 01:15:39 +0200","progname":"my-app","severity":"INFO","correlation":"dd.service=my-app dd.trace_id=2976451780376429536 dd.span_id=0","message":"{"message":"2","prop":"12345"}"}127.0.0.1 - - [31/Jul/2022:01:15:39 +0200] "GET /controller/test_controller HTTP/1.1" 200 - 0.0024
so the 2nd log gets actually outputted! But DD somehow sees only the first log..
(I know there is even a 3rd one shown in this message.. but that's just Sinatra automatic behavior for every http call reaching the api). What do you guys think is the problem?
Monolog version 2
Noticed when using in a Laravel project and trying to ingest logs into ELK.
Something like this:
Log::debug('Testing new lines in log context.', ['yep' => "This\nhas\nNew Lines."]);
Outputs
[2022-01-25 09:53:22] local.DEBUG: Testing new lines in log context. {"yep":"This
has
New Lines."}
Expecting
[2022-01-25 09:53:22] local.DEBUG: Testing new lines in log context. {"yep":"This\\nhas\\nNew Lines."}
Just wonder if there is a setting I can alter to make it escape the new line characters?
Note: The data containing new lines is perfectly fine, wanting to log this information out in its entirety, but the logging library does not output valid json?
https://github.com/Seldaek/monolog/issues/1629
Use single quotes and they will be escaped. '\n' doesn't output the newline.
Another option is to json_encode what you want to log if you do not want the newlines to be outputted.
php > print_r(['e' => "a\nb"]); <- double quotes
Array
(
[e] => a
b <- newline is outputted
)
php > print_r(['e' => 'a\nb']); <- single quotes
Array
(
[e] => a\nb <- newline is not outputted
)
php > print_r(json_encode(['e' => "a\nb"])); <- json_encode
{"e":"a\nb"} <- newline is not outputted
Log::debug('Testing new lines in log context.', ['yep' => 'This\nhas\nNew Lines.']);
Log::debug('Testing new lines in log context.', json_encode(['yep' => "This\nhas\nNew Lines."]));
The following JSON is a transaction what will be sent to the Ripple Network to query accounts that hold cryptographic assets at a Gateway (somewhat like a bank, more like a trust account between its clients). This script is to be used in conjunction with PHP to fetch a Gateway's issued balances and ignored it's hot-wallet or day-to-day operations wallet. My question is what is the proper way to:
a. Assign JSON within a Ruby variable?
b. What is the best way to escape double quotes and deal with newlines where brackets and square brackets occur within the JSON syntax?
The JSON follows:
ripple_path="/home/rippled/build/rippled"
conf = "--conf /etc/rippled/rippled.cfg"
puts "About to set the JSON lines "
gatewayStart = "\"method\": \"gateway_balances\","
paramsLine = "\"params\": [ {"
accountLine = "\"account\": \"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH\","
hotwalletLine = "\"hotwallet\": \"rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C\","
liLine = "\"ledger_index\": \"validated\","
strictLine = "\"strict\": "
trueLine = true
endLine = " } ] }"
balancesLine = "#{gatewayStart} #{paramsLine} #{accountLine} #>{hotwalletLine} #{liLine} #{strictLine} #{trueLine} #{endLine}"
lineString = "#{balancesLine.to_s}"
linetoJSON = "#{lineString}"
puts "linetoJSON: #{linetoJSON} "
cmd2=`#{ripple_path} #{conf} json gateway_balances #{linetoJSON}`
cmder="#{ripple_path} #{conf} json gateway_balances #{linetoJSON}"
puts "Done."
The output is:
root#xagate:WorkingDirectory# ruby gatewaybal.rb
About to set the JSON lines
linetoJSON: "method": "gateway_balances", "params": [ { "account":
"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH", "hotwallet": "rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C", "ledger_index": "validated", "strict":rue } ] }
Loading: "/etc/rippled/rippled.cfg"
rippled [options] <command> <params>
General Options:
-h [ --help ] Display this message.
.....
Done.
It is noteworthy that this command also returns a badSyntax error when executed manually via the command line. Please see here for the mirror of this issue raised on the ripple forums.
jsonLine = "'{ \"account\": \"rGgS5Hw3PhSp3VNT43PDTXze9YfdthHUH\", \"hotwallet\": \"rKYNhsT3aLymkGH7WL7ZUHkm6RE27iuM4C\", \"ledger_index\": \"validated\", \"strict\": true }'"
Is the proper way to assign this JSON within a single variable; this solution was provided by JoelKatz. The completed code is now available on GitHub.
I have a filter that looks like so:
multiline {
pattern => "(^.+Exception.*)|(^\tat .+)"
negate => false
what => "previous"
}
But for some reason, it's not attaching to the previous line for lines with ^\tat. Sometimes it does, but most of the time it doesn't. It attaches to the line way far back. I don't see anything wrong with my code.
Does anyone know if this is a bug?
Edit: This worked properly just now but couple minutes after it doesn't work again. Is it a buffer overflow? How would I debug this?
Edit: Example of success:
2014-06-20 09:09:07,989 http-bio-8080-exec-629 WARN com.rubiconproject.rfm.adserver.filter.impl.PriorityFilter - Request : NBA_DIV=Zedge_Tier1_App_MPBTAG_320x50_ROS_Android&NBA_APPID=4E51A330AD7A0131112022000A93D4E6&NBA_PUBID=111657&NBA_LOCATION_LAT=&NBA_LOCATION_LNG=&NBA_KV=device_id_sha-1_key=5040e46d15bd2f37b3ba58860cc94c1308c0ca4b&_v=2_0_0&id=84472439740784460, Response : Unable to Score Ads.. Selecting first one and Continuing...
java.lang.IndexOutOfBoundsException: Index: 8, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:604)
at java.util.ArrayList.get(ArrayList.java:382)
Edit: Example of failure:
2014-06-20 09:02:31,139 http-bio-8080-exec-579 WARN com.rubiconproject.rfm.adserver.web.AdRequestController - Request : car=vodafone UK&con=0&model=iPhone&bdl=com.racingpost.general&sup=adm,dfp,iAd&id=8226846&mak=Apple&sze=320x50&TYP=1&rtyp=json&app=F99D88D0FDEC01300BF5123139244773&clt=MBS_iOS_SDK_2.4.0&dpr=2.000000&apver=10.4&osver=7.1&udid=115FC62F-D4FF-44E0-8D92-5A060043EFDD&pub=111407&tud=3&osn=iPhone OS&, Response : No Ad Selected to Serve..Exiting
at java.util.ArrayList.get(ArrayList.java:382)
My file has 13000+ lines, and when it errors, it attaches to couple hundred lines back. But strangely each attaches to a line with the exact same offset in between (by offset I mean those couple hundred lines that it skips).
Your logs is java stack logs.
You can try to use this pattern. Use the date as the pattern, which is the beginning of each log.
input {
stdin{}
}
filter {
multiline {
pattern => "^(?>\d\d){1,2}-(?:0?[1-9]|1[0-2])-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])"
what => "previous"
}
}
output {
stdout {
codec => "rubydebug"
}
}
This pattern parses the date, if the line do not start with date, logstash will multiline it.
I have try it with your logs, it's worked on both two logs.
Hope this can help you.
I am aware of a number of posts on this topic - the most comprehensive of which is here.
My particular gremlin looks like this: (without further logging)
2012-04-17T00:28:50+00:00 DEBUG (7): HEADERS ALREADY SENT: <pre>[0] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Controller/Response/Http.php:51
[1] /domains/bodhi.uk.com/subdomains/dev/lib/Zend/Controller/Response/Abstract.php:766
[2] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Controller/Response/Http.php:89
[3] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Controller/Varien/Front.php:188
[4] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Model/App.php:349
[5] /domains/bodhi.uk.com/subdomains/dev/app/Mage.php:640
[6] /domains/bodhi.uk.com/subdomains/dev/index.php:80
This can occur loading the homepage on dev.bodhi.uk.com
Following the accepted answer in the linked post, creates a list of around 500 files, that are being included - not a practical method for debugging (especially when I may not recognise the issue when looking at it - I am currently assuming that I am looking for whitespace prior to a
For reference the output of the above is here on pastebin
Using the advice in the last comment (that appeared to work for the OP), and adding:
$headers = array();
foreach ($this->_headers as $i => $header) {
$headers[$header['name']] = $header;
}
$this->_headers = array_values($headers);
Mage::Log(array_values($headers));
Gives the unexpected responce:
2012-04-17T00:28:50+00:00 DEBUG (7): Array
(
[0] => Array
(
[name] => Content-Type
[value] => text/html; charset=UTF-8
[replace] =>
)
)
Ultimately I need to prevent this error, and I am struggling for a next step!
PS: I have also tried Alan's responce in comment 1, however the format of Abstract.php has changed (I am using version 1.6.2), and as I get no further output, I suspect I am not logging things correctly.
Thanks in advance!
Try ob_start() it may be helpful to you just put this code at the top of your file