Is there a way to disable the database logging of laravel when a job failed?
For example I'm just trying to write a log message, which would enough for this specific job:
job.php
public function handle()
{
//making an API request to an external API, storing some data inside cache
}
public function failed(Throwable $exception)
{
Log::info("external API update failed");
}
I already tried to edit the config/queue.php file by:
'failed' => [
'driver' => env('QUEUE_FAILED_DRIVER', 'database-uuids'),
'database' => null,
'table' => null,
],
This doesn't work, any idea how to get the database logging of failed jobs disabled?
I believe you have to set config/queue.php to the following:
'failed' => [
'driver' => null
'database' => null,
'table' => null,
],
You should be able to find clues within the laravel/framework folder within the vendor folder in your project, e.g.
vendor/laravel/framework/src/Illuminate/Queue/DatabaseFailedJobProvider.php.
The saving to database is done in the DatabaseFailedJobProvider class, the log function specifically.
Based on the QueueServiceProvider's registerFailedJobServices function, you would need to set the driver to null or the 'null' string in order to get it to run the NullFailedJobProvider, where the log function is empty.
Related
I'm using Illuminate\Queue outside of a Laravel app inside an add-on for a CMS. So the only instances of Laravel or Illuminate are these packages that I've required:
"illuminate/queue": "^8.83",
"illuminate/bus": "^8.83",
"illuminate/contracts": "^8.83"
I'm first trying to use the Database for the queue as the default driver since the CMS is database driven, then provide options to SQS etc. I've setup everything so the migrations create my queue tables and everything seems to be wired together when I make the following call to push something to the queue.
/** #var \Illuminate\Queue\QueueManager $queue */
$queue->push('test', ['foo' => 'bar']);
Then it ends in the following error. The parameter bindings are not working or something. It's leaving the ? in the values list.
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"exp_dgq_jobs" ("queue", "attempts", "reserved_at", "available_at", "created_at"' at line 1 (SQL: insert into "exp_dgq_jobs" ("queue", "attempts", "reserved_at", "available_at", "created_at", "payload") values (default, 0, ?, 1674567590, 1674567590, {"uuid":"6bf7a17e-dda3-4fed-903a-8714e5a2d146","displayName":"test","job":"test","maxTries":null,"maxExceptions":null,"failOnTimeout":false,"backoff":null,"timeout":null,"data":{"foo":"bar"}}))
I've step debugged the whole request and it feels like a bug, but then again this is really the first time I've used Laravel or one of it's packages, so maybe I'm missing something? This function explicitly sets reserved_at to null, and the Connection->prepareBindings() method doesn't do anything with the ?, it just leaves it as that value, so the query fails.
protected function buildDatabaseRecord($queue, $payload, $availableAt, $attempts = 0)
{
return ['queue' => $queue, 'attempts' => $attempts, 'reserved_at' => null, 'available_at' => $availableAt, 'created_at' => $this->currentTime(), 'payload' => $payload];
}
What am I missing? Everything just looks right to me an I'm kind of at a loss. I'm making this with PHP 7.4 in mind (for the time being). Maybe I'll try 8.1 to see if that changes anything with the Illuminate packages. Using MySQL 8 too.
Update: potentially relevant screenshot just before the error.
Update 2: I tried PHP 8.1 and latest Laravel 9 packages, didn't make a difference.
For more clarity on how I"m creating my QueueManager:
<?php $queue = new Queue;
$queue->addConnection([
'driver' => 'database',
'table' => ee('db')->dbprefix . 'dgq_jobs',
'queue' => 'default',
'retry_after' => 90,
'after_commit' => false,
]);
$databaseConfig = $provider->make('DatabaseConfig');
$queue->addConnector('database', function () use ($databaseConfig) {
$pdo = new PDO(
sprintf('mysql:host=%s; dbname=%s', $databaseConfig['host'], $databaseConfig['database']),
$databaseConfig['username'],
$databaseConfig['password']
);
$connection = new Connection($pdo);
$connectionResolver = new ConnectionResolver(['default' => $connection]);
$connectionResolver->setDefaultConnection('default');
return new DatabaseConnector($connectionResolver);
});
return $queue->getQueueManager();
I was able to reproduce the error you were seeing. I haven't looked too deeply but I think it may be due to the PDO object not setting up the connection exactly as the Illuminate Queue library expects.
This modification to using the Illuminate\Database library to create the connection solved the issue in my test environment:
$database = new \Illuminate\Database\Capsule\Manager;
$queue = new \Illuminate\Queue\Capsule\Manager;
$database->addConnection([
'driver' => 'mysql',
'host' => 'localhost',
'database' => 'db_name',
'username' => 'username',
'password' => '',
'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',
'prefix' => '',
]);
$queue->addConnector('database', function () use ($database) {
$connection = $database->getConnection();
$connectionResolver = new \Illuminate\Database\ConnectionResolver(['default' => $connection]);
$connectionResolver->setDefaultConnection('default');
return new \Illuminate\Queue\Connectors\DatabaseConnector($connectionResolver);
});
$queue->addConnection([
'driver' => 'database',
'table' => 'jobs_table',
'queue' => 'default',
'retry_after' => 90,
'after_commit' => false,
]);
$queue->getQueueManager()->push('SendEmail', ['message' => 'test']);
Like the state itself, I want to create multiple Log files for different processes.
I am using Laravel-8.
I have also created the 1 logging file with the help of logging.php in the config folder.
I want to keep track of my created commands with their own separate log file.
How can I create my separate log file for that commands?
I did this by having only a log for daily. "config/logging.php"
'channels' => [
'stack' => [
'driver' => 'stack',
'channels' => ['daily'],
'ignore_exceptions' => false,
]...,
]
You should use config/logging.php to create custom channel for each command
e.g.
'CHANNEL_NAME' => [
'driver' => 'single',
'path' => storage_path('logs/LOG_FILE_NAME.log'),
'level' => 'DESIRED_LEVEL',
],
and change CHANNEL_NAME, LOG_FILE_NAME and DESIRED_LEVEL values as you want.
Note: 'level' => 'DESIRED_LEVEL' is optional. it can be single level or array of levels (laravel v8.x).
Then when you want to log anything in your command class, Log like following code
Usage:
Log::channel('CHANNEL_NAME')
->info('Something happened!');
Below code will generate custom log file.
try{
//Your code here
}catch(\Exception $e){
$cusLog = new Logger('stack');
$err_file_name = $task_id."_".time();
$syncHistory['file_name'] = $err_file_name;
$cusLog->pushHandler(new StreamHandler(storage_path('logs/'.$err_file_name.'.log')), Logger::INFO);
$cusLog->info('error', ['Exception'=>$e->getTraceAsString(),'message'=>$e->getMessage()]);
}
Hope this will be useful.
By default Laravel Vapor pushes the laravel.log file to strerr output. Which is the picked up by Lambda and thrown to Cloudwatch. Quite hard to look through unless you are looking via the Vapor UI.
Look for an easy way to do this and push them directly to Cloudwatch (with multiple files).
Firstly added this awesome Library
composer require maxbanton/cwh
Then add this to your log config...
'cloudwatch' => [
'driver' => 'custom',
'via' => \App\Logging\CloudWatchLoggerFactory::class,
'formatter' => Monolog\Formatter\JsonFormatter::class,
'cloudwatch_stream_name' => 'laravel',
'sdk' => [
'region' => 'eu-west-1',
'version' => 'latest',
'credentials' => [
'key' => env('AWS_CW_ACCESS'),
'secret' => env('AWS_CW_SECRET')
]
],
'retention' => 730,
'level' => 'debug',
],
You'll need to add AWS_CW_ACCESS and AWS_CW_SECRET keys for an IAM user with access to Cloudwatch.
Then add App/Logging/CloudWatchLoggerFactory.php with the following contents..
<?php
namespace App\Logging;
use Aws\CloudWatchLogs\CloudWatchLogsClient;
use Maxbanton\Cwh\Handler\CloudWatch;
use Monolog\Formatter\JsonFormatter;
use Monolog\Logger;
class CloudWatchLoggerFactory
{
/**
* Create a custom Monolog instance.
*
* #param array $config
* #return \Monolog\Logger
*/
public function __invoke(array $config)
{
$sdkParams = $config["sdk"];
$tags = $config["tags"] ?? [ ];
$name = $config["name"] ?? 'cloudwatch';
// Instantiate AWS SDK CloudWatch Logs Client
$client = new CloudWatchLogsClient($sdkParams);
// Log group name, will be created if none
$groupName = config('app.name') . '-' . config('app.env');
// Log stream name, will be created if none
// $streamName = config('app.hostname');
$streamName = $config["cloudwatch_stream_name"];
// Days to keep logs, 14 by default. Set to `null` to allow indefinite retention.
$retentionDays = $config["retention"];
// Instantiate handler (tags are optional)
$handler = new CloudWatch($client, $groupName, $streamName, $retentionDays, 10000, $tags);
$handler->setFormatter(new JsonFormatter());
// Create a log channel
$logger = new Logger($name);
// Set handler
$logger->pushHandler($handler);
//$logger->pushProcessor(new CompanyLogProcessor()); //Use this if you want to adjust the JSON output using a log processor
return $logger;
}
}
You can then use that as any log... Ie Log::channel('cloudwatch')->info('hey');
To force the default laravel.log to here AND show in vapor just add this as a stack
'vapor' => [
'driver' => 'stack',
'channels' => ['stderr', 'cloudwatch'],
'ignore_exceptions' => false,
],
Then set logging.default setting to vapor in your envvars.
If you want additional logging channels just copy the cloudwatch channel setting with a new one and make sure you adjust the cloudwatch_stream_name.
Thanks to the other answer I found on Stackoverflow helping me get to here. I wanted to log this directly under answer for Laravel Vapor as I imagine many others will get stuck trying to do this!
I'm using the following change my log path:
\Log::useDailyFiles(...)
But I still get log entries in /storage/logs/. How can I use only my log path?
Laravel already registers an instance of the logger when bootstrapping the ConfigureLogging class. So when you use Log::useDailyFiles() you're just adding an additional log handler, that's why you also get log entries in the standard storage/logs/laravel.log.
To override the default log handler, Laravel offers the configureMonologUsing method available on the application instance. So in your bootstrap/app.php file just before the return $app; statement, add the following:
$app->configureMonologUsing(function($monolog) use ($app) {
$monolog->pushHandler(
(new Monolog\Handler\RotatingFileHandler(
// Set the log path
'/custom/path/to/custom.log',
// Set the number of daily files you want to keep
$app->make('config')->get('app.log_max_files', 5)
))->setFormatter(new Monolog\Formatter\LineFormatter(null, null, true, true))
);
});
The second parameter passed to the RotatingFileHandler tries to get a configuration value for log_max_files from config/app.php to determine how many daily log files it should keep, and if it doesn't find one it defaults to 5. If you want to keep an unlimited number of daily log files just pass 0 instead.
You can read more about logging configuration in the Laravel Documentation.
Laravel 5 : bootstrap/app.php
CUSTOM DAILY LOG :
$app->configureMonologUsing(function($monolog) use ($app) {
$monolog->pushHandler(
(new Monolog\Handler\RotatingFileHandler(
// Set the log path
$app->storagePath().'/logs/app_error.log',
// Set the number of daily files you want to keep
$app->make('config')->get('app.log_max_files', 30)
))->setFormatter(new Monolog\Formatter\LineFormatter(null, null, true, true))
);
});
SINGLE LOG :
$app->configureMonologUsing(function($monolog) use ($app) {
$handler = new Monolog\Handler\StreamHandler($app->storagePath().'/logs/app_error.log');
$handler->setFormatter(new \Monolog\Formatter\LineFormatter(null, null, true, true));
$monolog->pushHandler($handler);
});
For those still coming across this post, I believe changing your log file location is now easier in newer versions of Laravel. I am currently using 8.x.
In your /config/logging.php, you can define the path for your single and daily logs. Then, update whichever one you are looking to change.
'single' => [
'driver' => 'single',
'path' => "/your/desired/log/path/file.log", // edit here
'level' => env('LOG_LEVEL', 'debug'),
],
'daily' => [
'driver' => 'daily',
'path' => "/your/desired/log/path/file.log", // edit here
'level' => env('LOG_LEVEL', 'debug'),
'days' => 14,
]
I want to send Monolog logs from my Laravel 5.1 application to Loggly.com online log management service. From all possible environment, including local development.
I have found some outdated libs and complicated ways to do this. So I ended up with very simple solution. Actually, Laravel Monolog Handler already have Loggly Handler out of the box.
Add config info to config/services.php:
'loggly' => array(
'key' => 'ENTER_YOUR_LOGGLY_TOKEN_HERE',
'tag' => 'ProjectName_' .strtolower(env('APP_ENV')),
),
Than add Monolog handler in bootstrap/app.php, before $app is returned:
/*
|--------------------------------------------------------------------------
| Setup Loggly Handler
|--------------------------------------------------------------------------
*/
$app->configureMonologUsing(function($monolog) {
$handler = new \Monolog\Handler\LogglyHandler(config('services.loggly.key'),\Monolog\Logger::DEBUG);
$handler->setTag(config('services.loggly.tag'));
$monolog->pushHandler($handler);
});
Voila! You are getting your Monolog Logs in Loggly dashboard.
UPDATE: (thanks #thitami)
Based on laravel.com/docs/5.6/upgrade
The configureMonologUsing Method
If you were using the configureMonologUsing method to customize the Monolog instance for your application, you should now create a custom Log channel. For more information on how to create custom channels, check out the full logging documentation.
I was able to manage having Laravel's default local log behaviour, and pushing to Loggly in the same time, by tweaking mladen-janjetovic's code a bit. Tested on Laravel 5.3
config/services.php:
'loggly' => [
'key' => 'ENTER_YOUR_LOGGLY_TOKEN_HERE',
'tag' => 'ProjectName_' .strtolower(env('APP_ENV')),
],
bootstrap/app.php:
/*
|--------------------------------------------------------------------------
| Push to Loggly, and save locally.
|--------------------------------------------------------------------------
*/
$app->configureMonologUsing(function($monolog) use ($app) {
$log = $app->make(Illuminate\Log\Writer::class);
$logglyHandler = new \Monolog\Handler\LogglyHandler(config('services.loggly.key'));
$logglyHandler->setTag(config('services.loggly.tag'));
if (config('app.env') == 'production')
{
// Push to Loggly and save local if in production
$log->getMonolog()->pushHandler($logglyHandler);
$log->useFiles(storage_path('/logs/laravel.log'));
}
else
{
// Otherwise, save only locally
$log->useFiles(storage_path('/logs/laravel.log'));
}
});
Alternatively, you may use Monolog-Cascade to do this.
Monolog-Cascade is a Monolog extension that allows you to set up and configure multiple loggers and handlers from a single config file.
Here is a sample config file for Monolog-Cascade using Loggly. This would log to you stdOut and to Loggly:
---
handlers:
console:
class: Monolog\Handler\StreamHandler
level: DEBUG
stream: php://stdout
error_loggly_handler:
class: Monolog\Handler\LogglyHandler
level: ERROR
token: xxxx-xxxx-xxxxxxxx
tags: [cascade, waterfall]
loggers:
my_logger:
handlers: [console, error_loggly_handler]
If you're interested, here is a blog post on Cascade => https://medium.com/orchard-technology/enhancing-monolog-699efff1051d
[Disclaimer]: I am the main contributor of Monolog-Cascade.
Got mine working with little configuration with Laravel 8.
Just use the built-in monolog handler for Loggly.
Edit your app/config/logging.php
use Monolog\Handler\LogglyHandler;
'channels' => [
'stack' => [
'driver' => 'stack',
'channels' => ['single', 'loggly'],
'ignore_exceptions' => false,
],
'loggly' => [
'driver' => 'monolog',
'level' => env('LOG_LEVEL', 'debug'),
'handler' => LogglyHandler::class,
'with' => [
'token' => env('LOGGLY_TOKEN'),
],
],
]
For more advanced logging (for my case I need to set the tag as it was missing in the built-in handler's constructor.
Copy the built-in handler where you can find it within vendor folder
(e.g: vendor/monolog/monolog/src/Monolog/Handler/LogglyHandler.php) into your app folder of choice (e.g: app/Logging/CustomLogglyHandler.php).
Modify the constructor to set the tags, and you need to change some of the imports as we're on different namespaces.
// app/Logging/CustomLogglyHandler.php
namespace App\Logging;
use Monolog\Handler\AbstractProcessingHandler;
use Monolog\Handler\MissingExtensionException;
use Monolog\Logger;
use Monolog\Formatter\FormatterInterface;
use Monolog\Formatter\LogglyFormatter;
use function array_key_exists;
use CurlHandle;
use Monolog\Handler\Curl\Util as CurlUtil;
public function __construct(string $token, array|string $tag = [], $level = Logger::DEBUG, bool $bubble = true)
{
if (!extension_loaded('curl')) {
throw new MissingExtensionException('The curl extension is needed to use the LogglyHandler');
}
$this->token = $token;
if (is_array($tag)) {
$this->tag = $tag;
} else {
$this->tag = [$tag];
}
parent::__construct($level, $bubble);
}
// config/logging.php
'loggly' => [
'driver' => 'monolog',
'level' => env('LOG_LEVEL', 'debug'),
'handler' => CustomLogglyHandler::class,
'with' => [
'token' => env('LOGGLY_TOKEN'),
'tag' => strtolower(env('APP_NAME', 'Laravel')) . '_' . strtolower(env('APP_ENV', 'production'))
],
],
To expand on Hassan's contribution (posting as an answer, as I still don't have enough reputation to post a comment).
If you have a need to use daily logs locally, you could use following code:
$logFile = 'laravel'.'.txt';
$log->useDailyFiles(storage_path().'/logs/'.$logFile);
Of course, logfile name is totally arbitrary. In this example, format will be as such:
laravel-YYYY-MM-DD.txt
Edit:
with an upgrade to 5.4 this line does not work anymore:
$log = $app->make(Illuminate\Log\Writer::class);
As a workaround, you can create Writer instance manually, injecting $monolog available from configureMonologUsing closure:
$log = new Illuminate\Log\Writer($monolog);