Laravel Vapor Custom Logs to Amazon AWS Cloudwatch as JSON - laravel

By default Laravel Vapor pushes the laravel.log file to strerr output. Which is the picked up by Lambda and thrown to Cloudwatch. Quite hard to look through unless you are looking via the Vapor UI.
Look for an easy way to do this and push them directly to Cloudwatch (with multiple files).

Firstly added this awesome Library
composer require maxbanton/cwh
Then add this to your log config...
'cloudwatch' => [
'driver' => 'custom',
'via' => \App\Logging\CloudWatchLoggerFactory::class,
'formatter' => Monolog\Formatter\JsonFormatter::class,
'cloudwatch_stream_name' => 'laravel',
'sdk' => [
'region' => 'eu-west-1',
'version' => 'latest',
'credentials' => [
'key' => env('AWS_CW_ACCESS'),
'secret' => env('AWS_CW_SECRET')
]
],
'retention' => 730,
'level' => 'debug',
],
You'll need to add AWS_CW_ACCESS and AWS_CW_SECRET keys for an IAM user with access to Cloudwatch.
Then add App/Logging/CloudWatchLoggerFactory.php with the following contents..
<?php
namespace App\Logging;
use Aws\CloudWatchLogs\CloudWatchLogsClient;
use Maxbanton\Cwh\Handler\CloudWatch;
use Monolog\Formatter\JsonFormatter;
use Monolog\Logger;
class CloudWatchLoggerFactory
{
/**
* Create a custom Monolog instance.
*
* #param array $config
* #return \Monolog\Logger
*/
public function __invoke(array $config)
{
$sdkParams = $config["sdk"];
$tags = $config["tags"] ?? [ ];
$name = $config["name"] ?? 'cloudwatch';
// Instantiate AWS SDK CloudWatch Logs Client
$client = new CloudWatchLogsClient($sdkParams);
// Log group name, will be created if none
$groupName = config('app.name') . '-' . config('app.env');
// Log stream name, will be created if none
// $streamName = config('app.hostname');
$streamName = $config["cloudwatch_stream_name"];
// Days to keep logs, 14 by default. Set to `null` to allow indefinite retention.
$retentionDays = $config["retention"];
// Instantiate handler (tags are optional)
$handler = new CloudWatch($client, $groupName, $streamName, $retentionDays, 10000, $tags);
$handler->setFormatter(new JsonFormatter());
// Create a log channel
$logger = new Logger($name);
// Set handler
$logger->pushHandler($handler);
//$logger->pushProcessor(new CompanyLogProcessor()); //Use this if you want to adjust the JSON output using a log processor
return $logger;
}
}
You can then use that as any log... Ie Log::channel('cloudwatch')->info('hey');
To force the default laravel.log to here AND show in vapor just add this as a stack
'vapor' => [
'driver' => 'stack',
'channels' => ['stderr', 'cloudwatch'],
'ignore_exceptions' => false,
],
Then set logging.default setting to vapor in your envvars.
If you want additional logging channels just copy the cloudwatch channel setting with a new one and make sure you adjust the cloudwatch_stream_name.
Thanks to the other answer I found on Stackoverflow helping me get to here. I wanted to log this directly under answer for Laravel Vapor as I imagine many others will get stuck trying to do this!

Related

laravel queue disable database logging for failed jobs

Is there a way to disable the database logging of laravel when a job failed?
For example I'm just trying to write a log message, which would enough for this specific job:
job.php
public function handle()
{
//making an API request to an external API, storing some data inside cache
}
public function failed(Throwable $exception)
{
Log::info("external API update failed");
}
I already tried to edit the config/queue.php file by:
'failed' => [
'driver' => env('QUEUE_FAILED_DRIVER', 'database-uuids'),
'database' => null,
'table' => null,
],
This doesn't work, any idea how to get the database logging of failed jobs disabled?
I believe you have to set config/queue.php to the following:
'failed' => [
'driver' => null
'database' => null,
'table' => null,
],
You should be able to find clues within the laravel/framework folder within the vendor folder in your project, e.g.
vendor/laravel/framework/src/Illuminate/Queue/DatabaseFailedJobProvider.php.
The saving to database is done in the DatabaseFailedJobProvider class, the log function specifically.
Based on the QueueServiceProvider's registerFailedJobServices function, you would need to set the driver to null or the 'null' string in order to get it to run the NullFailedJobProvider, where the log function is empty.

API Platform - Custom controller / action description annotations not working

Really sorry if I've missed something here, but I have searched the issues and docs high and low, and I cannot find why this is not working. Perhaps I need another tutorial on using the internet ;)
Consider the following entity annotation, in the openapi_context section the body.description and responses.200.description have no effect at all, and its driving me slightly mad... You should know that I am using de/normalization contexts but that doesn't seem related so I have left it out.
/**
* User entity
*
* #API\ApiResource(
* collectionOperations={
* "authenticate"={
* "method"="POST",
* "status"=200,
* "path"="/authenticate",
* "controller"=UserLoginController::class,
* "openapi_context"={
* "responses"={"200"={"description"="API token and secret"}},
* "body"={"description"="Login details"},
* "summary"="User authentication",
* "description"="Provides auth tokens on success",
* },
* },
* }
* )
*
*/
class User implements UserInterface
{
...
}
The result (blue blocks as expected, red blocks not working):
I have spent way too much time on this issue as it is, I would really appreciate it if someone could help me put this to bed. I have checked/tried the following to no avail;
Creating Custom Operations and Controllers
Custom Symfony Action with API Platform bundle
[Question] Documentation API with Swagger #143
Changing Operations in the OpenAPI Documentation
Composer versions (relevant parts):
{
"require": {
"php": ">=7.2.5",
"api-platform/core": "^2.6",
"symfony/framework-bundle": "5.2.*"
}
}
After some experimenting, I found that the openapi_context annotation attribute does indeed seems to ignore the response documentation. It does however allow you to provide the request body description that you are missing:
#[ApiResource(
collectionOperations: [
'test' => [
'method' => 'POST',
'path' => '/test',
'openapi_context' => [
'summary' => 'The endpoint summary',
'description' => 'The endpoint description',
'requestBody' => [
'description' => 'The endpoint request body description', // This one
'content' => [
'application/json' => [
'schema' => [
'$ref' => '#/components/schemas/MyResource-some.group'
],
],
],
],
],
]
],
)]
I'm using PHP 8.0.3 and API Platform 2.6.3 while writing this, changing the annotations to docblocks should result in the same outcome for you though.
In order to document the endpoint response specification however, I had to implement a custom OpenApiFactoryInterface:
<?php declare(strict_types = 1);
namespace App;
use ApiPlatform\Core\OpenApi\Factory\OpenApiFactoryInterface;
use ApiPlatform\Core\OpenApi\Model\Operation;
use ApiPlatform\Core\OpenApi\Model\PathItem;
use ApiPlatform\Core\OpenApi\Model\RequestBody;
use ApiPlatform\Core\OpenApi\OpenApi;
use ArrayObject;
use UnexpectedValueException;
class MyResourceOpenApiFactory implements OpenApiFactoryInterface
{
private OpenApiFactoryInterface $openApiFactory;
public function __construct(OpenApiFactoryInterface $openApiFactory)
{
$this->openApiFactory = $openApiFactory;
}
public function __invoke(array $context = []): OpenApi
{
$openApi = ($this->openApiFactory)($context);
$components = $openApi->getComponents();
$schemas = $components->getSchemas();
if (null === $schemas) {
throw new UnexpectedValueException('Failed to obtain OpenApi schemas');
}
$pathItem = new PathItem(
'MyResource test endpoint',
'A test summary',
'A test description',
null,
null,
// Your custom post operation
new Operation(
'testMyResourceCollection', // the operation route name
[
'MyResource' // your resource name
],
[
// response specifications
'201' => [
'description' => 'test endpoint 201 response description',
'content' => [
'application/json' => [
'schema' => [
'$ref' => '#/components/schemas/MyResource-read', // your resource (read) schema
],
],
],
],
],
'A test endpoint summary',
'A test endpoint description',
null,
[],
new RequestBody(
'A test request body description',
new ArrayObject([
'application/json' => [
'schema' => [
'$ref' => '#/components/schemas/MyResource-write', // your resource (write) schema
],
],
]),
),
),
);
$paths = $openApi->getPaths();
$paths->addPath('/my_resources/test', $pathItem);
return $openApi;
}
}
Have this OpenApiFactoryInterface implementation wired by the service container as a decorator to api_platform.openapi.factory:
App\MyResourceOpenApiFactory:
decorates: 'api_platform.openapi.factory'
autoconfigure: false
Change the references to the example MyResource name to a resource name of your choice (like User).
Sidenote:
This whole process of customizing OpenApi endpoint documentation in API Platform is currently quite convoluted in my opinion. Use the implementation I provided as a reference to your own implementation, as you most likely need to make a few adjustments to it in order to make it satisfy your specific use case.

Pusher and Laravel 5.3 Event Broadcasting

Trying to use Laravel 5.3 with pusher, but it seems its not working correct in my code.
My .env is correct
PUSHER_APP_ID= myappid
PUSHER_KEY= mykey
PUSHER_SECRET= mysecret
This is my 'pusher' configurations in broadcasting.php
'pusher' => [
'driver' => 'pusher',
'key' => env('PUSHER_KEY'),
'secret' => env('PUSHER_SECRET'),
'app_id' => env('PUSHER_APP_ID'),
'options' => [
'cluster' => 'eu',
'encrypted' => true,
],
],
I created an event, here it is
<?php
namespace App\Events;
use Illuminate\Broadcasting\Channel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Broadcasting\PresenceChannel;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
class ProposalEvent implements ShouldBroadcast
{
use InteractsWithSockets, SerializesModels;
public $data;
/**
* Create a new event instance.
*
* #return void
*/
public function __construct($data)
{
$this->data = $data;
}
/**
* Get the channels the event should broadcast on.
*
* #return Channel|array
*/
public function broadcastOn()
{
return ['test-channel'];
// return new PrivateChannel('test-channel');
// return new PresenceChannel('test-channel');
}
}
my javascript
Pusher.logToConsole = true;
var pusher = new Pusher("mykey", {
cluster: 'eu',
encrypted: true
});
var channel = pusher.subscribe('test-channel');
channel.bind('App\\Events\\ProposalEvent', function(data) {
alert(data);
});
and finally in my view
event(new App\Events\ProposalEvent('some data'));
unfortunately this is not working for me, but when i use pusher->trigger like this, without event, it working fine, and i see message in pusher debug console
$options = array(
'cluster' => 'eu',
'encrypted' => true
);
$pusher = new Pusher(
'mykey',
'mysecret',
'myid',
$options
);
$data['message'] = 'some data';
$pusher->trigger('test-channel', 'my-event', $data);
I have searched for solution in Laravel documentation and other resources. There are questions with same problem in stackoverflow, but there are no response.I will be grateful if somebody can help me, because i can't find solution for several days
I was stuck in the same situation and found out that I wasn't using the queue!
In the documentation it says
Before broadcasting events, you will also need to configure and run a queue listener. All event broadcasting is done via queued jobs so that the response time of your application is not seriously affected.
I deleted the config/queue.php file before since I thought I wasn't using it. Maybe you're doing the same thing as me or having some problem with the queue.
Try to pass pusher credentials directly through config/broadcasting.php
It worked for me.
'default' => 'pusher',
'connections' => [
'pusher' => [
'driver' => 'pusher',
'key' => '***',
'secret' => '**',
'app_id' => '**',
'options' => [
],
],
],
enter code here
triggering event in the views is a mistake when a page is loaded the php is execute first then js is loaded wich mean that you fired event with php before js listner wich will displaye an error this can the first case
the second case is that you didn't downloaded https://curl.haxx.se/docs/caextract.html
if you did go to php.ini and in [curl]you will find somthing like curl-info unecomment it
and right the path to culr file

Laravel change log path

I'm using the following change my log path:
\Log::useDailyFiles(...)
But I still get log entries in /storage/logs/. How can I use only my log path?
Laravel already registers an instance of the logger when bootstrapping the ConfigureLogging class. So when you use Log::useDailyFiles() you're just adding an additional log handler, that's why you also get log entries in the standard storage/logs/laravel.log.
To override the default log handler, Laravel offers the configureMonologUsing method available on the application instance. So in your bootstrap/app.php file just before the return $app; statement, add the following:
$app->configureMonologUsing(function($monolog) use ($app) {
$monolog->pushHandler(
(new Monolog\Handler\RotatingFileHandler(
// Set the log path
'/custom/path/to/custom.log',
// Set the number of daily files you want to keep
$app->make('config')->get('app.log_max_files', 5)
))->setFormatter(new Monolog\Formatter\LineFormatter(null, null, true, true))
);
});
The second parameter passed to the RotatingFileHandler tries to get a configuration value for log_max_files from config/app.php to determine how many daily log files it should keep, and if it doesn't find one it defaults to 5. If you want to keep an unlimited number of daily log files just pass 0 instead.
You can read more about logging configuration in the Laravel Documentation.
Laravel 5 : bootstrap/app.php
CUSTOM DAILY LOG :
$app->configureMonologUsing(function($monolog) use ($app) {
$monolog->pushHandler(
(new Monolog\Handler\RotatingFileHandler(
// Set the log path
$app->storagePath().'/logs/app_error.log',
// Set the number of daily files you want to keep
$app->make('config')->get('app.log_max_files', 30)
))->setFormatter(new Monolog\Formatter\LineFormatter(null, null, true, true))
);
});
SINGLE LOG :
$app->configureMonologUsing(function($monolog) use ($app) {
$handler = new Monolog\Handler\StreamHandler($app->storagePath().'/logs/app_error.log');
$handler->setFormatter(new \Monolog\Formatter\LineFormatter(null, null, true, true));
$monolog->pushHandler($handler);
});
For those still coming across this post, I believe changing your log file location is now easier in newer versions of Laravel. I am currently using 8.x.
In your /config/logging.php, you can define the path for your single and daily logs. Then, update whichever one you are looking to change.
'single' => [
'driver' => 'single',
'path' => "/your/desired/log/path/file.log", // edit here
'level' => env('LOG_LEVEL', 'debug'),
],
'daily' => [
'driver' => 'daily',
'path' => "/your/desired/log/path/file.log", // edit here
'level' => env('LOG_LEVEL', 'debug'),
'days' => 14,
]

How to send Log event from Laravel to Loggly?

I want to send Monolog logs from my Laravel 5.1 application to Loggly.com online log management service. From all possible environment, including local development.
I have found some outdated libs and complicated ways to do this. So I ended up with very simple solution. Actually, Laravel Monolog Handler already have Loggly Handler out of the box.
Add config info to config/services.php:
'loggly' => array(
'key' => 'ENTER_YOUR_LOGGLY_TOKEN_HERE',
'tag' => 'ProjectName_' .strtolower(env('APP_ENV')),
),
Than add Monolog handler in bootstrap/app.php, before $app is returned:
/*
|--------------------------------------------------------------------------
| Setup Loggly Handler
|--------------------------------------------------------------------------
*/
$app->configureMonologUsing(function($monolog) {
$handler = new \Monolog\Handler\LogglyHandler(config('services.loggly.key'),\Monolog\Logger::DEBUG);
$handler->setTag(config('services.loggly.tag'));
$monolog->pushHandler($handler);
});
Voila! You are getting your Monolog Logs in Loggly dashboard.
UPDATE: (thanks #thitami)
Based on laravel.com/docs/5.6/upgrade
The configureMonologUsing Method
If you were using the configureMonologUsing method to customize the Monolog instance for your application, you should now create a custom Log channel. For more information on how to create custom channels, check out the full logging documentation.
I was able to manage having Laravel's default local log behaviour, and pushing to Loggly in the same time, by tweaking mladen-janjetovic's code a bit. Tested on Laravel 5.3
config/services.php:
'loggly' => [
'key' => 'ENTER_YOUR_LOGGLY_TOKEN_HERE',
'tag' => 'ProjectName_' .strtolower(env('APP_ENV')),
],
bootstrap/app.php:
/*
|--------------------------------------------------------------------------
| Push to Loggly, and save locally.
|--------------------------------------------------------------------------
*/
$app->configureMonologUsing(function($monolog) use ($app) {
$log = $app->make(Illuminate\Log\Writer::class);
$logglyHandler = new \Monolog\Handler\LogglyHandler(config('services.loggly.key'));
$logglyHandler->setTag(config('services.loggly.tag'));
if (config('app.env') == 'production')
{
// Push to Loggly and save local if in production
$log->getMonolog()->pushHandler($logglyHandler);
$log->useFiles(storage_path('/logs/laravel.log'));
}
else
{
// Otherwise, save only locally
$log->useFiles(storage_path('/logs/laravel.log'));
}
});
Alternatively, you may use Monolog-Cascade to do this.
Monolog-Cascade is a Monolog extension that allows you to set up and configure multiple loggers and handlers from a single config file.
Here is a sample config file for Monolog-Cascade using Loggly. This would log to you stdOut and to Loggly:
---
handlers:
console:
class: Monolog\Handler\StreamHandler
level: DEBUG
stream: php://stdout
error_loggly_handler:
class: Monolog\Handler\LogglyHandler
level: ERROR
token: xxxx-xxxx-xxxxxxxx
tags: [cascade, waterfall]
loggers:
my_logger:
handlers: [console, error_loggly_handler]
If you're interested, here is a blog post on Cascade => https://medium.com/orchard-technology/enhancing-monolog-699efff1051d
[Disclaimer]: I am the main contributor of Monolog-Cascade.
Got mine working with little configuration with Laravel 8.
Just use the built-in monolog handler for Loggly.
Edit your app/config/logging.php
use Monolog\Handler\LogglyHandler;
'channels' => [
'stack' => [
'driver' => 'stack',
'channels' => ['single', 'loggly'],
'ignore_exceptions' => false,
],
'loggly' => [
'driver' => 'monolog',
'level' => env('LOG_LEVEL', 'debug'),
'handler' => LogglyHandler::class,
'with' => [
'token' => env('LOGGLY_TOKEN'),
],
],
]
For more advanced logging (for my case I need to set the tag as it was missing in the built-in handler's constructor.
Copy the built-in handler where you can find it within vendor folder
(e.g: vendor/monolog/monolog/src/Monolog/Handler/LogglyHandler.php) into your app folder of choice (e.g: app/Logging/CustomLogglyHandler.php).
Modify the constructor to set the tags, and you need to change some of the imports as we're on different namespaces.
// app/Logging/CustomLogglyHandler.php
namespace App\Logging;
use Monolog\Handler\AbstractProcessingHandler;
use Monolog\Handler\MissingExtensionException;
use Monolog\Logger;
use Monolog\Formatter\FormatterInterface;
use Monolog\Formatter\LogglyFormatter;
use function array_key_exists;
use CurlHandle;
use Monolog\Handler\Curl\Util as CurlUtil;
public function __construct(string $token, array|string $tag = [], $level = Logger::DEBUG, bool $bubble = true)
{
if (!extension_loaded('curl')) {
throw new MissingExtensionException('The curl extension is needed to use the LogglyHandler');
}
$this->token = $token;
if (is_array($tag)) {
$this->tag = $tag;
} else {
$this->tag = [$tag];
}
parent::__construct($level, $bubble);
}
// config/logging.php
'loggly' => [
'driver' => 'monolog',
'level' => env('LOG_LEVEL', 'debug'),
'handler' => CustomLogglyHandler::class,
'with' => [
'token' => env('LOGGLY_TOKEN'),
'tag' => strtolower(env('APP_NAME', 'Laravel')) . '_' . strtolower(env('APP_ENV', 'production'))
],
],
To expand on Hassan's contribution (posting as an answer, as I still don't have enough reputation to post a comment).
If you have a need to use daily logs locally, you could use following code:
$logFile = 'laravel'.'.txt';
$log->useDailyFiles(storage_path().'/logs/'.$logFile);
Of course, logfile name is totally arbitrary. In this example, format will be as such:
laravel-YYYY-MM-DD.txt
Edit:
with an upgrade to 5.4 this line does not work anymore:
$log = $app->make(Illuminate\Log\Writer::class);
As a workaround, you can create Writer instance manually, injecting $monolog available from configureMonologUsing closure:
$log = new Illuminate\Log\Writer($monolog);

Resources