I am trying to save the user activities in a json file but when ever the file size gets bigger and multiple users working on same time the json file deletes the old records.
this is my Trait
trait CustomLogActivity
{
protected static function bootCustomLogActivity()
{
foreach (static::getModelEvents() as $event) {
static::$event(function ($model) use ($event) {
$model->recordActivity($event);
});
}
}
protected static function getModelEvents()
{
return ['created', 'updated', 'deleted'];
}
protected function recordActivity($event)
{
$activity = [
'user_id' => Auth::id(),
'type' => $event,
'subject' => (new \ReflectionClass($this))->getShortName(),
'timestamp' => now()
];
if ($event === 'updated') {
$activity['old_properties'] = $this->getOriginal();
$activity['new_properties'] = $this->getAttributes();
} else {
$activity['properties'] = $this->getAttributes();
}
$this->appendToLog($activity);
}
protected function appendToLog($activity)
{
$logFile = 'activity.json';
$log = json_encode($activity);
Storage::append($logFile, $log);
}
protected function getActivityType($event)
{
$type = strtolower((new \ReflectionClass($this))->getShortName());
return "{$event}_{$type}";
}
}
As I mentioned in some comments, I will post it as an answer so it is explanatory for anyone having these types of issues:
The error you are having is called: concurrency.
I am assuming 2 processes uses the file at the same time, so both reads the current content, but one of them after that writes, the other process already has data in memory (so the new data is not get by this process), but not the new content, so it will overwrite the file...
First of all, use a Queue (events) to send data, and then use Redis, or a database or something that is super fast for this, but not literally a file, you can lose it instantly, but not a database...
You can still use a file bu I would not recommend to do so because it depends a lot on your infrastructure:
If you have a load balancer with 10 machines, are you going to have 10 different files (one per machine)?
How do you combine them?
So what I would do is just have a queue (triggered by using an event) and let that queue, with a single worker, handle this super specific task. But you will have to have in mind the speed, if you are getting more events in the queue than the single worker can resolve, you will have to find a solution for that
Related
I have a controller with a download action in typo3. For some time I have implemented it like this and it is working:
function downloadAction() {
// ...
// send headers ...
// ...
if ($fh = fopen($this->file, 'r')) {
while (!feof($fh)) {
echo fread($fh, $chunkSize); // send file in little chunks to output buffer
flush();
}
fclose($fh);
}
exit; // Stopp middlewares and so on.
}
I am wondering if I should/could return an object of type ResponseInterface in typo3 11. So it is obviously that exit stopps the middleware pipeline and other things and I don't really know if there are any side effects.
I tried the following to return a ResponseInterface :
function downloadAction(): ResponseInterface {
// ...
return $this->responseFactory->createResponse();
->withAddedHeader(...)
// ...
->withBody($this->streamFactory->createStreamFromFile($this->file))
->withStatus(200, 'OK');
}
The problem is that the solution with the ResponseInterface works only with small files. The problem seems to be in Bootstrap::handleFrontendRequest().
protected function handleFrontendRequest(ServerRequestInterface $request): string
{
// ...
if (headers_sent() === false) {
// send headers
}
$body = $response->getBody(); // get the stream
$body->rewind();
$content = $body->getContents(); // Problem: Read the hole stream into RAM instead of
// sending it in chunks to the output buffer
// ...
return $content;
}
typo3 tries to read the whole stream/file into RAM. That crashes the application.
So how should I trigger a file download these days with typo3?
I'm developing one of my first applications with the Laravel 4 framework (which, by the way, is a joy to design with). For one component, there is an AJAX request to query an external server. The issue is, I want to cache these responses for a certain period of time only if they are successful.
Laravel has the Cache::remember() function, but the issue is there seems to be no "failed" mode (at least, none described in their documentation) where a cache would not be stored.
For example, take this simplified function:
try {
$server->query();
} catch (Exception $e) {
return Response::json('error', 400);
}
I would like to use Cache::remember on the output of this, but only if no Exception was thrown. I can think of some less-than-elegant ways to do this, but I would think that Laravel, being such an... eloquent... framework, would have a better way. Any help? Thanks!
This is what worked for me:
if (Cache::has($key)) {
$data = Cache::get($key);
} else {
try {
$data = longQueryOrProcess($key);
Cache::forever($key, $data); // only stored when no error
} catch (Exception $e) {
// deal with error, nothing cached
}
}
And of course you could use Cache::put($key, $data, $minutes); instead of forever
I found this question, because I was looking for an answer about this topic.
In the meanwhile I found a solution and like to share it here:
(also check out example 2 further on in the code)
<?php
/**
* Caching the query - Example 1
*/
function cacheQuery_v1($server)
{
// Set the time in minutes for the cache
$minutes = 10;
// Check if the query is cached
if (Cache::has('db_query'))
{
return Cache::get('db_query');
}
// Else run the query and cache the data or return the
// error response if an exception was catched
try
{
$query = $server->query(...);
}
catch (Exception $e)
{
return Response::json('error', 400);
}
// Cache the query output
Cache::put('db_query', $query, $minutes);
return $query;
}
/**
* Caching the query - Example 2
*/
function cacheQuery_v2($server)
{
// Set the time in minutes for the cache
$minutes = 10;
// Try to get the cached data. Else run the query and cache the output.
$query = Cache::remember('db_query', $minutes, function() use ($server) {
return $server->query(...);
});
// Check if the $query is NULL or returned output
if (empty($query))
{
return Response::json('error', 400);
}
return $query;
}
I recommend you to use Laravel's Eloquent ORM and/or the Query Builder to operate with the Database.
Happy coding!
We're working around this by storing the last good value in Cache::forever(). If there's an error in the cache update callback, we just pull the last value out of the forever key. If it's successful, we update the forever key.
The main objective of this question is to have a system running permanently on the server to broadcast music.
I would like to create an artisan console function that will automatically launch a track from its parent game every 32 seconds. Unfortunately the sleep() function blocks the parent loop.
Two questions (or maybe three):
What would be the solution to prevent the parent loop from stopping?
Does this method consume too many resources? Is there a better way?
public function fetch($games) {
foreach ($games as $game) {
$tracks = Track::inRandomOrder()->where('game_id', $game->id)->limit($game->tracks_number)->get();
foreach ($tracks as $key => $track) {
broadcast(new NewTrack($track));
if($key + 1 == count($tracks)) {
$this->fetch($this->games);
}
sleep(32);
}
}
}
Many thanks for your feedback
I have a cron job that scrapes a list of items on a website and then inserts or updates records in a database. When I scrape the page, I want to create records for new ones that haven't been created yet, otherwise update any existing ones. Currently I'm doing something like this:
// pretend there is a "Widget" model defined
function createOrUpdateWidget(widgetConfig) {
return Widget.find(widgetConfig.id)
.then(function(widget) {
if (widget === null) {
return Widget.create(widgetConfig);
}
else {
widget.updateAttributes(widgetConfig);
}
});
}
function createOrUpdateWidgets(widgetConfigObjects) {
var promises = [];
widgetConfigObjects.forEach(function(widgetConfig) {
promises.push(createOrUpdateWidget(widgetConfig));
});
return Sequelize.Promise.all(promises);
}
createOrUpdateWidgets([...])
.done(function() {
console.log('Done!');
});
This seems to work fine, but I'm not sure if I'm doing this "correctly" or not. Do all promises that perform DB interactions need to run serially, or is how I have them defined ok? Is there a better way to do this kind of thing?
What you're doing is pretty idiomatic and perfectly fine, the only room for improvement is to utilize the fact Sequelize uses Bluebird for promises so you get .map for free, which lets you convert:
function createOrUpdateWidgets(widgetConfigObjects) {
var promises = [];
widgetConfigObjects.forEach(function(widgetConfig) {
promises.push(createOrUpdateWidget(widgetConfig));
});
return Sequelize.Promise.all(promises);
}
Into:
function createOrUpdateWidgets(widgetConfigObjects) {
return Sequelize.Promise.map(widgetConfig, createOrUpdateWidget)
}
Other than that minor improvement - you're chaining promises correctly and seem to have the correct hang of it.
I am looking to remove a Doctrine Extensions life cycle event listener from within a controller.
I need to remove the listener for update events because I need to update all nodes in the tree at once. Something that is not supported by the library, but is possible by directly setting the correct left, right, level etc...
Is it possible to remove a life cycle even from within a controller? What is a possible solution for this situation.
I thought something like this might work, but it did not
$evm = $em->getEventManager();
$listener = new \Gedmo\Tree\TreeListener();
$evm->removeEventListener( array( 'postUpdate' ), $listener );
yes it will work, but there are different events used:
$listenerInst = null;
$em; /* entity manager */
foreach ($em->getEventManager()->getListeners() as $event => $listeners) {
foreach ($listeners as $hash => $listener) {
if ($listener instanceof WantedListenerClass) {
$listenerInst = $listener;
break 2;
}
}
}
$listenerInst || die('Listener is not registered in the event manager');
// then you can remove events you like:
$evm = $em->getEventManager();
$evm->removeEventListener(array('onFlush'), $listenerInst);