I am working with Meilisearch in my Laravel application, and am trying to figure out the best way I can mock my Meilisearch indexes.
The tests themselves are currently not that advanced, for example my test to simply create an index looks like this
public function test_create_index()
{
$model = Package::factory()->create([
'language' => 'DA'
]);
sleep(1);
$this->assertIsArray($this->meiliClient->index('Package_DA')->getDocument($model->id));
$this->meiliClient->index('Package_DA')->deleteDocument($model->id);
}
I am using Laravel Observer to fire a method when a new model is created, which calls my Meilisearch Service to create a new index and insert a new record.
The downside here though is that in order for Meilisearch to have time to register the new record in the index, I must sleep the script for 1 second, or else it won't have time to update. This is something I have to do everytime I call the Meilisearch client. Combine this with 8 calls in total, and my test file will take over 8 seconds to run.
Next issue is, as shown in the last code line, that I have to manually delete the created record from the index once the test is run. What I would like to do is to fake the Meilisearch calls somehow to not actually create the records, but simply test if the creation succeeded, similar to when you fake Events or Job dispatches.
I am considering making a separate trait to handle this, but I am not sure if it is even possible to achieve the faking of Meilisearch Indexes in the way that I want it.
I'm open for ideas or suggestions on how this could be done, or if it would even be possible
Thanks
Related
I am currently making a turn based strategy game with laravel (mysql DB with InnoDB) engine and want to make sure that I don't have bugs due to race conditions, duplicate requests, bad actors etc...
Because these kind of bugs are hard to test, I wanted to get some clarification.
Many actions in the game can only occur once per turn, like buying a new unit. Here is a simplified bit of code for purchasing a unit.
$player = Player::find($player_id);
if($player->gold >= $unit_price && $player->has_purchased == false){
$player->has_purchased = true;
$player->gold -= $unit_price;
$player->save();
$unit = new Unit();
$unit->player_id = $player->id;
$unit->save();
}
So my concern would be if two threads both made it pass the if statement and then executed the block of code at the same time.
Is this a valid concern?
And would the solution be to wrap everything in a database transaction like https://betterprogramming.pub/using-database-transactions-in-laravel-8b62cd2f06a5 ?
This means that a good portion of my code will be wrapped around database transactions because I have a lot of instances that are variations of the above code for different actions.
Also there is a situation where multiple users will be able to update a value in the database so I want to avoid a situation where 2 users increment the value at the same time and it only gets incremented once.
Since you are using Laravel to presumably develop a web-based game, you can expect multiple concurrent connections to occur. A transaction is just one part of the equation. Transactions ensure operations are performed atomically, in your case it ensures that both the player and unit save are successful or both fail together, so you won't have the situation where the money is deducted but the unit is not granted.
However there is another facet to this, if there is a real possibility you have two separate requests for the same player coming in concurrently then you may also encounter a race condition. This is because a transaction is not a lock so two transactions can happen at the same time. The implication of this is (in your case) two checks happen on the same player instance to ensure enough gold is available, both succeed, and both deduct the same gold, however two distinct units are granted at the end (i.e. item duplication). To avoid this you'd use a lock to prevent other threads from obtaining the same player row/model, so your full code would be:
DB::transaction(function () use ($unit_price) {
$player = Player::where('id',$player_id)->lockForUpdate()->first();
if($player->gold >= $unit_price && $player->has_purchased == false){
$player->has_purchased = true;
$player->gold -= $unit_price;
$player->save();
$unit = new Unit();
$unit->player_id = $player->id;
$unit->save();
}
});
This will ensure any other threads trying to retrieve the same player will need to wait until the lock is released (which will happen at the end of the first request).
There's more nuances to deal with here as well like a player sending a duplicate request from double-clicking for example, and that can get a bit more complex.
For you purchase system, it's advisable to implement DB:transaction since it protects you from false records. Checkout the laravel docs for more information on this https://laravel.com/docs/9.x/database#database-transactions As for reactive data you need to keep track of, simply bind a variable to that data in your frontEnd, then use the variable to update your DB records.
In the case you need to exit if any exception or error occurs. If an exception is thrown the data will not save and rollback all the transactions. I recommand to use transactions as possible as you can. The basic format is:
DB::beginTransaction();
try {
// database actions like create, update etc.
DB::commit(); // finally commit to database
} catch (\Exception $e) {
DB::rollback(); // roll back if any error occurs
// something went wrong
}
See the laravel docs here
End Goal
The aim is for my application to fire off potentially a lot of emails to the Redis queue (This bit is working) and then Redis throttle the processing of these to only a set number of emails every selected number of minutes.
For this example, I have a test job that appends the time to a file and I am attempting to throttle it to once every 60 seconds.
The story so far....
So far, I have the application successfully pushing a test amount of 50 jobs to the Redis queue. I can log in to Horizon and see these 50 jobs in the "processjob" queue. I can also log in to redis-cli and see 50 sets under the list key "queues:processjob".
My issue is that as soon as I attempt to put the throttle on, only 1 job runs and the rest fail with the following error:
Predis\Response\ServerException: ERR Error running script (call to f_29cc07bd431ccbf64637e5dcb60484560fdfa2da): #user_script:10: WRONGTYPE Operation against a key holding the wrong kind of value in /var/www/html/smhub/vendor/predis/predis/src/Client.php:370
If I remove the throttle, all works file and 5 jobs are instantly ran.
I thought maybe it was the incorrect key name but if I change the following:
public function handle()
{
//
Redis::throttle('queues:processjob')->allow(1)->every(60)->then(function(){
Storage::disk('local')->append('testFile.txt',date("Y-m-d H:i:s"));
}, function (){
return $this->release(10);
});
}
to this:
public function handle()
{
//
Redis::funnel('queues:processjob')->limit(1)->then(function(){
Storage::disk('local')->append('testFile.txt',date("Y-m-d H:i:s"));
}, function (){
return $this->release(10);
});
}
then it all works fine.
My thoughts...
Something tells me that the issue is that the redis key is of type "list" and that the jobs are all under a single list. That being said, if it didn't work this way, how would we throttle a queue as the throttle requires a unique key.
For anybody else that is having issues attempting to get this to work and is getting the same issue as I was, this is what resolved my issues:
The Fault
I assumed that Redis::throttle('queues:processjob') was meant to be referring to the queue that you wanted to be throttled. However, after some re-reading of the documentation and testing of the code, I realized that this was not the case.
The Fix
Redis::throttle('queues:processjob') is meant to point to it's own 'holding' queue and so must be a unique Redis key name. Therefore, changing it to Redis::throttle('throttle:queues:processjob') worked fine for me.
The workings
When I first looked in to this, I assumed that that Redis::throttle('this') throttled the queue that you specified. To some degree this is correct but it will not work if the job was created via another means.
Redis::throttle('this') actually creates a new 'holding' queue where the jobs go until the condition(s) you specify are met. So jobs will go to the queue 'this' in this example and when the throttle trigger is released, they will be passed to the queue specified in their execution code. In this case, 'queues:processjob'.
I hope this helps!
i'm feeling confused, which are best practices between laravel observer or MySQL Trigger.
In Laravel, My Code looks like
public function updated(My_Activity $my_activity)
{
$activity = new Activity_Log();
$activity->activity_id = $my_activity->id;
$activity->status = $my_activity->status;
$activity->description = $my_activity->description;
$activity->save();
}
In MySQL,
BEGIN
INSERT INTO Activity_Log
SET id = OLD.id, status = OLD.status, description = OLD.description
END
What is the best practice ? Is there a good impact for one of them in the future?
I prefer Laravel Observer option because it allows you to keep your business logic in your application (source control). Additionally you keep your business logic at the same abstraction level by using Eloquent.
For the same reason Laravel introduced a Task scheduler. It allows you to keep your cron entries under source control.
https://laravel.com/docs/5.5/scheduling
In the past, you may have generated a Cron entry for each task you
needed to schedule on your server. However, this can quickly become a
pain, because your task schedule is no longer in source control and
you must SSH into your server to add additional Cron entries.
I've a Laravel 5.2 application, I'm using the scheduler to runs a script every 30 minutes by now. But I'm wondering if that time can be retrieved from the database, I want that the admin user configure that time from the webpage, I have the field in the database. But I'm not really sure if the scheduler can update that time from the database, so, is it possible?
Also, I setted:
->sendOutputTo("/var/www/html/laravelProject/public/output")
to save the output in a file named output, but it's not seems that is working even when the cron is executed, I checked the cron log files and there only shows that there is not MTA installed, but I don't want for the moments to send a e-mail, I just want to save the output in a file, so, what am I missing there?
You can use when
The when method may be used to limit the execution of a task based on the result of a given truth test.
$schedule->command('yourcommand:execute')->everyMinutes()->when(function () {
if($timefromdb){
return true;
}
else{
return false;
}
});
https://laravel.com/docs/master/scheduling#schedule-frequency-options
So I have Laravel 5.2 project where Redis is used as cache driver.
There is a controller, which has method that connects Redis and increases a value and adds value to the set each time this method is called, just like
$redis = Redis::connection();
$redis->incr($value);
$redis->sadd("set", $value);
But the problem is that sometimes there are many connections and many calls for this method at the same time, and there is a data loss, because if two callers call this method while $value is 2, after incr it will become 3, but should be 4 (after two incrs basically).
I have thought about using Redis transactions, but I can't imagine when should I call multi command to start a queue and when to exec it.
Also I had an idea to collect all incrs and sadds as strings to another set and then transact them with cron job, but it would cost too much RAM.
So, any suggestions, how can this data loss be avoided?
Laravel uses Predis as a Redis driver.
To execute a transaction with Predis you have to invoke the transaction method of the driver and give it a callback:
$responses = $redis->transaction(function ($tx) {
$redis->incr($value);
$redis->sadd("set", $value);
});