Laravel queue technique: Fail, Retry, or Delay

The article was forwarded from the professional Laravel developer community with the original link: https://learnku.com/laravel/t...

When creating queue jobs, listeners, or subscribers to push into a queue, you may begin to think that once dispatched, it is entirely up to you for the queue worker to decide how to handle your logic.

Hmm...It's not that you can't interact with the queue worker from within your job, but it's often unnecessary even if you do.

This amazing operation happened because of the trait InteractsWithQueue..When a queued job is being pulled out of the queue, this [CallQueuedListener](https://github.com/laravel/framework/blob/5.8/src/Illuminate/Events/CallQueuedListener.php#L90-L104) checks if it is using the InteractsWithQueue trait, and if so, the framework injects the underlying instance of "Queued jobs" internally.

This Task instance is similar to a driver that wraps a real Job class and contains information such as queue connections and attempts.

background

I'll take a transcoded Job as an example.This is a task to convert broadcast audio files to 192kbps MP3 format.Because this is set in a free transcode queue, it has a limited role.

Check Attempts

attempts() is the first method called, and as the name implies, it returns the number of attempts with a queue job always starting with an attempt.

This method is intended to be used with other methods... like fail or release. For illustration purposes, we will inform the user of the number of retries: every time we try to convert (convert code) in an idle queue, we will inform the user of the number of retries we are trying to cancel the future conversion (convert code).

<?php
namespace App\Jobs;
use App\Podcast;
use Transcoder\Transcoder;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use App\Notifications\PodcastTranscoded;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use App\Notifications\RetyingPodcastTranscode;
class TranscodePodcast
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    /**
     * Transcoder Instance
     *
     * @var \App\Podcast
     */
    protected $podcast;
    /**
     * Create a new transcoded podcast instance.
     *
     * @param \App\Podcast $podcast
     * @return void
     */
    public function __construct(Podcast $podcast)
    {
        $this->podcast = $podcast;
    }
    /**
     * Execute queue job.
     *
     * @param \Transcoder\Transcoder $podcast
     * @return void
     */
    public function handle(Transcoder $transcoder)
    {
        // Tell users how many retries we have
        if ($this->attempts() > 1) {
            $this->podcast->publisher->notify(new RetryingPodcastTranscode($this->podcast, $this->attempts());
        }
        $transcoded = $this->transcoder->setFile($event->podcast)
            ->format('mp3')
            ->bitrate(192)
            ->start();
            

        // Associate transcoded podcast with original Podcast
        $this->podcast->transcode()->associate($transcoded);
        
        // Notify the publisher of podcast that his podcast is ready

        $this->publisher->notify(new PodcastTranscoded($this->podcast));
    }
}

Telling users the number of times we will try something again is useful when the logic fails beforehand, allowing users (or developers) to check for problems, but of course you can do more.

Personally, I like to do this after Job Job fails. If there is still time to try again, tell him that we will try again later. Of course, this is just an example.

Delete Job Queue

The second method is delete(). As you might guess, you can delete the current Queue Job from the queue.
This is convenient when the queue Job or listener should not be processed after queuing for a variety of reasons, for example, consider the scenario where the publisher uploading the podcast is disabled for any reason (such as a TOS conflict) before transcoding occurs and we should not process the podcast.

We'll add that code in the previous example:

<?php
namespace App\Jobs;
use App\Podcast;
use Transcoder\Transcoder;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use App\Notifications\PodcastTranscoded;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use App\Notifications\RetyingPodcastTranscode;
class TranscodePodcast
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    /**
     * Transcoder Instance
     *
     * @var \App\Podcast
     */
    protected $podcast;
    /**
     * Create a new transcoded podcast instance.

     *
     * @param \App\Podcast $podcast
     * @return void
     */
    public function __construct(Podcast $podcast)
    {
        $this->podcast = $podcast;
    }
    /**
     * Execute queue job.
     *
     * @param \Transcoder\Transcoder $podcast
     * @return void
     */
    public function handle(Transcoder $transcoder)
    {
        // If the Publisher is disabled, delete this queue job
        if ($this->podcast->publisher->isDeactivated()) {
            $this->delete();
        }
        // Tell users how many retries we have
        if ($this->attempts() > 1) {
            $this->podcast->publisher->notify(new RetryingPodcastTranscode($this->podcast, $this->attempts());
        }
        $transcoded = $this->transcoder->setFile($event->podcast)
            ->format('mp3')
            ->bitrate(192)
            ->start();
            
        // Associate transcoded podcast with original Podcast
        $this->podcast->transcode()->associate($transcoded);
        
        // Notify the publisher of podcast that his podcast is ready
        $this->publisher->notify(new PodcastTranscoded($this->podcast));
    }
}

If you need to delete jobs on a model that may have been deleted, you may need to
 Settings[$deleteWhenMissingModels](https://laravel.com/docs/5.8/queues#ignoring-missing-models)Is True T Avoid dealing with things that don't exist.

Failed queue job

This is very convenient when you need control over artificially destroying logic, because using an empty "return" statement marks "Job" as successfully completed.You can force queued jobs to fail, expect exceptions, and allow handlers to retry later if possible.

This allows you to have better control over when your job fails and can also use the Fail() method, which allows you to Perform any cleaning maneuvers after failure Such as notifying the user or deleting something.

In this example, if the podcast cannot be retrieved from storage for any reason, such as when the CDN is closed, the job will fail and a custom exception will be thrown.

<?php
namespace App\Jobs;
use App\Podcast;
use Transcoder\Transcoder;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use App\Exceptions\PodcastUnretrievable;
use App\Notifications\PodcastTranscoded;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use App\Notifications\RetyingPodcastTranscode;
class TranscodePodcast
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    /**
     * Transcoder instance
     *
     * @var \App\Podcast
     */
    protected $podcast;
    /**
     * Create a new transcoded Podcast instance.
     *
     * @param \App\Podcast $podcast
     * @return void
     */
    public function __construct(Podcast $podcast)
    {
        $this->podcast = $podcast;
    }
    /**
     * Execute queue job.
     *
     * @param \Transcoder\Transcoder $podcast
     * @return void
     */
    public function handle(Transcoder $transcoder)
    {
        // If the Publisher is disabled, delete this queue job
        if ($this->podcast->publisher->isDeactivated()) {
            $this->delete();
        }
        //If podcast cannot be retrieved from the storage, we will fail.
        if ($this->podcast->fileDoesntExists()) {
            $this->fail(new PodcastUnretrievable($this->podcast));
        }
        // Tell users how many retries we have
        if ($this->attempts() > 1) {
            $this->podcast->publisher->notify(new RetryingPodcastTranscode($this->podcast, $this->attempts());
        }
        
        $transcoded = $this->transcoder->setFile($event->podcast)
                ->format('mp3')
                ->bitrate(192)
                ->start();
            
        // Associate transcoded podcast with original Podcast
        $this->podcast->transcode()->associate($transcoded);
        
        // Notify the publisher of podcast that his podcast is ready
        $this->publisher->notify(new PodcastTranscoded($this->podcast));
    }
}

Now, enter the final approach.

Release (Delay) Queue job

This may be the most useful method of trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trait trai
 Queue job Rate Limit.

In addition to rate limits, you can also use it in situations where it is not available but you want to use it in the near future without preemptive failures.

In the last example, we will delay transcoding for later use: if the transcoder is in heavy use, we will delay transcoding for five minutes until the load decreases.

<?php
namespace App\Jobs;
use App\Podcast;
use Transcoder\Transcoder;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use App\Exceptions\PodcastUnretrievable;
use App\Notifications\PodcastTranscoded;
use Illuminate\Queue\InteractsWithQueue;
use App\Notifications\TranscoderHighUsage;
use Illuminate\Foundation\Bus\Dispatchable;
use App\Notifications\RetyingPodcastTranscode;
class TranscodePodcast
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
    /**
     * Transcoder Instance
     *
     * @var \App\Podcast
     */
    protected $podcast;
    /**
     * Create a new transcoded podcast instance.
     *
     * @param \App\Podcast $podcast
     * @return void
     */
    public function __construct(Podcast $podcast)
    {
        $this->podcast = $podcast;
    }
    /**
     * Execute queue job.
     *
     * @param \Transcoder\Transcoder $podcast
     * @return void
     */
    public function handle(Transcoder $transcoder)
    {
        // If the Publisher is disabled, delete this queue job
        if ($this->podcast->publisher->isDeactivated()) {
            $this->delete();
        }
        // If podcast cannot be retrieved from the storage, we will fail.
        if ($this->podcast->fileDoesntExists()) {
            $this->fail(new PodcastUnretrievable($this->podcast));
        }
        
        // If transcoder usage is high, we will
        // t Delay transcoding for 5 minutes. Otherwise we may run the risk of delaying the transcoder process 
        // It records all transcoding subprocesses.
        if ($transcoder->getLoad()->isHigh()) {
            $delay = 60 * 5;
            $this->podcast->publisher->notify(new TranscoderHighUsage($this->podcast, $delay));
            $this->release($delay);
        }
        // Tell users how many retries we have
        if ($this->attempts() > 1) {
            $this->podcast->publisher->notify(new RetryingPodcastTranscode($this->podcast, $this->attempts());
        }
        
        $transcoded = $this->transcoder->setFile($event->podcast)
                ->format('mp3')
                ->bitrate(192)
                ->start();
            
        // Associate transcoded podcast with original Podcast
        $this->podcast->transcode()->associate($transcoded);
        
        // Notify the publisher of podcast that his podcast is ready
        $this->publisher->notify(new PodcastTranscoded($this->podcast));
    }
}

We can use special methods, such as getting some time slots assigned to the transcoder and delaying the operation if the time slots are full.
All you can do in the queue is that.Enjoy queuing.

Tags: PHP Laravel github

Posted on Thu, 07 Nov 2019 21:25:26 -0500 by stevel