On Thu Sep 22 15:00:31 2011, mspiegelmock@gmail.com wrote:
Show quoted text> I think the problem lies in the first chunk of grab_a_job
> sub _grab_a_job {
> my ( $client, $dbh, @jobs ) = @_;
>
> ## Got some jobs! Randomize them to avoid contention between workers.
> @jobs = shuffle(@jobs);
>
> It does select by priority, but it selects 50 at a time. If the number
> of top priority jobs is < 50, it will select other jobs of a lower
> priority at the same time. Then it shuffles them, so you end up with
> jobs in a random order.
>
> On Thu, Sep 22, 2011 at 11:57 AM, J. Shirley via RT
> <bug-TheSchwartz-Moosified@rt.cpan.org> wrote:
> > <URL:
https://rt.cpan.org/Ticket/Display.html?id=71128 >
> >
> > On Wed Sep 21 15:18:02 2011, REVMISCHA wrote:
> >> Because the worker grabs a block (50) of jobs at a time and shuffles
> >> them, priority gets lost. Needs to re-sort by priority after shuffling.
> >> I've temporarily solved this problem in my workers by setting
> >> $TheSchwartz::Moosified::FIND_JOB_BATCH_SIZE = 1;
> >
> >
> > Hi,
> >
> > Sorry to be dense but I'm not exactly sure how to best reproduce this.
> > Could you provide some steps or a failing unit test and I'll get it
fixed?
Show quoted text> >
> > I see in the job finding code that it does prioritize and attempts to
> > query using that, but obviously it seems that isn't working correctly.
> >
Oh, ok, sorry - I thought you were saying that the expected behavior was
broken rather than just not being ideal.
If we resort by priority, it's going to be the same order that it was
passed into _grab_a_job and then there may be worker contention.
I think the fix may be to split them up and only randomize based on
priority (that is, don't randomize unless priority is the same).
Will that solve your issue?
