Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the MCE CPAN distribution.

Report information
The Basics
Id: 107384
Status: resolved
Priority: 0/
Queue: MCE

People
Owner: Nobody in particular
Requestors: kaw [...] heise.de
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 1.800



Subject: Cyclical include of MCE::Map leads to failures
Date: Mon, 28 Sep 2015 15:22:48 +0200
To: bug-MCE [...] rt.cpan.org
From: Kai Wasserbäch <kaw [...] heise.de>
Hi, I have a funny problem when using mce_map, which I believe is a bug. I have code, that fetches a set of rows from a database, using DBIx::Class. Afterwards I want to initialise an Object for each row. The code looks like this (in, say, module MyObject::List): my @aRows = $oDbh->resultset('TblName')->search({ field1 => 1 field2 => 'foo' }, { order_by => {-asc => 'date.start'}, join => 'date' }); my @aObjs = map { MyObject->new({ id => $_->tbl_name_id, data => $_ }) } @aRows; return \@aObjs MyObject uses another object (MyOtherObject::List), which in turn also relies on MCE::Map for faster object list generation. Now, as long as I have the map {} there, everything works, but as soon as I replace it with mce_map I get an error, telling me "tbl_name_id" can't be called on an undefined value (»Can't call method "tbl_name_id" on an undefined value at [...]«). Since DBIx::Class needs its Schema reference and thawing can lose that, I override thaw in MCE. I've defined a common method for this: sub _thaw { state $oDbh = My::DBIX::Schema->new; # ie. calling <http://search.cpan.org/~ribasushi/DBIx-Class-0.082820/lib/DBIx/Class/Schema.pm#thaw> return $oDbh->thaw(@_); } which is passed to MCE like use MCE::Map thaw => \&_thaw ; $oDbh is my DBIx::Class schema. The problem seems to be, that MCE::Map is included multiple times (MyObject::List (has "use MCE::Map") builds list of MyObjects. MyObject uses another object list (MyOtherObject::List), which in turn uses MCE::Map) and something gets overwritten in between those includes. If I remove the "use MyOtherObject::List" from MyObject, everything works again. I'm using Perl (and the modules) as present in Debian testing: - Perl 5.20.2 - DBIx::Class 0.082820-1 - Moo 2.000001-3 - MCE 1.608-1 Let me know, if you need something else. Cheers, Kai -- Kai Wasserbäch, Softwareentwicklung c’t – Magazin für Computertechnik Heise Medien GmbH & Co. KG Karl-Wiechert-Allee 10 30625 Hannover Tel.: +49 [0] 511 5352-338 Fax: +49 [0] 511 5352-417 E-Mail: kaw@heise.de GnuPG: 0x07C9D0B9 / 126D 0DC8 7EF2 E088 3A86 2A6C 4142 C9A8 07C9 D0B9 URL: <http://ct.de/> Registergericht: Amtsgericht Hannover HRA 26709 Persönlich haftende Gesellschafterin: Heise Medien Geschäftsführung GmbH Registergericht: Amtsgericht Hannover, HRB 60405 Geschäftsführer: Ansgar Heise, Dr. Alfons Schräder Encrypted communication (PGP/MIME) preferred.
Subject: Re: [rt.cpan.org #107384] Cyclical include of MCE::Map leads to failures
Date: Tue, 29 Sep 2015 08:23:43 -0400
To: bug-MCE [...] rt.cpan.org
From: Mario Roy <marioeroy [...] gmail.com>
Hello Kai, Your use case of MCE is interesting. Unfortunately, the MCE models are not met to be used in a circular fashion. A suggestion is to use the native map if already a worker. my @aObjs = MCE->wid ? map { MyObject->new({ id => $_->tbl_name_id, data => $_ }) } : mce_map { MyObject->new({ id => $_->tbl_name_id, data => $_ }) }; MCE::Map::finish unless MCE->wid; Kind regards, Mario
Closing this ticket as the MCE Models lack support for nested parallelism. The MCE Core API does allow nested parallelism on UNIX platforms only. More work was done in the upcoming 1.7 release with the Core API allowing nested parallelism utilizing less resources. Kind regards, Mario
Hi Kai, Moments ago, I released MCE 1.799_01 allowing cyclical include of MCE Core, MCE Models and MCE Queue. https://metacpan.org/release/MARIOROY/MCE-1.799_01 Best, Mario