Bug #56011 for File-Map: share mappings between variables

Sat Mar 27 19:16:27 2010 user42 [...] zip.com.au - Ticket created

Subject:	share mappings between variables
Date:	Sun, 28 Mar 2010 10:16:10 +1100
To:	bug-File-Map [...] rt.cpan.org
From:	Kevin Ryde <user42 [...] zip.com.au>

As an idea for a feature, if File::Map is asked to map the same file twice it could just point the second variable at the space already mmapped for the first, releasing when both have finished. I jigged up a bit of re-use like this in my iterator thingie. The saving is address space, since each new mmap() takes a new bite out of your 2Gb or whatever allowance, which might go rather quickly with a big file used in multiple objects. The simplest case would be what I do, mapping a whole file readonly. If the inode and dev numbers are the same then re-use the map already made. Grabbing a sub-region of an existing map oughtn't be too hard. I suppose attention would have to be paid to some of the MAP_SHARED or whatever flags. A starting point could be to re-use only if the flags are the same.

Sun Mar 28 19:26:07 2010 LEONT [...] cpan.org - Taken

Mon Mar 29 05:48:37 2010 LEONT [...] cpan.org - Severity Wishlist added

Mon Mar 29 06:40:18 2010 LEONT [...] cpan.org - Correspondence added

On Sat Mar 27 19:16:27 2010, user42@zip.com.au wrote: Show quoted text

> As an idea for a feature, if File::Map is asked to map the same file > twice it could just point the second variable at the space already > mmapped for the first, releasing when both have finished. > > I jigged up a bit of re-use like this in my iterator thingie. The > saving is address space, since each new mmap() takes a new bite out of > your 2Gb or whatever allowance, which might go rather quickly with a big > file used in multiple objects. > > The simplest case would be what I do, mapping a whole file readonly. If > the inode and dev numbers are the same then re-use the map already made. > Grabbing a sub-region of an existing map oughtn't be too hard. > I suppose attention would have to be paid to some of the MAP_SHARED or > whatever flags. A starting point could be to re-use only if the flags > are the same.

I'm going to have to reject this feature request. There are too many issues involved (including portability and threading issues) to make this a good idea in the general case IMO. Having said that, I do agree there are plenty of use cases where this is useful. I encourage you to separate the logic in a module and release that to CPAN. Leon

Mon Mar 29 06:40:19 2010 The RT System itself - Status changed from 'new' to 'open'

Mon Mar 29 06:40:20 2010 LEONT [...] cpan.org - Status changed from 'open' to 'rejected'

Fri Apr 02 20:34:40 2010 user42 [...] zip.com.au - Correspondence added

Subject:	Re: [rt.cpan.org #56011] share mappings between variables
Date:	Sat, 03 Apr 2010 11:33:24 +1100
To:	bug-File-Map [...] rt.cpan.org
From:	Kevin Ryde <user42 [...] zip.com.au>

"Leon Timmermans via RT" <bug-File-Map@rt.cpan.org> writes: Show quoted text

> > portability and threading issues

I think the simplest cases may be feasible, and work up to greater sophistication. Show quoted text

> logic in a module

Of course the benefit of sharing comes when all code shares, which would mean basically nobody should use a plain mmap but instead always go through the re-use/cache/share bit. You'd even like the :mmap layer to share, so a map_handle() on something with a :mmap layer just gave back a pointer to that existing address space, if that could be arranged. I wondered if there was any sort of C library that would do some combining/tracking etc already. As far as I can tell the unix system calls can't do it themselves mainly because they only give back a pointer, not a descriptor or handle of some sort.

Fri Apr 02 20:34:41 2010 The RT System itself - Status changed from 'rejected' to 'open'

Tue Apr 20 07:35:38 2010 LEONT [...] cpan.org - Status changed from 'open' to 'rejected'