Skip Menu |

This queue is for tickets about the File-Map CPAN distribution.

Report information
The Basics
Id: 63817
Status: rejected
Priority: 0/
Queue: File-Map

People
Owner: Nobody in particular
Requestors: user42 [...] zip.com.au
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: sensibleness of mmapping
Date: Sat, 11 Dec 2010 10:28:46 +1100
To: bug-File-Map [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
As a final idea coming out of what I've tinkered with mmapping, File::Map might offer a way for an application to check how sensible it is to mmap a given file, or demand that it only be done when sensible. So far my idea of "sensible" is * If a filehandle already has an :mmap layer then it's not sensible to mmap again, plain read should be just about as good. (Or unless in the distant future that existing mapped memory could be accessed directly.) * If the mmap would use up too much address space then it's not a good idea. This depends of course on what flavour of address space an mmap uses, but my idea was to look at the total current mmaps plus the prospective new map, and consider it not sensible to go past 1/5 of the available address space, with that space being guessed as 1/4 of the C pointer size. The net effect a modest 200Mb on a 32-bit system. The total mmapped would be across all threads, presuming all take from the same address space, and File::Map may know this if it's thread-away. The 1/5 and 1/4 fractions are rather conservative. I imagine data space is generally at least 1/2 a C pointer, and could even be almost all of it. At any rate the point would be that File::Map could provide some info or checks, better at least than what an application would know.
Interesting proposal, but it has a major problem: sensible is very situational. Different people have needs (that they may not even know themselves). Different architectures have different limitations (that may be very hard to determine). Different usage patterns have different effects (that may not be measurable). I don't think this is implementable in any generic way, so I have to reject this. Some of the functionality may make sense in a third party module though. Leon
Subject: Re: [rt.cpan.org #63817] sensibleness of mmapping
Date: Tue, 14 Dec 2010 08:41:12 +1100
To: bug-File-Map [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Leon Timmermans via RT" <bug-File-Map@rt.cpan.org> writes: Show quoted text
> > sensible is very situational.
The converse is easier to think about. Eg. it's not sensible to mmap if doing so would use up almost all the data address space. Show quoted text
> Different architectures have different limitations (that > may be very hard to determine).
Yes, which is why an mmap module can help to detect, understand, etc such things. :) If nothing else, I would like to know the total space, across all threads, currently used by File::Map maps, each rounded up to the page size or whatever unit is the real amount used up from the system. Treat that as statistics or informational, but which may also be used by applications as a building block for sensibleness. Other statistics could be just the count of mmaps, and maybe even a count of adjacent or non-adjacent maps, somehow, as a bit of representation of fragmentation.