Skip Menu |

This queue is for tickets about the Fuse CPAN distribution.

Report information
The Basics
Id: 57517
Status: resolved
Priority: 0/
Queue: Fuse

People
Owner: DPAVLIN [...] cpan.org
Requestors: gerph [...] gerph.org
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 0.09_3
Fixed in: 0.09_4



Subject: Addition of file handles on open files
Hiya, Whilst trying to write a filesystem using the Fuse module, I found it surprising that there were no 'file handles' available when you opened a file. The only way you can know what file was referenced is by its name, which is not useful if your filesystem is intended to return different results for each file opened. It turned out in my case to not matter, but consider a FS which returned a different story every time you opened it. Or, more practically, a FS which returned the contents of the latest checked in file - and whilst you were operating on the file the latest checked in file changed. There are ways around this - files becoming invariant whilst it has any open instances of itself or similar - but these are not ideal. The way that Fuse appears to do this is that the filesystem updates a property ('fh') in the fuse_file_info structure on open, to contain the context for the opened file. I've put together a change which allows you to return a second parameter from open containing this value, which is then passed to all the functions which operate on open files (read, write, flush, release). Because we're retaining a reference to the SV that was returned (and is otherwise unused in the perl) we also increment the refcount on open, and decrement it on release - I think that's all that's necessary to prevent a leak, but I've never done any XS work before this so I can't be certain. I'd expect that under normal circumstances you'd open a file, set up a hashref containing properties for the file that you've opened and use 'return (0, $handle);' to return it. If you don't return a handle, the old behaviour remains - undef will be passed to the implementation in place of the handle (which the implementation wasn't expecting so won't care about). In addition, I've also added the ability to set the 'direct_io', 'keepcache' and 'nonseekable' properties by changing a hashref which is passed to the 'open' call. The archive I've attached contains the Fuse.pm and Fuse.xs files in their entirity, together with the diffs from 0.09_3. There is also a rudimentary example FS, based on the example one in the Fuse distribution, which shows the file handle to be working. It seems to be working in my more complex MythTV filesystem that I'm still trying to make useful. I couldn't actually test the nonseekable property as the fuse I have seems to be 2.4, so I don't have that property available to me here - I've just followed what the documentation says should be available. I've not added anything to the tests, because I'm not sure how to do that really, but I hope that the change to add the file handles is useful.
Subject: Fuse-filehandles.zip
Download Fuse-filehandles.zip
application/zip 13.8k

Message body not shown because it is not plain text.

committed to branch in development repository: http://github.com/dpavlin/perl-fuse/commits/rt-57517-file-handles
Thanks for patch, new version is pushed to CPAN
From: daniel.frett [...] ccci.org
On Sat May 15 16:36:47 2010, gerph wrote: Show quoted text
> Hiya, > > Whilst trying to write a filesystem using the Fuse module, I found it > surprising that there were no 'file handles' available when you opened > a file. The only way you can know what file was referenced is by its > name, which is not useful if your filesystem is intended to return > different results for each file opened. It turned out in my case to not > matter, but consider a FS which returned a different story every time > you opened it. Or, more practically, a FS which returned the contents > of the latest checked in file - and whilst you were operating on the > file the latest checked in file changed. There are ways around this - > files becoming invariant whilst it has any open instances of itself or > similar - but these are not ideal. > > The way that Fuse appears to do this is that the filesystem updates a > property ('fh') in the fuse_file_info structure on open, to contain the > context for the opened file. I've put together a change which allows > you to return a second parameter from open containing this value, which > is then passed to all the functions which operate on open files (read, > write, flush, release). Because we're retaining a reference to the SV > that was returned (and is otherwise unused in the perl) we also > increment the refcount on open, and decrement it on release - I think > that's all that's necessary to prevent a leak, but I've never done any > XS work before this so I can't be certain. > > I'd expect that under normal circumstances you'd open a file, set up a > hashref containing properties for the file that you've opened and use > 'return (0, $handle);' to return it. If you don't return a handle, the > old behaviour remains - undef will be passed to the implementation in > place of the handle (which the implementation wasn't expecting so won't > care about). > > In addition, I've also added the ability to set the 'direct_io', > 'keepcache' and 'nonseekable' properties by changing a hashref which is > passed to the 'open' call. > > The archive I've attached contains the Fuse.pm and Fuse.xs files in > their entirity, together with the diffs from 0.09_3. There is also a > rudimentary example FS, based on the example one in the Fuse > distribution, which shows the file handle to be working. It seems to be > working in my more complex MythTV filesystem that I'm still trying to > make useful. > > I couldn't actually test the nonseekable property as the fuse I have > seems to be 2.4, so I don't have that property available to me here - > I've just followed what the documentation says should be available. > > I've not added anything to the tests, because I'm not sure how to do > that really, but I hope that the change to add the file handles is > useful.
using this feature in threaded mode I get the following seg fault when triggering a release callback: panic: free from wrong pool at /usr/local/lib64/perl5/site_perl/5.10.0/x86_64-linux- thread-multi/Fuse.pm line 115, <Sock_172.21.1.234:6001> line 7. Segmentation fault
From: daniel.frett [...] ccci.org
On Thu Feb 10 15:05:53 2011, dfrett wrote: Show quoted text
> using this feature in threaded mode I get the following seg fault when > triggering a release > callback: > > panic: free from wrong pool at > /usr/local/lib64/perl5/site_perl/5.10.0/x86_64-linux- > thread-multi/Fuse.pm line 115, <Sock_172.21.1.234:6001> line 7. > Segmentation fault
just pushed a patch in github that makes the handles shared across threads when running in threaded mode. the patch is here: https://github.com/dpavlin/perl-fuse/pull/4
From: gerph [...] gerph.org
On Tue Feb 15 14:11:27 2011, dfrett wrote: Show quoted text
> On Thu Feb 10 15:05:53 2011, dfrett wrote:
> > using this feature in threaded mode I get the following seg fault
> when
> > triggering a release > > callback: > > > > panic: free from wrong pool at > > /usr/local/lib64/perl5/site_perl/5.10.0/x86_64-linux- > > thread-multi/Fuse.pm line 115, <Sock_172.21.1.234:6001> line 7. > > Segmentation fault
> > > just pushed a patch in github that makes the handles shared across > threads when running in > threaded mode. the patch is here: https://github.com/dpavlin/perl- > fuse/pull/4
Ah. I'd not tried the threaded mode because of concerns over how it handled such things - and clearly I didn't consider this when I made my patch. I'll try to give your updated version a whirl shortly.
On Tue Feb 15 17:19:23 2011, gerph wrote: Show quoted text
> Ah. I'd not tried the threaded mode because of concerns over how it > handled such things - and clearly I didn't consider this when I made my > patch. I'll try to give your updated version a whirl shortly. >
File handles are supported (as of 0.12 and later) in all functions, as well as directory handles for opendir/readdir/closedir/fsyncdir operations.
This issue should now be resolved. If you have further issues, please reopen this ticket and let us know. Thanks! -- Derrik Pates demon@now.ai