Skip Menu |

This queue is for tickets about the Fuse CPAN distribution.

Report information
The Basics
Id: 33903
Status: resolved
Priority: 0/
Queue: Fuse

People
Owner: Nobody in particular
Requestors: morten.guldager [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 0.09_3



Subject: big (more than 2G) files
Date: Sat, 8 Mar 2008 08:42:59 +0100
To: bug-Fuse [...] rt.cpan.org
From: "Morten Guldager" <morten.guldager [...] gmail.com>
'Aloha! Seems that Fuse's "getattr" call can't return a file size higher than 2G (2^31) Tested with: - Fuse-0.6 and Fuse-0.8 - Kubuntu Linux, kernel 2.6.22-14-generic - perl v5.8.8 The provided loopback.pl program shows it as well as my own application. SImple demo provided here: <code> #!/usr/bin/perl use strict; use warnings; use Fuse; # unmount with: fusermount -u /tmp/fuse-test-mount Fuse::main(mountpoint => '/tmp/fuse-test-mount', getattr => \&getattr, getdir => \&getdir); sub getattr { my ($p) = @_; return qw( 0 1 16877 0 0 0 0 0 0 0 0 0 0 ) if $p eq '/'; my $size = 3000000000; # 3G return 0, 2, 33261, 0, 0, 0, 0, $size, 0, 0, 0, 0, 0; } sub getdir { return "This-is-a-big-file", 0; } </code> $ ls -l /tmp/fuse-test-mount/ total 0 -rwxr-xr-x 0 root root 18446744072414584320 1970-01-01 00:00 This-is-a-big-file -- /Morten %-)
I believe this is fixed in 0.09_2 which is on the way to CPAN. Could you please test it?
Subject: Re: [rt.cpan.org #33903] Resolved: big (more than 2G) files
Date: Sun, 16 Mar 2008 19:32:16 +0100
To: bug-Fuse [...] rt.cpan.org
From: "Morten Guldager" <morten.guldager [...] gmail.com>
On Sun, Mar 16, 2008 at 3:11 PM, Dobrica Pavlinusic via RT < bug-Fuse@rt.cpan.org> wrote: Show quoted text
> <URL: http://rt.cpan.org/Ticket/Display.html?id=33903 > > > According to our records, your request has been resolved. If you have any > further questions or concerns, please respond to this message. >
A _little_ better, but not perfect. It seems like the code is still 32 bit, but now uses an unsigned int. The maximum filesize is now 2^32-1 and not 2^31-1 as before. Change $size to 5G and try it again.. my $size = 5000000000; # 5G $ ls -l /tmp/fuse-test-mount total 0 -rwxr-xr-x 0 root root 4294967295 1970-01-01 00:00 This-is-a-big-file -- /Morten %-)
Subject: Re: "*** Possible SPAM, PLIVA IT ***" SCORE:11.3 Re: [rt.cpan.org #33903] Resolved: big (more than 2G) files
Date: Sun, 16 Mar 2008 20:40:45 +0100
To: Morten Guldager via RT <bug-Fuse [...] rt.cpan.org>
From: Dobrica Pavlinusic <dpavlin [...] rot13.org>
On Sun, Mar 16, 2008 at 02:32:42PM -0400, Morten Guldager via RT wrote: Show quoted text
> > Queue: Fuse > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=33903 > > > On Sun, Mar 16, 2008 at 3:11 PM, Dobrica Pavlinusic via RT < > bug-Fuse@rt.cpan.org> wrote: >
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=33903 > > > > > According to our records, your request has been resolved. If you have any > > further questions or concerns, please respond to this message. > >
> > A _little_ better, but not perfect. It seems like the code is still 32 bit, > but now uses an unsigned int.
Grrr... First I used unsigned long, but size_t seemed like a right thing to do... Show quoted text
> The maximum filesize is now 2^32-1 and not 2^31-1 as before. > > Change $size to 5G and try it again.. > > my $size = 5000000000; # 5G > > > $ ls -l /tmp/fuse-test-mount > total 0 > -rwxr-xr-x 0 root root 4294967295 1970-01-01 00:00 This-is-a-big-file
could you try to edit line 67 in Fuse.xs from result->st_size = (size_t)POPi; to something like: result->st_size = (unsingled long)POPi; Does this fixes problem? There must be some cross-platform way to find size of st_size, but what it is, I still don't know. -- Dobrica Pavlinusic 2share!2flame dpavlin@rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin
Subject: Re: "*** Possible SPAM, PLIVA IT ***" SCORE:11.3 Re: [rt.cpan.org #33903] Resolved: big (more than 2G) files
Date: Sun, 16 Mar 2008 20:50:53 +0100
To: bug-Fuse [...] rt.cpan.org
From: "Morten Guldager" <morten.guldager [...] gmail.com>
On Sun, Mar 16, 2008 at 8:41 PM, dpavlin@rot13.org via RT < bug-Fuse@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=33903 > > > On Sun, Mar 16, 2008 at 02:32:42PM -0400, Morten Guldager via RT wrote:
> > > > Queue: Fuse > > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=33903 > > > > > On Sun, Mar 16, 2008 at 3:11 PM, Dobrica Pavlinusic via RT < > > bug-Fuse@rt.cpan.org> wrote: > >
> > > <URL: http://rt.cpan.org/Ticket/Display.html?id=33903 > > > > > > > According to our records, your request has been resolved. If you have
> any
> > > further questions or concerns, please respond to this message. > > >
> > > > A _little_ better, but not perfect. It seems like the code is still 32
> bit,
> > but now uses an unsigned int.
> > Grrr... First I used unsigned long, but size_t seemed like a right thing > to do... >
> > The maximum filesize is now 2^32-1 and not 2^31-1 as before. > > > > Change $size to 5G and try it again.. > > > > my $size = 5000000000; # 5G > > > > > > $ ls -l /tmp/fuse-test-mount > > total 0 > > -rwxr-xr-x 0 root root 4294967295 1970-01-01 00:00 This-is-a-big-file
> > could you try to edit line 67 in Fuse.xs from > > result->st_size = (size_t)POPi; > > to something like: > > result->st_size = (unsingled long)POPi; > > Does this fixes problem? > > There must be some cross-platform way to find size of st_size, but what > it is, I still don't know. >
Nope, even if I spelled unsigned correct it did not help :-( -- /Morten %-)
Subject: Re: [rt.cpan.org #33903] Resolved: big (more than 2G) files
Date: Wed, 19 Mar 2008 20:43:54 +0100
To: Morten Guldager via RT <bug-Fuse [...] rt.cpan.org>
From: Dobrica Pavlinusic <dpavlin [...] rot13.org>
Could you please test 0.09_3 which is on the way to CPAN? Tests now include checks for files up to 64 Gb, so this problem is hopefully fixed. -- Dobrica Pavlinusic 2share!2flame dpavlin@rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin
Subject: Re: [rt.cpan.org #33903] Resolved: big (more than 2G) files
Date: Thu, 20 Mar 2008 10:46:49 +0100
To: bug-Fuse [...] rt.cpan.org
From: "Morten Guldager" <morten.guldager [...] gmail.com>
On Wed, Mar 19, 2008 at 8:44 PM, dpavlin@rot13.org via RT < bug-Fuse@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=33903 > > > Could you please test 0.09_3 which is on the way to CPAN? > > Tests now include checks for files up to 64 Gb, so this problem is > hopefully fixed.
Perfect! Just created a "virtual" 100TB file. This rocks! The package failed the "make test" stage. I ignored that and went on with the "make install". This will probably backfire soon or later Here is a trace of the first bunch of errors from "make test" mogul@linuxine:~/Fuse-0.09_3$ make test rm -f blib/arch/auto/Fuse/Fuse.so cc -shared -L/usr/local/lib -pthread -lfuse -lrt -ldl Fuse.o -lpthread -o blib/arch/auto/Fuse/Fuse.so \ \ chmod 755 blib/arch/auto/Fuse/Fuse.so PERL_DL_NONLAZY=1 /usr/bin/perl "-Iblib/lib" "-Iblib/arch" test.pl test/s/mount.....ok 1/3# mounting examples/loopback_t.pl to /tmp/fusemnt-mogul # Mounted in 0.6 secs test/s/mount.....ok test/chmod.......sh: cannot create file: Directory nonexistent test/chmod.......NOK 1/4# Failed test 'set unexecutable' # in test/chmod.t at line 7. test/chmod.......ok 2/4 # Failed test 'set executable' # in test/chmod.t at line 9. test/chmod.......NOK 3/4 test/chmod.......NOK 4/4# Failed test 'executable' # in test/chmod.t at line 10. # Looks like you failed 3 tests of 4. test/chmod.......dubious Test returned status 3 (wstat 768, 0x300) DIED. FAILED tests 1, 3-4 Failed 3/4 tests, 25.00% okay I pressed ctrl-C before the test ran to the end. After that I checked the mount point: mogul@linuxine:~/Fuse-0.09_3$ mount | grep fusemnt /dev/fuse on /tmp/fusemnt-mogul type fuse (rw,nosuid,nodev,user=mogul) mogul@linuxine:~/Fuse-0.09_3$ ls /tmp/fusemnt-mogul ls: /tmp/fusemnt-mogul: Transport endpoint is not connected -- /Morten %-)
On Ons. 19. Mar. 2008 15:44:24, dpavlin@rot13.org wrote: Show quoted text
> Could you please test 0.09_3 which is on the way to CPAN? > > Tests now include checks for files up to 64 Gb, so this problem is > hopefully fixed. >
Using Fuse.pm 0.09_3 with fuse 2.8.0-pre2 Running stock perl 5.8.8 shipped with CentOS 5.2 Linux atlas.atc.no 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:19:18 EDT 2009 i686 i686 i386 GNU/Linux All tests are successful but when trying to write beyond 2.1 Gbytes write() is called with a negative offset (-2147483648) $ make test rm -f blib/arch/auto/Fuse/Fuse.so gcc -shared -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -L/usr/local/lib -pthread -lfuse -lrt -ldl Fuse.o -lpthread -o blib/arch/auto/Fuse/Fuse.so \ \ chmod 755 blib/arch/auto/Fuse/Fuse.so PERL_DL_NONLAZY=1 /usr/bin/perl "-Iblib/lib" "-Iblib/arch" test.pl test/s/mount.....ok 1/3# mounting examples/loopback_t.pl to /tmp/fusemnt-root # Mounted in 0.7 secs test/s/mount.....ok test/chmod.......ok test/chown.......ok test/getattr.....ok test/getdir......ok test/link........ok test/mkdir.......ok test/mknod.......ok test/open........ok test/pod.........ok test/read........ok test/readlink....ok test/rename......ok test/rmdir.......ok test/statfs......ok 1/7# statfs: 1702057286, 4096, 1000000, 500000, 500000, 1000000, 500000, 0, 0, 255, 4096, 0, 0, 0, 0, 0 test/statfs......ok test/symlink.....ok test/truncate....ok test/unlink......ok test/utime.......ok test/write.......ok test/s/umount....ok All tests successful. Files=21, Tests=339, 4 wallclock secs ( 1.81 cusr + 0.71 csys = 2.52 CPU)
The following seems to do the trick, although you'll have to excuse my inexperience with 'diff'... basically, just use 'newSVnv(off)' instead of 'newSViv(off)': *** Fuse-0.09_3/Fuse.xs 2008-03-19 20:30:42.000000000 +0100 --- Fuse-0.09_3/Fuse.xs.fixed 2009-07-14 10:15:22.000000000 +0200 *************** *** 374,380 **** SAVETMPS; PUSHMARK(SP); XPUSHs(sv_2mortal(newSVpv(file,0))); ! XPUSHs(sv_2mortal(newSViv(off))); PUTBACK; rv = call_sv(_PLfuse_callbacks[12],G_SCALAR); SPAGAIN; --- 374,380 ---- SAVETMPS; PUSHMARK(SP); XPUSHs(sv_2mortal(newSVpv(file,0))); ! XPUSHs(sv_2mortal(newSVnv(off))); PUTBACK; rv = call_sv(_PLfuse_callbacks[12],G_SCALAR); SPAGAIN; *************** *** 449,455 **** PUSHMARK(SP); XPUSHs(sv_2mortal(newSVpv(file,0))); XPUSHs(sv_2mortal(newSViv(buflen))); ! XPUSHs(sv_2mortal(newSViv(off))); PUTBACK; rv = call_sv(_PLfuse_callbacks[15],G_SCALAR); SPAGAIN; --- 449,455 ---- PUSHMARK(SP); XPUSHs(sv_2mortal(newSVpv(file,0))); XPUSHs(sv_2mortal(newSViv(buflen))); ! XPUSHs(sv_2mortal(newSVnv(off))); PUTBACK; rv = call_sv(_PLfuse_callbacks[15],G_SCALAR); SPAGAIN; *************** *** 488,494 **** PUSHMARK(SP); XPUSHs(sv_2mortal(newSVpv(file,0))); XPUSHs(sv_2mortal(newSVpvn(buf,buflen))); ! XPUSHs(sv_2mortal(newSViv(off))); PUTBACK; rv = call_sv(_PLfuse_callbacks[16],G_SCALAR); SPAGAIN; --- 488,494 ---- PUSHMARK(SP); XPUSHs(sv_2mortal(newSVpv(file,0))); XPUSHs(sv_2mortal(newSVpvn(buf,buflen))); ! XPUSHs(sv_2mortal(newSVnv(off))); PUTBACK; rv = call_sv(_PLfuse_callbacks[16],G_SCALAR); SPAGAIN;