Bug #17625 for SVN-Web: Revision view seems to hang if binary files are involved

Mon Feb 13 17:41:17 2006 Guest - Ticket created

Subject:

Revision view seems to hang if binary files are involved

Sorry, I'm not sure what version I have installed, but I just set it up last week, so it should be recent. I've encountered a problem viewing a particular revision number if that revision happens to involve binary files. The perl proc running SVN seems to just eat CPU and I have to kill it. All other revisions that don't involve binary files work fine. Perhaps the perl diff package is actually choking on the binary files - but probably they shouldn't even be called if binary files are part of the checkin. I would be happy with a "binary files differ" type message if a checkin involves binary files that have obviously changed (based on md5/checksum/whatever). I don't expect to actually see diffs for binary files. Great module overall. Show quoted text

---PERL VERSION--- C:\Documents and Settings\s2admin>perl -v This is perl, v5.8.7 built for MSWin32-x86-multi-thread (with 7 registered patches, see perl -V for more detail) Copyright 1987-2005, Larry Wall Binary build 813 [148120] provided by ActiveState http://www.ActiveState.com ActiveState is a division of Sophos. Built Jun 6 2005 13:36:37 Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5 source kit. Complete documentation for Perl, including FAQ lists, should be found on this system using `man perl' or `perldoc perl'. If you have access to the Internet, point your browser at http://www.perl.org/, the Perl Home Page. ---OS--- Windows 2000 Server

---Apache--- 2.0.54

Wed Feb 15 11:39:24 2006 NIKC [...] cpan.org - Correspondence added

Does the attached patch (to Revision.pm) fix the problem for you? N

=== lib/SVN/Web/Revision.pm ================================================================== --- lib/SVN/Web/Revision.pm (revision 30085) +++ lib/SVN/Web/Revision.pm (local) @@ -218,6 +218,11 @@ $kind = $root2->check_path($path); next if $kind == $SVN::Node::none; + # Skip the diff if either of the revisions have a non-text MIME type + my $mt1 = $root1->node_prop($path, 'svn:mime-type') || 'text/plain'; + my $mt2 = $root2->node_prop($path, 'svn:mime-type') || 'text/plain'; + next if ($mt1 !~ m{^text/}) or ($mt2 !~ m{^text/}); + $self->{REV}->{paths}{$path}{diff} = Text::Diff::diff ($root2->file_contents($path), $root1->file_contents($path), @@ -237,6 +242,11 @@ my $root1 = $fs->revision_root($rev); my $root2 = $fs->revision_root($self->{REV}->{paths}{$path}{copyfromrev}); + # Skip the diff if either of the files have a non-text MIME type + my $mt1 = $root1->node_prop($path, 'svn:mime-type') || 'text/plain'; + my $mt2 = $root2->node_prop($path, 'svn:mime-type') || 'text/plain'; + next if ($mt1 !~ m{^text/}) or ($mt2 !~ m{^text/}); + # If the files have differing MD5s then do a diff if($root1->file_md5_checksum($path) ne $root2->file_md5_checksum($src)) { $self->{REV}->{paths}{$path}{diff} = Text::Diff::diff

Wed Feb 15 11:39:26 2006 The RT System itself - Status changed from 'new' to 'open'

Wed Feb 15 13:32:01 2006 mikee [...] s2technologies.com - Correspondence added

Subject:	RE: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Wed, 15 Feb 2006 10:30:58 -0800
To:	<bug-SVN-Web [...] rt.cpan.org>
From:	"Mike Ellery" <mikee [...] s2technologies.com>

Yes - that seems to do it. Maybe in the next version you could just add a line like "Binary File" for each binary file that is part of the revision. It looks like I still have a few revisions, unfortunately, that cause the diff package to hang. These revisions appear to involve only text files, but they might have odd formatting or perhaps the line endings aren't consistent. If I find something more conclusive, I'll let you know - but this is more likely a problem with the diff package. The only thing SVN::Web might be able to do is protect the diff package from files that cause it harm (as you did for binary files). Thanks, Mike Ellery Show quoted text

> -----Original Message----- > From: via RT [mailto:bug-SVN-Web@rt.cpan.org] > Sent: Wednesday, February 15, 2006 8:39 AM > To: mikee@s2technologies.com > Subject: [rt.cpan.org #17625] Revision view seems to hang if > binary files are involved > > Does the attached patch (to Revision.pm) fix the problem for you? > > N >

Download smime.p7s
application/x-pkcs7-signature 3k

Message body not shown because it is not plain text.

Fri Feb 24 04:30:49 2006 NIKC [...] cpan.org - Correspondence added

On Wed Feb 15 13:32:01 2006, mikee@s2technologies.com wrote: Show quoted text

> It looks like I still have a few revisions, unfortunately, that cause the > diff package to hang. These revisions appear to involve only text files, > but they might have odd formatting or perhaps the line endings aren't > consistent. If I find something more conclusive, I'll let you know - but > this is more likely a problem with the diff package. The only thing > SVN::Web might be able to do is protect the diff package from files that > cause it harm (as you did for binary files).

If the files in these revisions aren't confidential, could you put together a test case? To do that, create a new repo, and recreate the problem in there. Then attach the gzip'd 'svnadmin dump' of that repo to this ticket, so I can reproduce the problem locally. Thanks. N

Fri Mar 10 20:14:14 2006 mikee [...] s2technologies.com - Correspondence added

Subject:	RE: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Fri, 10 Mar 2006 17:09:35 -0800
To:	<bug-SVN-Web [...] rt.cpan.org>
From:	"Mike Ellery" <mikee [...] s2technologies.com>

Download smime.p7s
application/x-pkcs7-signature 3k

Message body not shown because it is not plain text.

Sorry for the long delay. I finally got around to creating a mini test repository. I just threw a few of the files in there that seemed to cause us problems. I'm attaching a gzip of the dump of my mini repository. It just has a few of the files with one bogus change applied to them. Let me know if this helps. Thanks, Mike Show quoted text

> -----Original Message----- > From: via RT [mailto:bug-SVN-Web@rt.cpan.org] > Sent: Friday, February 24, 2006 1:31 AM > To: mikee@s2technologies.com > Subject: [rt.cpan.org #17625] Revision view seems to hang if > binary files are involved > > On Wed Feb 15 13:32:01 2006, mikee@s2technologies.com wrote:

> > It looks like I still have a few revisions, unfortunately,

> that cause the

> > diff package to hang. These revisions appear to involve

> only text files,

> > but they might have odd formatting or perhaps the line

> endings aren't

> > consistent. If I find something more conclusive, I'll let

> you know - but

> > this is more likely a problem with the diff package. The only thing > > SVN::Web might be able to do is protect the diff package

> from files that

> > cause it harm (as you did for binary files).

> > If the files in these revisions aren't confidential, could you put > together a test case? > > To do that, create a new repo, and recreate the problem in > there. Then > attach the gzip'd 'svnadmin dump' of that repo to this > ticket, so I can > reproduce the problem locally. > > Thanks. > > N >

Download foo.dump.gz
application/x-gzip 3.2m

Message body not shown because it is not plain text.

Sun Mar 12 09:07:36 2006 nik [...] ngo.org.uk - Correspondence added

Subject:	Re: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Sun, 12 Mar 2006 12:04:31 +0000
To:	bug-SVN-Web [...] rt.cpan.org
From:	Nik Clayton <nik [...] ngo.org.uk>

mikee@s2technologies.com via RT wrote: Show quoted text

> Queue: SVN-Web > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=17625 > > > > Sorry for the long delay. I finally got around to creating a mini test > repository. I just threw a few of the files in there that seemed to cause > us problems. I'm attaching a gzip of the dump of my mini repository. It > just has a few of the files with one bogus change applied to them. > > Let me know if this helps.

Thanks for that. It sort of helps, but only in as much as it shows that I can't reproduce the problem you're seeing. Specifically, I created a fresh repository and loaded your dump file in to it. With SVN::Web I can browse all three revisions (I assume it's rev 3 that's giving you trouble?), and I can go to each file and use SVN::Web::Diff to show me the differences between rev 2 and 3 of each file. No hangs. This was using bin/svnweb-server. By the time you get this message I should have set up http://jc.ngo.org.uk/svnweb/mikee/ which runs under mod_perl, and should (hopefully) prove that this isn't a SVN::Web problem. I'm guessing it's a problem with the dependencies. To try and track this down, attached is 'versions.pl'. It prints the version number of all the loaded Perl modules. If you just run it you should see something like this: % perl versions.pl Carp 1.04 Exporter 5.58 strict 1.03 warnings 1.03 If you run % perl -MSVN::Web -MSVN::Web::Revision -MSVN::Web::Diff versions.pl you'll get output similar to the attached versions.txt. Can you take a look through the output you get, a look through versions.txt, and see if there are any obvious discrepencies. In particular, Algorithm::Diff and Text::Diff are possible candidates. Thanks for your efforts so far in trying to resolve this. N

#!/usr/local/bin/perl use warnings; use strict; no strict 'refs'; foreach my $pkg_file (sort keys %INC) { my $pkg_name = $pkg_file; $pkg_name =~ s{/}{::}g; $pkg_name =~ s/.pm$//; my $ver = ${"${pkg_name}::VERSION"}; $ver = '<undef>' if ! defined $ver; printf("%-50s %s\n", $pkg_name, $ver); }

Algorithm::Diff 1.1901 AutoLoader 5.60 BSDPAN <undef> BSDPAN::Override <undef> Carp 1.04 Class::Data::Inheritable 0.04 Class::Spiffy 0.12 Config <undef> Cwd 3.15 Data::Dumper 2.121_04 Devel::StackTrace 1.12 DynaLoader 1.05 Exception::Class 1.22 Exporter 5.58 Exporter::Heavy 5.58 Fcntl 1.05 File::Basename 2.73 File::Glob 1.04 File::Path 1.07 File::Spec 3.15 File::Spec::Unix 1.5 FileHandle 2.01 FindBin 1.44 I18N::LangTags 0.35 I18N::LangTags::Detect 1.03 IO 1.21 IO::File 1.11 IO::Handle 1.24 IO::Seekable 1.09 List::Util 1.18 Locale::Maketext 1.09 Locale::Maketext::Guts <undef> Locale::Maketext::GutsLoader <undef> Locale::Maketext::Lexicon 0.49 Locale::Maketext::Lexicon::Auto 0.02 Locale::Maketext::Lexicon::Gettext 0.14 Locale::Maketext::Simple 0.12 SVN::Base <undef> SVN::Core 1.3.0 SVN::Fs <undef> SVN::Repos <undef> SVN::Web 0.45 SVN::Web::Diff <undef> SVN::Web::Revision <undef> SVN::Web::X 1.1 Scalar::Util 1.18 SelectSaver 1.01 Symbol 1.06 Template 2.14 Template::Base 2.70 Template::Config 2.68 Template::Constants 2.68 Template::Document 2.74 Template::Exception 2.65 Template::Provider 2.81 Template::Service 2.77 Text::Diff 0.35 XSLoader 0.02 YAML 0.53 YAML::Base <undef> base 2.07 bytes 1.02 constant 1.05 integer 1.00 mixin <undef> overload 1.03 re 0.04 strict 1.03 utf8 1.05 vars 1.01 warnings 1.03 warnings::register 1.00

Sun Mar 12 09:56:31 2006 mikee [...] s2technologies.com - Correspondence added

Subject:	RE: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Sun, 12 Mar 2006 06:55:57 -0800
To:	<bug-SVN-Web [...] rt.cpan.org>
From:	"Mike Ellery" <mikee [...] s2technologies.com>

Download smime.p7s
application/x-pkcs7-signature 3k

Message body not shown because it is not plain text.

I never reproduced the problem with this sample repository either - I guess I should have tried first. I did have a lower version of Algorithm::Diff. I upgraded, but it hangs on a particular revision in my main repository. Do you know of any way I could dump just the files involved in a particular revision from my repo? I tried svnadmin dump -r NUM, but it still seems to give me the whole enchilada, which is over 5 gigs. I'm attaching my versions file - there are a number of differences - let me know if you think I need to correct some/all of the differences. Looks like we probably have different operating environments too - I run Apache and perl (ActiveState) on windoze. Also, I never could get mod-perl working, so I run as CGI. I'll see if I can repro the problem with a small repository locally. If you know how I can dump a particular revision, let me know. Oh, BTW, I'd appreciate it if you could dispose of the repository that I sent you -- I don't think these particular files are very sensitive, but everything in our source base is proprietary (strictly speaking). Thanks, Mike Show quoted text

> -----Original Message----- > From: nik@ngo.org.uk via RT [mailto:bug-SVN-Web@rt.cpan.org] > Sent: Sunday, March 12, 2006 6:08 AM > To: mikee@s2technologies.com > Subject: Re: [rt.cpan.org #17625] Revision view seems to hang > if binary files are involved > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=17625 > > > mikee@s2technologies.com via RT wrote:

> > Queue: SVN-Web > > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=17625 > > > > > > > Sorry for the long delay. I finally got around to creating

> a mini test

> > repository. I just threw a few of the files in there that

> seemed to cause

> > us problems. I'm attaching a gzip of the dump of my mini

> repository. It

> > just has a few of the files with one bogus change applied to them. > > > > Let me know if this helps.

> > Thanks for that. > > It sort of helps, but only in as much as it shows that I > can't reproduce the > problem you're seeing. > > Specifically, I created a fresh repository and loaded your > dump file in to it. > > With SVN::Web I can browse all three revisions (I assume it's > rev 3 that's > giving you trouble?), and I can go to each file and use > SVN::Web::Diff to > show me the differences between rev 2 and 3 of each file. No hangs. > > This was using bin/svnweb-server. > > By the time you get this message I should have set up > > http://jc.ngo.org.uk/svnweb/mikee/ > > which runs under mod_perl, and should (hopefully) prove that > this isn't a > SVN::Web problem. I'm guessing it's a problem with the dependencies. > > To try and track this down, attached is 'versions.pl'. It prints the > version number of all the loaded Perl modules. > > If you just run it you should see something like this: > > % perl versions.pl > Carp 1.04 > Exporter 5.58 > strict 1.03 > warnings 1.03 > > If you run > > % perl -MSVN::Web -MSVN::Web::Revision -MSVN::Web::Diff > versions.pl > > you'll get output similar to the attached versions.txt. Can > you take a look > through the output you get, a look through versions.txt, and > see if there > are any obvious discrepencies. In particular, Algorithm::Diff and > Text::Diff are possible candidates. > > Thanks for your efforts so far in trying to resolve this. > > N > >

Message body is not shown because sender requested not to inline it.

Mon Mar 13 12:03:32 2006 nik [...] ngo.org.uk - Correspondence added

Subject:	Re: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Mon, 13 Mar 2006 17:02:59 +0000
To:	bug-SVN-Web [...] rt.cpan.org
From:	Nik Clayton <nik [...] ngo.org.uk>

mikee@s2technologies.com via RT wrote: Show quoted text

> I never reproduced the problem with this sample repository either - I guess > I should have tried first.

Ah. OK, that won't have told us much. Show quoted text

> I did have a lower version of Algorithm::Diff. I upgraded, but it hangs on > a particular revision in my main repository. Do you know of any way I could > dump just the files involved in a particular revision from my repo? I tried > svnadmin dump -r NUM, but it still seems to give me the whole enchilada, > which is over 5 gigs.

"svnadmin help dump" suggests that the "-r" comes *after* the repo path argument. I recommend you get the revs before and after the affected revision too. If it's rev 400 that's the problem then: svnadmin dump /path/to/repo -r 309:401 > dump should do it. Show quoted text

> I'm attaching my versions file - there are a number of differences - let me > know if you think I need to correct some/all of the differences.

You're a few revs behind on SVN::Web. I corrected some issues that led to large memory usage in 0.44, so you should update to at least that. 0.46 was released a few days ago. I'd also be interested in whether you can reproduce the problem using svnweb-server. Basically, run svnweb-server --root /path Where "/path" is the full path to the directory that contains config.yaml. Then connect to that webserver (which runs on port 8080 by default) and see if that reproduces the problem. Show quoted text

> Looks like > we probably have different operating environments too - I run Apache and > perl (ActiveState) on windoze. Also, I never could get mod-perl working, so > I run as CGI.

Perl on Windows is outside my experience I'm afraid. Still, see if the suggestions above help. Show quoted text

> Oh, BTW, I'd appreciate it if you could dispose of the repository that I > sent you -- I don't think these particular files are very sensitive, but > everything in our source base is proprietary (strictly speaking).

Done. N

Mon Mar 13 15:39:17 2006 mikee [...] s2technologies.com - Correspondence added

Subject:	RE: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Mon, 13 Mar 2006 12:35:15 -0800
To:	<bug-SVN-Web [...] rt.cpan.org>
From:	"Mike Ellery" <mikee [...] s2technologies.com>

Download smime.p7s
application/x-pkcs7-signature 3k

Message body not shown because it is not plain text.

I upgraded to the latest SVN::Web. The problem still exists and I tried running the standalone server (as you suggest) - it shows the same behavior. The particular revision that I gives me trouble is #4818 in our repository. I tried dumping 4717-4719, but it still wants to basically dump the entire state of the repository at the starting revision range (4717, in this case) -- so that results in an unwieldy multi-gigabyte file. So, the best I could do was an incremental dump of 4818. I don't know if this is useful, but I'm attaching it. I'm pretty sure the files involved are those that I copied off to the mini repository that I originally sent you, so maybe the incremental dump will give you some idea what might be the problem. There is at least one binary file involved (the CHM file), but I don't know if that matters. Thanks, Mike Show quoted text

> -----Original Message----- > From: nik@ngo.org.uk via RT [mailto:bug-SVN-Web@rt.cpan.org] > Sent: Monday, March 13, 2006 9:04 AM > To: mikee@s2technologies.com > Subject: Re: [rt.cpan.org #17625] Revision view seems to hang > if binary files are involved > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=17625 > > > mikee@s2technologies.com via RT wrote:

> > I never reproduced the problem with this sample repository

> either - I guess

> > I should have tried first.

> > Ah. OK, that won't have told us much. >

> > I did have a lower version of Algorithm::Diff. I upgraded,

> but it hangs on

> > a particular revision in my main repository. Do you know

> of any way I could

> > dump just the files involved in a particular revision from

> my repo? I tried

> > svnadmin dump -r NUM, but it still seems to give me the

> whole enchilada,

> > which is over 5 gigs.

> > "svnadmin help dump" suggests that the "-r" comes *after* the > repo path > argument. I recommend you get the revs before and after the affected > revision too. If it's rev 400 that's the problem then: > > svnadmin dump /path/to/repo -r 309:401 > dump > > should do it. >

> > I'm attaching my versions file - there are a number of

> differences - let me

> > know if you think I need to correct some/all of the differences.

> > You're a few revs behind on SVN::Web. I corrected some > issues that led > to large memory usage in 0.44, so you should update to at least that. > 0.46 was released a few days ago. > > I'd also be interested in whether you can reproduce the problem using > svnweb-server. Basically, run > > svnweb-server --root /path > > Where "/path" is the full path to the directory that contains > config.yaml. Then connect to that webserver (which runs on > port 8080 by > default) and see if that reproduces the problem. >

> > Looks like

> > we probably have different operating environments too - I

> run Apache and

> > perl (ActiveState) on windoze. Also, I never could get

> mod-perl working, so

> > I run as CGI.

> > Perl on Windows is outside my experience I'm afraid. Still, > see if the > suggestions above help. >

> > Oh, BTW, I'd appreciate it if you could dispose of the

> repository that I

> > sent you -- I don't think these particular files are very

> sensitive, but

> > everything in our source base is proprietary (strictly speaking).

> > Done. > > N > >

Download svnweb.dump.gz
application/x-gzip 3.2m

Message body not shown because it is not plain text.

Mon Mar 13 17:33:37 2006 nik [...] ngo.org.uk - Correspondence added

Subject:	Re: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Mon, 13 Mar 2006 22:33:13 +0000
To:	bug-SVN-Web [...] rt.cpan.org
From:	Nik Clayton <nik [...] ngo.org.uk>

mikee@s2technologies.com via RT wrote: Show quoted text

> I upgraded to the latest SVN::Web. The problem still exists and I tried > running the standalone server (as you suggest) - it shows the same behavior. > > The particular revision that I gives me trouble is #4818 in our repository. > I tried dumping 4717-4719, but it still wants to basically dump the entire > state of the repository at the starting revision range (4717, in this case) > -- so that results in an unwieldy multi-gigabyte file.

The incremental dump won't, but I understand why you're having a problem with the dump. You need to use "svnadmin dump /path/to/repo -r 4817:4819" to generate the dump file. Then you want to send the output through svndumpfilter to extract just the file(s) that are interesting. See "svndumpfilter help include" and "svndumpfilter help exclude" to see how to include/exclude paths from the dump. Suppose it's just the files in /some/path in the repository that demonstrate this problem. This should do the trick. svnadmin dump /path/to/repo -r 4817:1819 | svndumpfilter include /some/path > repo.dump Try that. Then create a fresh repo, and load repo.dump in to it: svnadmin create new_repo svnadmin load new_repo < repo.dump Point an SVN::Web installation at 'new_repo', and see if that has the problem too. If it does, send me repo.dump (gzip'd as necessary) and we should be able to get somewhere. N

Tue Mar 14 17:28:51 2006 mikee [...] s2technologies.com - Correspondence added

Subject:	RE: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Tue, 14 Mar 2006 14:28:23 -0800
To:	<bug-SVN-Web [...] rt.cpan.org>
From:	"Mike Ellery" <mikee [...] s2technologies.com>

OK - thanks for the instructions. I created a filtered dump, as you suggest, but I'm unable to load it - I get this message: E:\>svnadmin load e:\svn\test < webrepo.dump <<< Started new transaction, based on original revision 4817 * adding path : seaside/trunk/help ...svnadmin: File not found: transaction '0-1', path 'seaside/trunk/help' Any thoughts? You can obtain the dump here http://ftp.bluehost.com/?domain=ftp@s2technologies.com (it's too large to send as attachment). If you know how to load it, maybe it will be useful to you. However, since I haven't been able to load it, I can't confirm that the problem is reproduced by this dump. Thanks, Mike Show quoted text

> -----Original Message----- > From: nik@ngo.org.uk via RT [mailto:bug-SVN-Web@rt.cpan.org] > Sent: Monday, March 13, 2006 2:34 PM > To: mikee@s2technologies.com > Subject: Re: [rt.cpan.org #17625] Revision view seems to hang > if binary files are involved > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=17625 > > > mikee@s2technologies.com via RT wrote:

> > I upgraded to the latest SVN::Web. The problem still

> exists and I tried

> > running the standalone server (as you suggest) - it shows

> the same behavior.

> > > > The particular revision that I gives me trouble is #4818 in

> our repository.

> > I tried dumping 4717-4719, but it still wants to basically

> dump the entire

> > state of the repository at the starting revision range

> (4717, in this case)

> > -- so that results in an unwieldy multi-gigabyte file.

> > The incremental dump won't, but I understand why you're > having a problem > with the dump. > > You need to use "svnadmin dump /path/to/repo -r 4817:4819" to > generate > the dump file. > > Then you want to send the output through svndumpfilter to > extract just > the file(s) that are interesting. See "svndumpfilter help > include" and > "svndumpfilter help exclude" to see how to include/exclude paths from > the dump. > > Suppose it's just the files in /some/path in the repository that > demonstrate this problem. This should do the trick. > > svnadmin dump /path/to/repo -r 4817:1819 | > svndumpfilter include /some/path > repo.dump > > Try that. Then create a fresh repo, and load repo.dump in to it: > > svnadmin create new_repo > svnadmin load new_repo < repo.dump > > Point an SVN::Web installation at 'new_repo', and see if that has the > problem too. If it does, send me repo.dump (gzip'd as > necessary) and we > should be able to get somewhere. > > N > >

Download smime.p7s
application/x-pkcs7-signature 3k

Message body not shown because it is not plain text.

Wed Mar 15 04:41:44 2006 nik [...] ngo.org.uk - Correspondence added

Subject:	Re: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Wed, 15 Mar 2006 09:41:18 +0000
To:	bug-SVN-Web [...] rt.cpan.org
From:	Nik Clayton <nik [...] ngo.org.uk>

mikee@s2technologies.com via RT wrote: Show quoted text

> OK - thanks for the instructions. I created a filtered dump, as you > suggest, but I'm unable to load it - I get this message: > > E:\>svnadmin load e:\svn\test < webrepo.dump > <<< Started new transaction, based on original revision 4817 > * adding path : seaside/trunk/help ...svnadmin: File not found: > transaction '0-1', path 'seaside/trunk/help' > > Any thoughts?

Yeah. 'seaside/trunk' doesn't exist in your test repo yet, so it's got nowhere to load it in to. Try this (assuming e:\svn\test is the path to your repo). mkdir wc # working copy cd wc svn checkout e:\svn\test cd test # Should be created by checkout svn mkdir seaside svn mkdir seaside\trunk svn commit -m 'Create parent directories' Then try the "svnadmin load ..." command. I've done that here, and it lets me reproduce the problem. If you do that, and point SVN::Web at the repo you'll see that it's revision 3 that's got the problem. This will possibly hang your browser and web server, so don't do that on a production server. That makes sense: rev 1 created the directories. rev 2 loaded the files (so all the commits are adds) rev 3 made changes I did "svn diff -r2:3" to see what had changed, and, as well as some textual differences I was shown a large amount of binary data. The total diff is 6.9MB. Digging in to the diff, it seems to be because seaside/trunk/help/s2shelptour.chm doesn't have an svn:mime-type property set. All the other .chm files have a svn:mime-type of application/octet-stream. The revision and diff actions ignore files that have a type that doesn't match /^text\// (if they have a type). Subversion behaves similarly, documented at http://svnbook.red-bean.com/nightly/en/svn.advanced.props.html. Of course, with Subversion, if the command line client tries to do a diff of a binary file it just hangs your command line for a bit. If SVN::Web tries to do it it hangs your web server. Hmm. There's no immediate fix for this. What you can do is: svn propset svn:mime-type application/octet-stream s2shelptour.chm and commit that. It'll fix any future changes involving this file, but that's a versioned change, so it won't fix the problematic revision in your repo. I can (and will) change SVN::Web's revision and diff actions to use an additional heuristic to decide if the file is binary or not, and skip the diff if they decide that it is. That'll help you when I roll out the next release of SVN::Web, which should hopefully be in a few days time. It won't help you if you use any other tools on the repo. Going forward, you should use Subversion's automatic property setting feature to ensure the .chm files are always flagged as application/octet-stream. See this documentation for more: http://svnbook.red-bean.com/nightly/en/svn.advanced.props.html#svn.advanced.props.auto. If you wanted to fix your repo, you would need to dump your repo, edit the dump file, and then load the repo again. I would do this by: 1. Find the revision in which s2shelptour.chm was added to the repo. Call this revision A. 2. Dump all the revisions up to, but not including A. svnadmin dump -r 1:(A-1) /path/to/repo > dumpfile.1 3. Dump revision A svnadmin dump -r A /path/to/repo > dumpfile.2 4. Dump all the revisions after A svnadmin dump -r (A+1):HEAD /path/to/repo > dumpfile.3 5. Edit dumpfile.2 by hand, and add the svn:mime-type property entry. 6. Load the dumps in to a new repo, and make sure it works properly. svnadmin create new-repo svnadmin load new-repo < dumpfile.1 svnadmin load new-repo < dumpfile.2 svnadmin load new-repo < dumpfile.3 Hope that helps. N

Wed Mar 15 09:17:30 2006 mikee [...] s2technologies.com - Correspondence added

Subject:	RE: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Wed, 15 Mar 2006 06:17:01 -0800
To:	<bug-SVN-Web [...] rt.cpan.org>
From:	"Mike Ellery" <mikee [...] s2technologies.com>

Great - this has been very helpful. Thanks for all the information. -Mike Show quoted text

> -----Original Message----- > From: nik@ngo.org.uk via RT [mailto:bug-SVN-Web@rt.cpan.org] > Sent: Wednesday, March 15, 2006 1:42 AM > To: mikee@s2technologies.com > Subject: Re: [rt.cpan.org #17625] Revision view seems to hang > if binary files are involved > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=17625 > > > mikee@s2technologies.com via RT wrote:

> > OK - thanks for the instructions. I created a filtered dump, as you > > suggest, but I'm unable to load it - I get this message: > > > > E:\>svnadmin load e:\svn\test < webrepo.dump > > <<< Started new transaction, based on original revision 4817 > > * adding path : seaside/trunk/help ...svnadmin: File not found: > > transaction '0-1', path 'seaside/trunk/help' > > > > Any thoughts?

> > Yeah. 'seaside/trunk' doesn't exist in your test repo yet, > so it's got > nowhere to load it in to. Try this (assuming e:\svn\test is > the path to > your repo). > > mkdir wc # working copy > cd wc > svn checkout e:\svn\test > cd test # Should be created by checkout > svn mkdir seaside > svn mkdir seaside\trunk > svn commit -m 'Create parent directories' > > Then try the "svnadmin load ..." command. I've done that > here, and it > lets me reproduce the problem. > > If you do that, and point SVN::Web at the repo you'll see that it's > revision 3 that's got the problem. This will possibly hang > your browser > and web server, so don't do that on a production server. > > That makes sense: > > rev 1 created the directories. > > rev 2 loaded the files (so all the commits are adds) > > rev 3 made changes > > I did "svn diff -r2:3" to see what had changed, and, as well as some > textual differences I was shown a large amount of binary data. The > total diff is 6.9MB. > > Digging in to the diff, it seems to be because > seaside/trunk/help/s2shelptour.chm doesn't have an svn:mime-type > property set. All the other .chm files have a svn:mime-type of > application/octet-stream. The revision and diff actions ignore files > that have a type that doesn't match /^text\// (if they have a type). > > Subversion behaves similarly, documented at > http://svnbook.red-bean.com/nightly/en/svn.advanced.props.html. > > Of course, with Subversion, if the command line client tries to do a > diff of a binary file it just hangs your command line for a bit. If > SVN::Web tries to do it it hangs your web server. > > Hmm. > > There's no immediate fix for this. What you can do is: > > svn propset svn:mime-type application/octet-stream > s2shelptour.chm > > and commit that. It'll fix any future changes involving this > file, but > that's a versioned change, so it won't fix the problematic > revision in > your repo. > > I can (and will) change SVN::Web's revision and diff actions > to use an > additional heuristic to decide if the file is binary or not, and skip > the diff if they decide that it is. That'll help you when I roll out > the next release of SVN::Web, which should hopefully be in a > few days time. > > It won't help you if you use any other tools on the repo. > > Going forward, you should use Subversion's automatic property setting > feature to ensure the .chm files are always flagged as > application/octet-stream. See this documentation for more: > http://svnbook.red-bean.com/nightly/en/svn.advanced.props.html > #svn.advanced.props.auto. > > If you wanted to fix your repo, you would need to dump your > repo, edit > the dump file, and then load the repo again. > > I would do this by: > > 1. Find the revision in which s2shelptour.chm was added to the repo. > Call this revision A. > > 2. Dump all the revisions up to, but not including A. > > svnadmin dump -r 1:(A-1) /path/to/repo > dumpfile.1 > > 3. Dump revision A > > svnadmin dump -r A /path/to/repo > dumpfile.2 > > 4. Dump all the revisions after A > > svnadmin dump -r (A+1):HEAD /path/to/repo > dumpfile.3 > > 5. Edit dumpfile.2 by hand, and add the svn:mime-type property entry. > > 6. Load the dumps in to a new repo, and make sure it works properly. > > svnadmin create new-repo > svnadmin load new-repo < dumpfile.1 > svnadmin load new-repo < dumpfile.2 > svnadmin load new-repo < dumpfile.3 > > Hope that helps. > > N > >

Download smime.p7s
application/x-pkcs7-signature 3k

Message body not shown because it is not plain text.

Mon Jun 19 06:30:02 2006 Guest - Correspondence added

On Wed Mar 15 04:41:44 2006, nik@ngo.org.uk wrote: Show quoted text

> mikee@s2technologies.com via RT wrote:

> > OK - thanks for the instructions. I created a filtered dump, as

you Show quoted text

> > suggest, but I'm unable to load it - I get this message: > > > > E:\>svnadmin load e:\svn\test < webrepo.dump > > <<< Started new transaction, based on original revision 4817 > > * adding path : seaside/trunk/help ...svnadmin: File not

found: Show quoted text

> > transaction '0-1', path 'seaside/trunk/help' > > > > Any thoughts?

> > Yeah. 'seaside/trunk' doesn't exist in your test repo yet, so it's > got > nowhere to load it in to. Try this (assuming e:\svn\test is the path > to > your repo). > > mkdir wc # working copy > cd wc > svn checkout e:\svn\test > cd test # Should be created by checkout > svn mkdir seaside > svn mkdir seaside\trunk > svn commit -m 'Create parent directories' > > Then try the "svnadmin load ..." command. I've done that here, and

it Show quoted text

> lets me reproduce the problem. > > If you do that, and point SVN::Web at the repo you'll see that it's > revision 3 that's got the problem. This will possibly hang your > browser > and web server, so don't do that on a production server. > > That makes sense: > > rev 1 created the directories. > > rev 2 loaded the files (so all the commits are adds) > > rev 3 made changes > > I did "svn diff -r2:3" to see what had changed, and, as well as some > textual differences I was shown a large amount of binary data. The > total diff is 6.9MB. > > Digging in to the diff, it seems to be because > seaside/trunk/help/s2shelptour.chm doesn't have an svn:mime-type > property set. All the other .chm files have a svn:mime-type of > application/octet-stream. The revision and diff actions ignore files > that have a type that doesn't match /^text\// (if they have a type). > > Subversion behaves similarly, documented at > http://svnbook.red-bean.com/nightly/en/svn.advanced.props.html. > > Of course, with Subversion, if the command line client tries to do a > diff of a binary file it just hangs your command line for a bit. If > SVN::Web tries to do it it hangs your web server. > > Hmm. > > There's no immediate fix for this. What you can do is: > > svn propset svn:mime-type application/octet-stream > s2shelptour.chm > > and commit that. It'll fix any future changes involving this file, > but > that's a versioned change, so it won't fix the problematic revision

in Show quoted text

> your repo. > > I can (and will) change SVN::Web's revision and diff actions to use

an Show quoted text

> additional heuristic to decide if the file is binary or not, and skip > the diff if they decide that it is. That'll help you when I roll out > the next release of SVN::Web, which should hopefully be in a few days > time. > > It won't help you if you use any other tools on the repo. > > Going forward, you should use Subversion's automatic property setting > feature to ensure the .chm files are always flagged as > application/octet-stream. See this documentation for more: > http://svnbook.red- > bean.com/nightly/en/svn.advanced.props.html#svn.advanced.props.auto. > > If you wanted to fix your repo, you would need to dump your repo,

edit Show quoted text

> the dump file, and then load the repo again. > > I would do this by: > > 1. Find the revision in which s2shelptour.chm was added to the repo. > Call this revision A. > > 2. Dump all the revisions up to, but not including A. > > svnadmin dump -r 1:(A-1) /path/to/repo > dumpfile.1 > > 3. Dump revision A > > svnadmin dump -r A /path/to/repo > dumpfile.2 > > 4. Dump all the revisions after A > > svnadmin dump -r (A+1):HEAD /path/to/repo > dumpfile.3 > > 5. Edit dumpfile.2 by hand, and add the svn:mime-type property

entry. Show quoted text

> > 6. Load the dumps in to a new repo, and make sure it works properly. > > svnadmin create new-repo > svnadmin load new-repo < dumpfile.1 > svnadmin load new-repo < dumpfile.2 > svnadmin load new-repo < dumpfile.3 > > Hope that helps. > > N

Thu Nov 09 08:17:18 2006 NIKC [...] cpan.org - Correspondence added

Mike, Did this discussion resolve everything for you? I'd like to close out the ticket, but I want to make sure everything's fine first. N

Thu Nov 09 11:51:44 2006 mikee [...] s2technologies.com - Correspondence added

Subject:	Re: [rt.cpan.org #17625] Revision view seems to hang if binary files are involved
Date:	Thu, 09 Nov 2006 08:50:34 -0800
To:	bug-SVN-Web [...] rt.cpan.org
From:	Mike Ellery <mikee [...] s2technologies.com>

I believe it is resolved. Thanks, Mike via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=17625 > > > Mike, > > Did this discussion resolve everything for you? I'd like to close out > the ticket, but I want to make sure everything's fine first. > > N

Sun Apr 29 14:40:35 2007 NIKC [...] cpan.org - Status changed from 'open' to 'resolved'