Bug #43908 for Perl-Critic: Add MANIFEST to the repository

Fri Mar 06 13:43:26 2009 mschwern [...] cpan.org - Ticket created

Subject:

Add MANIFEST to the repository

The MANIFEST is missing from the repository. I'm sure this is deliberate so I didn't just go ahead and commit it. Normally this would be a religious argument as to whether a generated file should be committed, but it's a static generated file (ie. its the same for everyone) and there's tests that don't work without it (00_modules, 00_config and 05_utils). So from a purely practical view it would be very nice if it were there so things work straight from a checkout.

Fri Mar 06 13:46:18 2009 mschwern [...] cpan.org - Correspondence added

Case in point, I did a "make manifest" and now 00_modules fails. t/00_modules..................................2213/2081 # Looks like you planned 2081 tests but ran 2214. t/00_modules.................................. Dubious, test returned 255 (wstat 65280, 0xff00) All 2081 subtests passed Did the MANIFEST get generated wrong? Did it pull in too many files? Who knows?

Fri Mar 06 13:46:20 2009 mschwern [...] cpan.org - Status changed from 'new' to 'open'

Fri Mar 06 19:49:17 2009 perl [...] galumph.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Fri, 06 Mar 2009 18:49:02 -0600
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Elliot Shank <perl [...] galumph.com>

Michael G Schwern via RT wrote: Show quoted text

> Did the MANIFEST get generated wrong? Did it pull in too many files? > Who knows?

We support EU::MM for install, but not for development. If you use M::B, the MANIFEST will be there when needed.

Fri Mar 06 19:50:18 2009 perl [...] galumph.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Fri, 06 Mar 2009 18:50:08 -0600
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Elliot Shank <perl [...] galumph.com>

Michael G Schwern via RT wrote: Show quoted text

> The MANIFEST is missing from the repository. I'm sure this is > deliberate so I didn't just go ahead and commit it.

Yup. It's deliberate. I don't see the point of putting stuff into source control that is generated.

Sat Mar 07 05:15:50 2009 mschwern [...] cpan.org - Correspondence added

I reran with Build.PL. 00_modules still failing. Still don't know why. t/00_modules..................................2212/2081 # Looks like you planned 2081 tests but ran 2214. t/00_modules.................................. Dubious, test returned 255 (wstat 65280, 0xff00) All 2081 subtests passed The reason you don't commit generated files is because typically they change per user's environment. Things like Makefiles and config files and the like. The MANIFEST remains exactly the same from computer to computer. It's supposed to be static. That it's generated is just a convenience. Regenerating the MANIFEST as part of the build process defeats the point of the MANIFEST. The MANIFEST exists to guarantee to the user that they got the complete source distribution. If the distribution regenerates the MANIFEST based on what they downloaded and unpacked this guarantee is lost. If any files are added to the distribution directory they may clog the MANIFEST. All of which leads to a non-static MANIFEST file and I'm left with a test that fails and I don't know why. This is not worth upholding the purity of not checking in any generated files. It's just one more thing that can go wrong. I also noticed MANIFEST is deleted as part of the clean target. This is a terrible idea as it wipes out part of the source distribution. The clean target should keep the distribution intact. After a user runs build clean they will be unable to check the integrity of their distribution.

Sat Mar 07 09:09:04 2009 perl [...] galumph.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sat, 07 Mar 2009 08:08:19 -0600
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Elliot Shank <perl [...] galumph.com>

Michael G Schwern via RT wrote: Show quoted text

> Regenerating the MANIFEST as part of the build process defeats the > point of the MANIFEST. The MANIFEST exists to guarantee to the user > that they got the complete source distribution. If the distribution > regenerates the MANIFEST based on what they downloaded and unpacked > this guarantee is lost.

The MANIFEST is included in the distribution tarball. It is already present at configuration time on users' machines and therefore any warnings about missing files will correctly appear. Show quoted text

> If any files are added to the distribution directory they may clog > the MANIFEST.

Not if MANIFEST.SKIP is correct. Show quoted text

> I also noticed MANIFEST is deleted as part of the clean target. This > is a terrible idea as it wipes out part of the source distribution.

The problem is that the automatic MANIFEST generation doesn't delete files that have been removed. I got sick and tired of having to delete the damn file by hand every time I wanted the file regenerated.

Sat Mar 07 09:17:24 2009 perl [...] galumph.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sat, 07 Mar 2009 08:16:56 -0600
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Elliot Shank <perl [...] galumph.com>

Michael G Schwern via RT wrote: Show quoted text

> Queue: Perl-Critic > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=43908 > > > I reran with Build.PL. 00_modules still failing. Still don't know why. > > t/00_modules..................................2212/2081 # Looks like you > planned 2081 tests but ran 2214. > t/00_modules.................................. Dubious, test returned > 255 (wstat 65280, 0xff00) > All 2081 subtests passed

Whoops. My bad on not updating the test count for is_enabled() testing on Policies. Fixed.

Sat Mar 07 16:08:51 2009 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sat, 07 Mar 2009 13:02:17 -0800
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

Elliot Shank via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=43908 > > > Michael G Schwern via RT wrote:

>> Regenerating the MANIFEST as part of the build process defeats the >> point of the MANIFEST. The MANIFEST exists to guarantee to the user >> that they got the complete source distribution. If the distribution >> regenerates the MANIFEST based on what they downloaded and unpacked >> this guarantee is lost.

> > The MANIFEST is included in the distribution tarball. It is already present at configuration time on users' machines and therefore any warnings about missing files will correctly appear.

It will be untrusted after the first build and cannot be used to recheck the distribution (Build distcheck). Also since the tests use it it might cause tests to fail. Show quoted text

>> If any files are added to the distribution directory they may clog >> the MANIFEST.

> > Not if MANIFEST.SKIP is correct.

It can't predict all the possible junk that might wind up in there. For example, you're missing anything having to do with git. If I check out using git-svn, which I usually do, its going to wind up a mess. Which reminds me, it should be using #!include_default so it can take advantage of updates to the default skips... but it can't because its going to be regenerated on user's machines which might not have a new enough ExtUtils::Manifest to recognize #!include_default or their default MANIFEST.SKIP might be out of date. Plugging holes in the MANIFEST.SKIP is an expensive game to play to keep from checking in one file. Show quoted text

>> I also noticed MANIFEST is deleted as part of the clean target. This >> is a terrible idea as it wipes out part of the source distribution.

> > The problem is that the automatic MANIFEST generation doesn't delete files that have been removed. I got sick and tired of having to delete the damn file by hand every time I wanted the file regenerated.

I figured it was something like that. Simply customize the manifest target, and allllll this goes away. sub ACTION_manifest { my $self = shift; unlink 'MANIFEST' or die "Can't unlink MANIFEST: $!"; return $self->SUPER::ACTION_manifest(@_); } Poof! -- 87. If the thought of something makes me giggle for longer than 15 seconds, I am to assume that I am not allowed to do it. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/list/

Sun Mar 08 09:24:44 2009 perl [...] galumph.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sun, 08 Mar 2009 08:24:09 -0500
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Elliot Shank <perl [...] galumph.com>

Michael G Schwern via RT wrote: Show quoted text

> It can't predict all the possible junk that might wind up in there. > For example, you're missing anything having to do with git. If I > check out using git-svn, which I usually do, its going to wind up a > mess.

Git is evil.

Sun Mar 08 09:33:11 2009 perl [...] galumph.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sun, 08 Mar 2009 08:32:44 -0500
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Elliot Shank <perl [...] galumph.com>

Michael G Schwern via RT wrote: Show quoted text

> It will be untrusted after the first build and cannot be used to recheck the > distribution (Build distcheck).

If it gets regenerated, nothing is going to be missing, so there's no need for distcheck.

Sun Mar 08 10:00:22 2009 perl [...] galumph.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sun, 08 Mar 2009 08:59:50 -0500
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Elliot Shank <perl [...] galumph.com>

Michael G Schwern via RT wrote: Show quoted text

> Plugging holes in the MANIFEST.SKIP is an expensive game to play to > keep from checking in one file.

Not really. Show quoted text

>>> I also noticed MANIFEST is deleted as part of the clean target. >>> This is a terrible idea as it wipes out part of the source >>> distribution.

>> >> The problem is that the automatic MANIFEST generation doesn't >> delete files that have been removed. I got sick and tired of >> having to delete the damn file by hand every time I wanted the file >> regenerated.

> > I figured it was something like that.

Also, if something is being regenerated, you shouldn't keep it around.

Wed Apr 21 22:22:23 2010 ELLIOTJS [...] cpan.org - Status changed from 'open' to 'rejected'

Fri Feb 24 21:54:27 2012 mschwern [...] cpan.org - Correspondence added

I'm reopening this because I'm again working on Perl-Critic from the repository and, once again, the tests are failing because of the missing MANIFEST. Once I generate the MANIFEST tests are still failing and I can't tell if it's because the MANIFEST was generated incorrectly or if something else is wrong. Show quoted text

> Also, if something is being regenerated, you shouldn't keep it around.

That's garbled version control dogma. The actual rule is if something is being generated **differently on different machines** you shouldn't check it in. For example MakeMaker's Makefile. There's nothing wrong with checking in a generated file which is supposed to be the same for everybody (ex. MANIFEST and META.*). In fact it's beneficial. * Like any other file in the distribution, it is useful to track its contents at different points in the history for historical bug investigations. * It avoids mistakes resulting from differing build environments. Most importantly, it removes a barrier to new developers. Ease of use should trump version control dogma. As much as possible, the code should work straight out of the repository. I would request somebody other than Elliot in the Perl::Critic project respond please. His selective single line responses are dismissive and disrespectful.

Fri Feb 24 21:54:28 2012 The RT System itself - Status changed from 'rejected' to 'open'

Fri Feb 24 23:40:55 2012 jeff [...] imaginative-software.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Fri, 24 Feb 2012 20:40:43 -0800
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Jeffrey Thalhammer <jeff [...] imaginative-software.com>

On Feb 24, 2012, at 6:54 PM, Michael G Schwern via RT wrote: Show quoted text

> I would request somebody other than Elliot in the Perl::Critic project respond please.

I can see both sides of the argument. But I have been bitten by the lack of MANIFEST a few times recently (and I do use M::B rather than EU::MM, so I'm not sure why). So I'm ok with stashing the MANIFEST in the repository. Since Schwern is the only person actively working on Perl::Critic right now, I don't mind accommodating his preferences. When/if Schwern moves on and Elliot comes back in, then we can switch it back. Let's just do the transition in one commit so it is easy to revert. For argument's sake though, how is this different from using Dist::Zilla? With DZ, you don't have a MANIFEST at all until you make a build. -Jeff

Sat Feb 25 01:21:51 2012 wyant [...] cpan.org - Correspondence added

On Fri Feb 24 23:40:55 2012, jeff@imaginative-software.com wrote: Show quoted text

> > On Feb 24, 2012, at 6:54 PM, Michael G Schwern via RT wrote: >

> > I would request somebody other than Elliot in the Perl::Critic

> project respond please.

OK. Occasionally I fall over the lack of a MANIFEST, usually when adding modules. Or when I update and someone else has added a module. When that happens, I grumble, do a realclean, then run Build.PL, and ./Build manifest. So far, that has always straightened things out.

Sat Feb 25 12:17:12 2012 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sat, 25 Feb 2012 09:17:01 -0800
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

On 2012.2.24 8:40 PM, Jeffrey Thalhammer via RT wrote: Show quoted text

> I can see both sides of the argument. But I have been bitten by the lack > of MANIFEST a few times recently (and I do use M::B rather than EU::MM, > so I'm not sure why). So I'm ok with stashing the MANIFEST in the repository. > > Since Schwern is the only person actively working on Perl::Critic right > now, I don't mind accommodating his preferences. When/if Schwern moves > on and Elliot comes back in, then we can switch it back.

While I'm happy about the conclusion, I'm not so happy about the "do whatever the current developer wants" rationale. The principle driving this should be about ease of use for new and existing developers. There's three counts of missing MANIFEST frustration (you, me and Tom) with little benefit for not checking it in. Under the cockroach law that means there's probably dozens more who have been stymied and never said anything. Tailoring the dev workflow to the particular quirks of whatever single developer happens to be working on it at the moment is a great way to ensure that you only ever have a single developer working on the project. Show quoted text

> Let's just do the transition in one commit so it is easy to revert.

I figure all that needs to be done is add the MANIFEST and then delete it to revert. Is there something else? Show quoted text

> For argument's sake though, how is this different from using Dist::Zilla? > With DZ, you don't have a MANIFEST at all until you make a build.

I don't use Dist::Zilla much, so I can't really comment on that. Though I do gripe about similar issues with Dist::Zilla distributions: the normal CPAN module workflow does not work fresh out of the repository and requires special setup. What I can say for Perl::Critic is neither `Build` nor `Build test` build the MANIFEST which leaves the tests in an unrunable state. A build which leaves the tests broken would clearly be considered a failure were this argument not muddled by VCS dogma. Maybe I don't have an "author" flag set, but that's yet more complexity devs have to deal with just to avoid checking in a file. This could be minimally resolved by having the build step generate the MANIFEST if it's missing, but this would lose the benefit of having a historical log of MANIFEST changes and retain the disadvantage that the MANIFEST might be generated incorrectly by each developer. -- Hating the web since 1994.

Sat Feb 25 20:26:16 2012 wyant [...] cpan.org - Correspondence added

On Sat Feb 25 12:17:12 2012, schwern@pobox.com wrote: Show quoted text

> > The principle driving this should be about ease of use for new and > existing developers. There's three counts of missing MANIFEST > frustration (you, me and Tom) with little benefit for not checking > it in. Under the cockroach law that means there's probably dozens > more who have been stymied and never said anything.

Well, I would have called it an occasional minor inconvenience, rather than frustration. I have never been, or felt, blocked. Show quoted text

> > Tailoring the dev workflow to the particular quirks of whatever > single developer happens to be working on it at the moment is a > great way to ensure that you only ever have a single developer > working on the project.

Exactly right. My take on it is that it is Jeff's sandbox. He has allowed me to play in it. If I do not like his games and his toys, the civilized thing for me to do is to thank him kindly, and go home. Well, OK, maybe try to convince him my game or my toy is better -- but remembering that it is, after all, his sandbox, and trying to accord him the same consideration that I would like if he were playing in mine. Show quoted text

> > This could be minimally resolved by having the build step generate the > MANIFEST if it's missing, but this would lose the benefit of having a > historical log of MANIFEST changes and retain the disadvantage that > the MANIFEST might be generated incorrectly by each developer.

I don't know that I have ever felt that the MANIFEST was generated incorrectly. There have been times when something I did rendered it incorrect, but a reset (spelled "./Build realclean") generally seems to fix things up. As for MANIFEST history, it seems to me that if it's derived, the history of the MANIFEST is less interesting than the project it's derived from -- at least under SVN. Under CVS, it's a different story, because commits are per-file. Under SVN, if the history says I added or deleted a file, it will say that whether or not MANIFEST is tracked. Development requires Module::Build. I find that its documentation seems to be written for everyone but me, but withal it seems to be pretty protean. I can think of at least two alternatives to hard-coding the MANIFEST: * Override the default target (whatever that is) to make it depend on the 'makemanifest' target (whatever it's called); * Generate the MANIFEST using code in Build.PL. As for Dist::Zilla, I don't use it either. Every time I look at it, I come away with the impression of a Procrustean, all-your-distribution-are-belong-to-us behemoth that can't wait to scribble all over my source. But I know that Jeff is looking into it, and if he works out the details and gets it going to his satisfaction, I will at least try letting it scribble all over his files.

Sun Feb 26 01:16:22 2012 jeff [...] imaginative-software.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sat, 25 Feb 2012 22:16:10 -0800
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Jeffrey Thalhammer <jeff [...] imaginative-software.com>

On Feb 25, 2012, at 9:17 AM, Michael G Schwern via RT wrote: Show quoted text

> While I'm happy about the conclusion, I'm not so happy about the "do whatever > the current developer wants" rationale.

I should have said that differently. The custom handling of the MANIFEST file contradicts the workflow implied by M::B and EU::MM. Our customization is not without merit, nor are M::B and EU::MM the final word on development workflows. But on the principle of least astonishment, I declare the MANIFEST shall be in the repository. -Jeff

Sun Feb 26 02:51:26 2012 thaljef [...] cpan.org - Correspondence added

All dists now stash a their MANIFEST in the repos. This issue is now resolved. -- Jeffrey Thalhammer Imaginative Software Systems www.imaginative-software.com

Sun Feb 26 02:51:27 2012 thaljef [...] cpan.org - Status changed from 'open' to 'resolved'

Sun Feb 26 18:48:14 2012 dolmen [...] cpan.org - Correspondence added

Le 2012-02-26 08:51:26, THALJEF a écrit : Show quoted text

> All dists now stash a their MANIFEST in the repos. This issue is now > resolved.

The committed MANIFEST is incomplete. It lacks generated files: - META.yml - META.json - README - lib/Perl/Critic/PolicySummary.pod -- Olivier Mengué - http://perlresume.org/DOLMEN

Sun Feb 26 18:48:15 2012 The RT System itself - Status changed from 'resolved' to 'open'

Mon Feb 27 00:53:51 2012 jeff [...] imaginative-software.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Sun, 26 Feb 2012 21:53:25 -0800
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Jeffrey Thalhammer <jeff [...] imaginative-software.com>

On Feb 26, 2012, at 3:48 PM, Olivier Mengué via RT wrote: Show quoted text

> The committed MANIFEST is incomplete. It lacks generated files:

Ugh. This is probably why we generated the MANIFEST in the first place :-/ So I've got to put all these files in the repo too, I guess. Now I'm starting to feel like the EU::MM and M::B workflow really is just broken. With M::B anyway, it seems to view authors and users as the same. But when an author makes a build, he/she may want certain files generated, and then removed upon cleanup. But for the user, those files should already be present (e.g. MANIFEST) and not removed on cleanup. Trouble is, the targets all do the same thing, no matter who you are. I think Elliot was trying to address all this, but the limitations of M::B may have gotten in the way. This is one reason I like Dist::Zilla. It establishes a distinct authoring phase *before* building where you can automate stuff, and you don't have to expose your users to how all that happened. Maybe I'll just bite the bullet and move the project to D::Z now, rather than trying to rig up M::B in a way that makes everyone happy. -Jeff

Mon Feb 27 04:49:33 2012 dolmen [...] cpan.org - Correspondence added

RT-Send-CC:

schwern [...] pobox.com, perl [...] galumph.com, jeff [...] imaginative-software.com

Le 2012-02-27 06:53:51, jeff@imaginative-software.com a écrit : Show quoted text

> > On Feb 26, 2012, at 3:48 PM, Olivier Mengué via RT wrote: >

> > The committed MANIFEST is incomplete. It lacks generated files:

> > Ugh. This is probably why we generated the MANIFEST in the first > place :-/ > > So I've got to put all these files in the repo too, I guess. Now I'm > starting to feel like the EU::MM and M::B workflow really is just > broken. > > With M::B anyway, it seems to view authors and users as the same.

That's a great feature for open source projects. Unfortunately SubVersion does not treat authors (commiters) and users (patch writers) contrary to Git (where non-committers can easily write a patch as multiple committed step and submit that). Show quoted text

> But > when an author makes a build, he/she may want certain files > generated, and then removed upon cleanup. But for the user, those > files should already be present (e.g. MANIFEST) and not removed on > cleanup. Trouble is, the targets all do the same thing, no matter > who you are.

The M::B solution would be to override the 'test' action to run 'manifest' when missing and override 'manifest' to require 'policysummary'. Show quoted text

> This is one reason I like Dist::Zilla. It establishes a distinct > authoring phase *before* building where you can automate stuff, and > you don't have to expose your users to how all that happened. > Maybe I'll just bite the bullet and move the project to D::Z now, > rather than trying to rig up M::B in a way that makes everyone > happy.

The tricky part with D::Z is this: embedding in the distribution itself some D::Z plugins that are part of the build process (generation of the PolicySummary). The trick is to use '=' as a prefix to load plugins using absolute names. For example, use [=inc::PolicySummary] in dist.ini to load inc/PolicySummary.pm. Also, one problem that D::Z will create is that it is really slow. And re-generating "PolicySummary.pod" and MANIFEST for every "dzil test" will make that much worse. -- Olivier Mengué - http://perlresume.org/DOLMEN

Mon Feb 27 14:21:28 2012 jeff [...] imaginative-software.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Mon, 27 Feb 2012 11:21:17 -0800
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Jeffrey Thalhammer <jeff [...] imaginative-software.com>

On Feb 27, 2012, at 1:49 AM, Olivier Mengué via RT wrote: Show quoted text

> The M::B solution would be to override the 'test' action to run > 'manifest' when missing and override 'manifest' to require 'policysummary'.

I've been thinking about this all morning. I think it all boils down to whether you feel you should be able to do the default build cycle directly on the code as it comes from the repository (e.g. "perl Build.PL; ./Build test"). For simple projects you can get away with using the default build cycle, but it is only by accident. More complex projects (like Perl::Critic) need certain things to happen before you can actually make a build. As I see it, this is what the "dist" action is for. The "dist" action is where you generate all the bits that have to be present for packaging. In other words, this is when you make something that can be built. So this is when the MANIFEST, META, PolicySummary, etc. should be created. By the time you (or a user) runs the "test" action, those files must already be present. The key implication here is that authors can't really expect to use the "test" and "clean" actions on the code in the repository. Rather, you need to use "disttest" and "distclean" if you really want to run the build cycle properly. The more I think about it, the more convinced I am that this is exactly how the M::B workflow is supposed to go. It's just that I (and perhaps others) have fallen into the habit of using the "test" and "clean" actions because it saves a bit of time. But that time is a necessary step in the package/build/test cycle for Perl::Critic. For those of you who are more seasoned developers, this all may be obvious to you. But I am just now coming to this realization. Ironically, I think this leads me back to *not* stashing the MANIFEST in the repository. -Jeff

Mon Feb 27 22:46:32 2012 wyant [...] cpan.org - Correspondence added

On Mon Feb 27 14:21:28 2012, jeff@imaginative-software.com wrote: Show quoted text

> For those of you who are more seasoned developers, this all may be > obvious to you. But I am just now coming to this realization. > Ironically, I think this leads me back to *not* stashing the > MANIFEST in the repository.

Go for it. Or revert for it. Either way is good with me. However it was originally decided not to have the MANIFEST in the repository, the current situation seems to be that putting it there is not as simple as it originally appeared. FWIW, I always use 'realclean', but mostly because when I'm cleaning at all I'm starting from scratch. I do 'test', except that I always 'disttest' as part of actually making a distribution. I suspect the recent spate of releases of Net::SSLeay to be an example of what happens when you _don't_ 'disttest'. Tom

Tue Feb 28 01:29:24 2012 jeff [...] imaginative-software.com - Correspondence added

Subject:	Re: [rt.cpan.org #43908] Add MANIFEST to the repository
Date:	Mon, 27 Feb 2012 22:28:36 -0800
To:	bug-Perl-Critic [...] rt.cpan.org
From:	Jeffrey Thalhammer <jeff [...] imaginative-software.com>

On Feb 27, 2012, at 1:49 AM, Olivier Mengué via RT wrote: Show quoted text

> The tricky part with D::Z is this: embedding in the distribution itself > some D::Z plugins that are part of the build process (generation of the > PolicySummary). The trick is to use '=' as a prefix to load plugins > using absolute names. For example, use [=inc::PolicySummary] in dist.ini > to load inc/PolicySummary.pm.

I haven't adopted the habit of putting D::Z plugins in my distribution. And I doubt I ever will. This whole discussion about the MANIFEST leads me to think that the development model for M::B and EU::MM is somewhat out of date. Both of them treat the distribution as something that can be re-packaged and re-released. In the past that was usually true, and it helped the open source community build itself. But these days, modern development uses a lot of tools to generate code. So the code that is in the distribution could be a long way from what was actually authored. Unless I ship the distribution with all the authoring tools that I used, then users can't re-package the code without getting those same tools for themselves. I don't think this is evil -- it's just progress. The development process is more sophisticated and the tool chain is longer than it used to be. But I don't think I'd want to ship my dist with D::Z plugins for the same reasons that I don't ship it with make or vim. These days, patching is done against the VCS, not the distribution[1]. So I feel that aspiring to make your distribution re-packagable to end users is a noble goal, but it is increasingly impractical. In that light, I don't feel so guilty about contorting the default M::B and EU::MM workflow to suit our needs. As long as it builds "the right way" for end users, then all is well. Show quoted text

> Also, one problem that D::Z will create is that it is really slow. And > re-generating "PolicySummary.pod" and MANIFEST for every "dzil test" > will make that much worse.

The performance is unfortunate, but not so bad that I want to abandon good engineering principles. So after much though, I've concluded that I do not agree with Schwern about these so-called "static generated files". I generally do believe the in "version control dogma" that you should never commit something that you can generate. I think the real debate here is more about *who* should be generating these files and *when* they should be doing it. My career as a software engineer is relatively short, and my career in open source is even shorter. So I welcome enlightenment from those of you with more experience and insight. But at this moment, all the above best describes my point of view. [1] This is one reason I found the RequireVcsKeywords policy helpful -- so that end users know where to get the actual (human-authored) source code if they want to make a patch. But that is another story. Thanks for letting me bend your ear. -Jeff

Tue Feb 28 05:23:22 2012 dolmen [...] cpan.org - Correspondence added

RT-Send-CC:

jeff [...] imaginative-software.com

Le 2012-02-28 07:29:24, jeff@imaginative-software.com a écrit : Show quoted text

> I haven't adopted the habit of putting D::Z plugins in my > distribution. And I doubt I ever will.

This doesn't have to be a habit. But it is the right and only way to make distribution content generation as part of a D::Z build process. Bundling custom D::Z plugins in inc/ is the D::Z counterpart of M::B subclassing. Show quoted text

> This whole discussion about the MANIFEST leads me to think that the > development model for M::B and EU::MM is somewhat out of date. > Both of them treat the distribution as something that can be re- > packaged and re-released. In the past that was usually true, and > it helped the open source community build itself. > > But these days, modern development uses a lot of tools to generate > code. So the code that is in the distribution could be a long way > from what was actually authored. Unless I ship the distribution > with all the authoring tools that I used, then users can't re- > package the code without getting those same tools for themselves.

The CPAN archive is the only thing I trust because I know it will be kept forever. But I usually do not trust an author source code repository due to its relative lifetime. Just see how hundreds of Marcel Grünauer repositories disappeared one day from GitHub because he decided to quit. Or see how an author may change of VCS because he prefers it (at one point Dave Rolsky used Mercurial which almost nobody else is using in the Perl community). So I think that any CPAN distribution should be rebuildable from its own content plus the BackPAN. This is why I was describing how to do that with D::Z in a way that still makes the distribution usable and rebuildable without direct access to the source code repository. The current M::B-based process is fine too. I have no problem with it (except that extending M::B for authoring purpose makes the maintenance of M::B harder). I'm using that myself for my Number::Phone::FR distribution (which also has code generation) until the day I will take time to migrate it to D::Z. Show quoted text

> I don't think this is evil -- it's just progress. The development > process is more sophisticated and the tool chain is longer than it > used to be. But I don't think I'd want to ship my dist with D::Z > plugins for the same reasons that I don't ship it with make or vim.

The current trend is to separate the build tools from the install tool. Our install tools are now big beasts because they are also authoring tools. I have good hope for Module::Build::Tiny. I plan to work on that at the QA Hackathon. Show quoted text

> These days, patching is done against the VCS, not the distribution[1]. > So I feel that aspiring to make your distribution re-packagable to > end users is a noble goal, but it is increasingly impractical. In > that light, I don't feel so guilty about contorting the default > M::B and EU::MM workflow to suit our needs. As long as it builds > "the right way" for end users, then all is well.

You seem to separate the world in two categories: - committers - "end users": users of the CPAN distribution I think that you omit one important category: the occasionnal patch submitter like myself. (S)He's not a committer on the project. (S)He may not be familiar with the VCS you are using. Show quoted text

> So after much though, I've concluded > that I do not agree with Schwern about these so-called "static > generated files". I generally do believe the in "version control > dogma" that you should never commit something that you can > generate.

I'm also of this point of view. And especially because the generated file depends on the version of the tool that generates it. For example, META.json is not created by M::B if you have no JSON module, and so will not be in MANIFEST in that environment. Think also to differences of default end-of-line (PolicySummaryGenerator use platform EOL instead of using a fixed Unix style, so the build toolchain is not portable on Win32). So generated files in a VCS are painful when different developers have different tool versions that you can't control. Of course committing those files can also help to be aware of those differences, but often with pain. -- Olivier Mengué - http://perlresume.org/DOLMEN

Bug #43908 for Perl-Critic: Add MANIFEST to the repository

Preferred bug tracker