Bug #21529 for Module-Build: Add META.yml to MANIFEST in dist dir, not in root

Sat Sep 16 20:58:53 2006 rjray [...] blackperl.com - Ticket created

Subject:

Adding META.yml to MANIFEST should happing in dist dir, not in root

When "Build dist" is run, if META.yml is not already in MANIFEST, it is added. However, it is added to the "real" MANIFEST, in the project root dir, not the MANIFEST that is in the distribution dir. This can make MANIFEST appear modified to source-control systems. Compare this to the way SIGNATURE is created and added to MANIFEST when signing is enabled. It is added to the MANIFEST file after it is copied into the Name-X.XX dist dir, not to the actual MANIFEST file. If M::B must add META.yml, it should be in a way that does not interfere with source control.

Mon Oct 01 05:31:29 2007 EWILHELM [...] cpan.org - Subject changed from 'Adding META.yml to MANIFEST should happing in dist dir, not in root' to 'Add META.yml to MANIFEST in dist dir, not in root'

Mon Oct 01 05:31:37 2007 EWILHELM [...] cpan.org - Severity Important changed to Wishlist

Mon Oct 01 05:38:53 2007 EWILHELM [...] cpan.org - Correspondence added

On Sat Sep 16 20:58:53 2006, RJRAY wrote: Show quoted text

> When "Build dist" is run, if META.yml is not already in MANIFEST, > it is added. However, it is added to the "real" MANIFEST, in the > project root dir, not the MANIFEST that is in the distribution dir. > This can make MANIFEST appear modified to source-control systems.

Yes, and some users just checkin the resultant manifest. Some even checkin the META.yml. This has been downgraded to wishlist. It can be done, but the behavior has to be optional.

Mon Oct 01 05:38:57 2007 The RT System itself - Status changed from 'new' to 'open'

Mon Oct 01 16:51:05 2007 rjray [...] blackperl.com - Correspondence added

From:

rjray [...] blackperl.com

On Mon Oct 01 05:38:53 2007, EWILHELM wrote: Show quoted text

> Yes, and some users just checkin the resultant manifest. Some even > checkin the META.yml. > > This has been downgraded to wishlist. It can be done, but the > behavior has to be optional.

I disagree with the thought here. META.yml is a generated component, not an actual artifact of the project (unless you choose it to be, I suppose). I would no more want to version-control it than I would .o files (or even the .c files generated from the processing of .xs files). If the user chooses to put META.yml in their MANIFEST file, M::B already does the right thing... it *doesn't* add it a second time. I'm just suggesting that if it *does* need to add META.yml to MANIFEST, that it does so in the dist dir (which is removed during cleanup). If the user chooses to keep META.yml in MANIFEST, and version-control every iteration of META.yml, they make that choice. As it currently stands, every time I do "Build dist", I have to then "svn revert MANIFEST" afterwards.

Mon Oct 01 20:04:48 2007 mschwern [...] cpan.org - Correspondence added

On Mon Oct 01 16:51:05 2007, RJRAY wrote: Show quoted text

> On Mon Oct 01 05:38:53 2007, EWILHELM wrote:

> > Yes, and some users just checkin the resultant manifest. Some even > > checkin the META.yml. > > > > This has been downgraded to wishlist. It can be done, but the > > behavior has to be optional.

> > I disagree with the thought here. META.yml is a generated component, not > an actual artifact of the project (unless you choose it to be, I > suppose). I would no more want to version-control it than I would .o > files (or even the .c files generated from the processing of .xs files).

That's a false analogy since the .c and .o files are generated by the user and change from machine to machine. The META.yml is designed to be static. Show quoted text

> If the user chooses to put META.yml in their MANIFEST file, M::B already > does the right thing... it *doesn't* add it a second time. I'm just > suggesting that if it *does* need to add META.yml to MANIFEST, that it > does so in the dist dir (which is removed during cleanup). If the user > chooses to keep META.yml in MANIFEST, and version-control every > iteration of META.yml, they make that choice. As it currently stands, > every time I do "Build dist", I have to then "svn revert MANIFEST" > afterwards.

There's arguments on both sides for this. MakeMaker takes the "it's generated stuff" and puts META.yml and SIGNATURE into the dist dir. Others want the complete distribution checked into their repository so the full distribution can be checked in and run from the checkout. It also makes it easier to see and check the generated files before distributing. To highlight the argument, there are some C projects which don't check in their configure or Makefiles and only ship configure.in and Makefile.in. You have to first run autoconf before you can build the thing from source. This is annoying, even though, technically, it's a "generated" file. To be honest, MakeMaker puts META.yml into the dist dir to make it completely transparent in order to sneak META.yml onto everyone's CPAN modules without anyone having to lift a finger. SIGNATURE is a special case. It should go into the dist dir since it's signing a very particular snapshot of the code. If it went into the source directory it would fall out of date as soon as anything is edited. MANIFEST is also generated, but a special case. You can't build the dist dir without it. What about other generated files like README? Where do they go? I'm in the "just check it in and be done with it" camp myself. I like seeing what I ship before I ship it. Why fight the tool? Add an "svn commit META.yml -m 'META.yml update'" to your release script. If it were to be optional it should be an explicit option and not some magic based on the contents of MANIFEST.

Mon Apr 07 01:23:04 2008 rjray [...] blackperl.com - Correspondence added

Show quoted text

> SIGNATURE is a special case. It should go into the dist dir since it's > signing a very particular snapshot of the code. If it went into the > source directory it would fall out of date as soon as anything is edited.

You could say the same about META.yml. Since at the very least the "version:" key differs from one build to the next, you could say that META.yml falls out of date as soon as you're preparing your next release. Show quoted text

> MANIFEST is also generated, but a special case. You can't build the > dist dir without it. > > What about other generated files like README? Where do they go?

Depends: are they generated during the general build, or during the dist target? If the former, I would thing they would be somewhere under blib/, as that is where everything else (even scripts) ends up during that phase. If during the dist phase, I would expect it to go under the release directory. I admit that I don't rely on generated MANIFEST or README files, so I haven't run into that before. Take your example of projects that distribute Makefile.in and configure.in instead of Makefile and/or configure. We already do that-- we distribute Makefile.PL and/or Build.PL, rather than Makefile and/or Build. Anyone installing a CPAN module has to start out by creating the build-file (or let CPAN/CPANPLUS do it for them). Show quoted text

> I'm in the "just check it in and be done with it" camp myself. I like > seeing what I ship before I ship it. Why fight the tool? Add an "svn > commit META.yml -m 'META.yml update'" to your release script. > > If it were to be optional it should be an explicit option and not some > magic based on the contents of MANIFEST.

I guess it just seems to me that it already is-- if META.yml is in MANIFEST, it doesn't get added a second time. And even if I already have a pre-existing META.yml file in the project directory, "./Build dist" will delete it and re-generate it. I suppose it comes down to this: I don't have an overly-complicated release process. Mine comes down to these steps: * Make sure everything I want in the release is committed (CVS or svn) * Tag it * If there are uncommitted changes I've chosen to not deliver, use the tag to co a clean copy somewhere else. Go there and create Build from Build.PL. * "./Build dist" * "cpan-upload <distfile>" * "./Build realclean" At the end of that last step, a "svn stat" shouldn't show me any modified files (if I had uncommitted changes at the outset, then I'm in a fresh area created by co'ing a tag). If META.yml is changed, then it represents something different than is in the tagged branch of the repo tree. If I somehow generate it before "Build dist" and commit it, the dist step is still going to delete and re-generate it. It just seems to me that a sequence of "perl Build.PL; ./Build dist; ./Build realclean" should leave a working-directory in the same state as when you started (provided you've properly configured your clean-up actions with $build->add_to_cleanup()). Randy

Wed Jul 01 11:22:30 2009 dolmen [...] cpan.org - Correspondence added

Le Lun. Avr. 07 01:23:04 2008, RJRAY a écrit : Show quoted text

> I guess it just seems to me that it already is-- if META.yml is in > MANIFEST, it doesn't get added a second time. And even if I already have > a pre-existing META.yml file in the project directory, "./Build dist" > will delete it and re-generate it.

M::B knows how to generate the MANIFEST. So use this feature and ignore both MANIFEST and META.yml in your SCM. Show quoted text

> I suppose it comes down to this: I don't have an overly-complicated > release process. Mine comes down to these steps: > > * Make sure everything I want in the release is committed (CVS or svn) > * Tag it > * If there are uncommitted changes I've chosen to not deliver, use the > tag to co a clean copy somewhere else. Go there and create Build from > Build.PL. > * "./Build dist" > * "cpan-upload <distfile>" > * "./Build realclean"

Add "./Build manifest" as a first step. Add "./Build disttest" before cpan-upload. Show quoted text

> At the end of that last step, a "svn stat" shouldn't show me any > modified files (if I had uncommitted changes at the outset, then I'm in > a fresh area created by co'ing a tag). If META.yml is changed, then it > represents something different than is in the tagged branch of the repo > tree. If I somehow generate it before "Build dist" and commit it, the > dist step is still going to delete and re-generate it.

So the problem comes from your SCM usage. Use the svn:ignore property to ignore MANIFEST and META.yml from your Subversion repository: rm MANIFEST META.yml svn propedit svn:ignore . svn commit MANIFEST META.yml . Here is my own list: Makefile.PL *.bak *.old *.tmp MANIFEST META.yml _build blib cover_db Build Build.bat *.tar.gz Show quoted text

> It just seems to me that a sequence of "perl Build.PL; ./Build dist; > ./Build realclean" should leave a working-directory in the same state as > when you started (provided you've properly configured your clean-up > actions with $build->add_to_cleanup()).

I agree so I would delete MANIFEST too, but it may not be what you want.

Wed Jul 01 13:54:48 2009 rjray [...] blackperl.com - Correspondence added

I have recently started doing this (and slowly converting my older projects to this model), so this is no longer an issue for me. But my initial point still stands-- if M::B is going to change a file *that already exists* in-place, it makes more sense to do it in the dist/ directory, since the change is part of the "Build dist" command anyway. Having "Build dist" alter things outside of dist/ seems counter-intuitive. But as I said, it's not an issue for me anymore. -- """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Randy J. Ray Silicon Valley Scale Modelers: http://www.svsm.org rjray@blackperl.com randy.j.ray@gmail.com

Wed Jul 01 14:01:46 2009 dagolden [...] cpan.org - Correspondence added

Subject:	Re: [rt.cpan.org #21529] Add META.yml to MANIFEST in dist dir, not in root
Date:	Wed, 1 Jul 2009 14:01:00 -0400
To:	bug-Module-Build [...] rt.cpan.org
From:	David Golden <dagolden [...] cpan.org>

On Wed, Jul 1, 2009 at 1:54 PM, Randy J Ray via RT<bug-Module-Build@rt.cpan.org> wrote: Show quoted text

> Having "Build dist" alter things outside of dist/ seems > counter-intuitive. But as I said, it's not an issue for me anymore.

I think if I were to change this, I would add a "meta" action to complement "distmeta" and have the first act on the top directory and the second on dist. Then distdir can depend on distmeta, but people can also run the dist action directly to see what it would look like if that's how they choose to do it. I suppose it might be possible to do something similar with manifest if a MANIFEST file doesn't exist. -- David