Bug #72867 for Module-Build: Modules with underscores in namespaces missing from provides section of META file

Thu Dec 01 07:37:32 2011 rrwo [...] cpan.org - Ticket created

Subject:

Modules with underscores in namespaces missing from provides section of META file

I am working on some "internal" modules for my employer, so the module namespaces have underscores, so as to differentiate them from CPAN modules that might have the same namespace, e.g. package _Company::Foo; or package App::_foo; These are valid package names, and Perl has no problems with them. (I've also been told in the past to use underscore prefixes for internal/local namespaces.) It seems that the underscore prevents Module::Build from identifying the package name. A workaround for me is to rename the package name, of course, but it would make sense to fix a bug that prevents it from recognising valid namespaces.

Thu Dec 01 08:52:12 2011 dagolden [...] cpan.org - Cc mst@shadowcat.co.uk added

Thu Dec 01 08:55:42 2011 dagolden [...] cpan.org - Correspondence added

This appears to be by design, so that "private" modules may be included in a distribution but not indexed by CPAN. Other than omitting the package from "provides" in META, are there other problems that this causes for you? -- David On Thu Dec 01 07:37:32 2011, RRWO wrote: Show quoted text

> I am working on some "internal" modules for my employer, so the module > namespaces have underscores, so as to differentiate them from CPAN > modules that might have the same namespace, e.g. > > package _Company::Foo; > > or > > package App::_foo; > > These are valid package names, and Perl has no problems with them. (I've > also been told in the past to use underscore prefixes for internal/local > namespaces.) > > It seems that the underscore prevents Module::Build from identifying the > package name. A workaround for me is to rename the package name, of > course, but it would make sense to fix a bug that prevents it from > recognising valid namespaces.

Thu Dec 01 08:55:43 2011 The RT System itself - Status changed from 'new' to 'open'

Thu Dec 01 10:58:42 2011 rrwo [...] cpan.org - Correspondence added

I am working on some utilities that process a private CPAN repository that includes private modules, and the CPAN modules that these utilities use fail because the private modules are not listed in the "provides" section. So in my case, I need the private modules to be indexed. An option in Build.PL to enable this would help, as I'd like them indexed in the private CPAN repository. On Thu Dec 01 08:55:42 2011, DAGOLDEN wrote: Show quoted text

> This appears to be by design, so that "private" modules may be included > in a distribution but not indexed by CPAN. > > Other than omitting the package from "provides" in META, are there other > problems that this causes for you? > > -- David

Thu Dec 01 12:57:17 2011 rt-cpan [...] trout.me.uk - Correspondence added

On Thu Dec 01 10:58:42 2011, RRWO wrote: Show quoted text

> > I am working on some utilities that process a private CPAN repository > that includes private modules, and the CPAN modules that these utilities > use fail because the private modules are not listed in the "provides" > section.

In the absence of a provides key, PAUSE seems to index such distributions. I suspect it's actually Module::Metadata not spotting them - could you have a look and see if you can put together a failing test (and if you're feeling really generous, a patch :) for it?

Thu Dec 01 14:52:57 2011 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #72867] Modules with underscores in namespaces missing from provides section of META file
Date:	Thu, 01 Dec 2011 11:52:47 -0800
To:	bug-Module-Build [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

On 2011.12.1 5:55 AM, David Golden via RT wrote: Show quoted text

> This appears to be by design, so that "private" modules may be included > in a distribution but not indexed by CPAN.

Do we need this heuristic? If you don't want something indexed, you can put it into no_index. If we do need this heuristic, maybe a simple switch to turn it off? -- But there's no sense crying over every mistake. You just keep on trying till you run out of cake. -- Jonathan Coulton, "Still Alive"

Thu Dec 01 15:09:01 2011 dagolden [...] cpan.org - Correspondence added

Subject:	Re: [rt.cpan.org #72867] Modules with underscores in namespaces missing from provides section of META file
Date:	Thu, 1 Dec 2011 15:08:32 -0500
To:	bug-Module-Build [...] rt.cpan.org
From:	David Golden <dagolden [...] cpan.org>

On Thu, Dec 1, 2011 at 12:57 PM, MSTROUT via RT <bug-Module-Build@rt.cpan.org> wrote: Show quoted text

> In the absence of a provides key, PAUSE seems to index such > distributions. I suspect it's actually Module::Metadata not spotting > them - could you have a look and see if you can put together a failing > test (and if you're feeling really generous, a patch :) for it?

It's not Module::Metadata. I checked. It's line 4807 in Module::Build::Base (in the repo version of the file): next if grep /^_/, split( /::/, $package ); # private package, ignore git blame says Randy Sims wrote it in 2005. I agree with Schwern that with "no_index" we might not need it anymore. -- David

Thu Dec 01 15:19:57 2011 kwilliams [...] cpan.org - Correspondence added

It seems like the real conflict here might be between 2 versions of what "private package" means. [And no, neither of them is what you're thinking.] IIUC, Robert seems to want it to mean "APIs only used within my company". But the established meaning within the tools is "stuff that's not part of any API". It's just like a "private" qualifier in Java, though less forceful. So I do think it's appropriate for anything with an underscore, or leading underscore, or whatever the convention is, not to be put into 'provides' unless the user overrides that. If you want a namespace for company stuff, just so you don't conflict with module Foo, why not just use MyCompany::Foo?

Thu Dec 01 15:42:55 2011 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #72867] Modules with underscores in namespaces missing from provides section of META file
Date:	Thu, 01 Dec 2011 12:42:42 -0800
To:	bug-Module-Build [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

On 2011.12.1 12:19 PM, Ken Williams via RT wrote: Show quoted text

> So I do think it's appropriate for anything with an underscore, or > leading underscore, or whatever the convention is, not to be put into > 'provides' unless the user overrides that.

While I do agree that the user's needs are valid, I don't agree about the need of a heuristic. First, underscores are not the convention for unindexed namespaces. The convention for "CPAN don't index this package" is... package My::Name::Here; The PAUSE indexer doesn't see that and I suspect Module::Build won't see that either. No line-by-line scanner will. Second, we don't need a heuristic. The user can declare them with no_index. http://search.cpan.org/perldoc?CPAN::Meta::Spec#no_index Heuristics only work if there's a clear convention to apply which will be obvious to all. There's no clear convention here and nothing to tell the user that a special rule was silently applied. If there's a lot of private modules to be ignored, Module::Build could accept a regex or subroutine and generate the list for the user. no_index => { directory => qr{^secret}, package => qr{_}, } -- You are wicked and wrong to have broken inside and peeked at the implementation and then relied upon it. -- tchrist in <31832.969261130@chthon>

Thu Dec 01 16:16:11 2011 kwilliams [...] cpan.org - Correspondence added

On Thu Dec 01 15:42:55 2011, schwern@pobox.com wrote: Show quoted text

> First, underscores are not the convention for unindexed namespaces. > The convention for "CPAN don't index this package" is... > > package > My::Name::Here;

Oops, I read part of this thread wrong. I thought PAUSE would also not index packages containing underscores, but it looks like it does. So the convention I thought existed doesn't actually exist. So I agree, there's no need to treat underscores any differently. -Ken

Fri Dec 02 05:44:51 2011 rrwo [...] cpan.org - Correspondence added

Thanks for looking into this. If nothing else, the behavior should be documented. Also, recommendations for "internal" module namespaces should be dicussed. I recall learning a long time ago that it was to prefix modules with "_".... I find it much more intuitive than using "MyCompany::" as a prefix (which is less useful when the company name is very long or a common word that is likely to clash with existing or future modules). --Rob On Thu Dec 01 16:16:11 2011, KWILLIAMS wrote: Show quoted text

> On Thu Dec 01 15:42:55 2011, schwern@pobox.com wrote:

> > First, underscores are not the convention for unindexed namespaces. > > The convention for "CPAN don't index this package" is... > > > > package > > My::Name::Here;

> > Oops, I read part of this thread wrong. I thought PAUSE would also not > index packages containing underscores, but it looks like it does. So > the convention I thought existed doesn't actually exist. > > So I agree, there's no need to treat underscores any differently.

Fri Dec 02 06:30:38 2011 dagolden [...] cpan.org - Severity Important changed to Normal

Fri Dec 02 08:26:21 2011 kwilliams [...] cpan.org - Correspondence added

On Fri Dec 02 05:44:51 2011, RRWO wrote: Show quoted text

> Also, recommendations for "internal" module namespaces should be > dicussed. I recall learning a long time ago that it was to prefix > modules with "_".... I find it much more intuitive than using > "MyCompany::" as a prefix (which is less useful when the company name is > very long or a common word that is likely to clash with existing or > future modules).

Those are two different things. The former is for things nobody is supposed to use. The latter is to avoid namespace clashes.