Bug #92783 for Net-Domain-TLD: support for new GTLDs

Thu Feb 06 10:15:45 2014 BARBIE [...] cpan.org - Ticket created

Subject:

support for new GTLDs

ICANN has now given approval for over 1400 new GTLDs, some of which were launched yesterday, and the rest being launched throughout the year. Any idea when support for all these will be added to this module?

Thu Feb 06 12:35:10 2014 alexp [...] cpan.org - Correspondence added

should be added in next release, hopefully very soon, thank you On Thu Feb 06 10:15:45 2014, BARBIE wrote: Show quoted text

> ICANN has now given approval for over 1400 new GTLDs, some of which > were launched yesterday, and the rest being launched throughout the > year. > > Any idea when support for all these will be added to this module?

Thu Feb 06 12:35:10 2014 The RT System itself - Status changed from 'new' to 'open'

Fri Feb 07 03:17:10 2014 BARBIE [...] cpan.org - Correspondence added

On Thu Feb 06 12:35:10 2014, ALEXP wrote: Show quoted text

> should be added in next release, hopefully very soon, thank you

Thanks Alex, very much appreciated.

Fri Feb 07 15:37:13 2014 rjbs [...] cpan.org - Correspondence added

I sorted this problem at work by parsing the root zone file with this code: my %tld; while (<>) { my ($domain) = split /\t/; next if $domain =~ tr/././ > 1; $domain =~ s/\.\z//; $tld{ lc $domain} = 1; } print "$_\n" for sort keys %tld; I wanted to check whether this would miss any data that are found in Net::Domain::TLD, so I added this: use Net::Domain::TLD 'tlds'; use Encode qw(from_to); use Encode::Punycode; sub dom { from_to($_, 'Punycode', 'utf-8') if s/^xn--//; return $_; } warn "missing " . dom . "\n" for grep { ! $tld{$_} } tlds; What I found was interesting: missing பரிட்சை missing বাংলা missing გე missing إختبار missing δοκιμή missing سودان missing טעסט missing परीक्षा missing 测试 missing испытание missing テスト missing آزمایشی missing 測試 missing پاکستان missing 테스트 missing um missing bq missing mf missing eh missing yu missing ss missing bl Net::Domain::TLD includes the "test" domains (see https://en.wikipedia.org/wiki/List_of_Internet_top-level_domains#Test_TLDs ) and a number of domains that are reserved but not operable. This is not a problem, necessarily, but I thought it worth noting as it may not be intended. -- rjbs

Thu Mar 20 22:30:54 2014 OSCHWALD [...] cpan.org - Correspondence added

Is there a timeline on the next release? This recently started causing problems for my employer.

Thu Mar 20 22:59:09 2014 alexp [...] cpan.org - Correspondence added

On Fri Feb 07 15:37:13 2014, RJBS wrote: Show quoted text

> I sorted this problem at work by parsing the root zone file with this > code: > > my %tld; > while (<>) { > my ($domain) = split /\t/; > next if $domain =~ tr/././ > 1; > $domain =~ s/\.\z//; > $tld{ lc $domain} = 1; > } > > print "$_\n" for sort keys %tld; > > I wanted to check whether this would miss any data that are found in > Net::Domain::TLD, so I added this: > > use Net::Domain::TLD 'tlds'; > use Encode qw(from_to); > use Encode::Punycode; > > sub dom { > from_to($_, 'Punycode', 'utf-8') if s/^xn--//; > return $_; > } > > warn "missing " . dom . "\n" for grep { ! $tld{$_} } tlds; > > What I found was interesting: > > missing பரிட்சை > missing বাংলা > missing გე > missing إختبار > missing δοκιμή > missing سودان > missing טעסט > missing परीक्षा > missing 测试 > missing испытание > missing テスト > missing آزمایشی > missing 測試 > missing پاکستان > missing 테스트 > missing um > missing bq > missing mf > missing eh > missing yu > missing ss > missing bl > > Net::Domain::TLD includes the "test" domains (see > https://en.wikipedia.org/wiki/List_of_Internet_top- > level_domains#Test_TLDs ) and a number of domains that are reserved > but not operable. > > This is not a problem, necessarily, but I thought it worth noting as > it may not be intended.

Someone, some time ago requested that test/reserved entries be added as well If memory serves correctly.

Thu Mar 20 23:02:13 2014 alexp [...] cpan.org - Correspondence added

On Thu Mar 20 22:59:09 2014, ALEXP wrote: Show quoted text

> On Fri Feb 07 15:37:13 2014, RJBS wrote:

> > I sorted this problem at work by parsing the root zone file with this > > code: > > > > my %tld; > > while (<>) { > > my ($domain) = split /\t/; > > next if $domain =~ tr/././ > 1; > > $domain =~ s/\.\z//; > > $tld{ lc $domain} = 1; > > } > > > > print "$_\n" for sort keys %tld; > > > > I wanted to check whether this would miss any data that are found in > > Net::Domain::TLD, so I added this: > > > > use Net::Domain::TLD 'tlds'; > > use Encode qw(from_to); > > use Encode::Punycode; > > > > sub dom { > > from_to($_, 'Punycode', 'utf-8') if s/^xn--//; > > return $_; > > } > > > > warn "missing " . dom . "\n" for grep { ! $tld{$_} } tlds; > > > > What I found was interesting: > > > > missing பரிட்சை > > missing বাংলা > > missing გე > > missing إختبار > > missing δοκιμή > > missing سودان > > missing טעסט > > missing परीक्षा > > missing 测试 > > missing испытание > > missing テスト > > missing آزمایشی > > missing 測試 > > missing پاکستان > > missing 테스트 > > missing um > > missing bq > > missing mf > > missing eh > > missing yu > > missing ss > > missing bl > > > > Net::Domain::TLD includes the "test" domains (see > > https://en.wikipedia.org/wiki/List_of_Internet_top- > > level_domains#Test_TLDs ) and a number of domains that are reserved > > but not operable. > > > > This is not a problem, necessarily, but I thought it worth noting as > > it may not be intended.

> > Someone, some time ago requested that test/reserved entries be added > as well If memory serves correctly.

Will work on adding these in next day or so, sorry been quite busy lately.

Thu Mar 20 23:38:14 2014 alexp [...] cpan.org - Correspondence added

On Thu Mar 20 23:02:13 2014, ALEXP wrote: Show quoted text

> On Thu Mar 20 22:59:09 2014, ALEXP wrote:

> > On Fri Feb 07 15:37:13 2014, RJBS wrote:

> > > I sorted this problem at work by parsing the root zone file with this > > > code: > > > > > > my %tld; > > > while (<>) { > > > my ($domain) = split /\t/; > > > next if $domain =~ tr/././ > 1; > > > $domain =~ s/\.\z//; > > > $tld{ lc $domain} = 1; > > > } > > > > > > print "$_\n" for sort keys %tld; > > > > > > I wanted to check whether this would miss any data that are found in > > > Net::Domain::TLD, so I added this: > > > > > > use Net::Domain::TLD 'tlds'; > > > use Encode qw(from_to); > > > use Encode::Punycode; > > > > > > sub dom { > > > from_to($_, 'Punycode', 'utf-8') if s/^xn--//; > > > return $_; > > > } > > > > > > warn "missing " . dom . "\n" for grep { ! $tld{$_} } tlds; > > > > > > What I found was interesting: > > > > > > missing பரிட்சை > > > missing বাংলা > > > missing გე > > > missing إختبار > > > missing δοκιμή > > > missing سودان > > > missing טעסט > > > missing परीक्षा > > > missing 测试 > > > missing испытание > > > missing テスト > > > missing آزمایشی > > > missing 測試 > > > missing پاکستان > > > missing 테스트 > > > missing um > > > missing bq > > > missing mf > > > missing eh > > > missing yu > > > missing ss > > > missing bl > > > > > > Net::Domain::TLD includes the "test" domains (see > > > https://en.wikipedia.org/wiki/List_of_Internet_top- > > > level_domains#Test_TLDs ) and a number of domains that are reserved > > > but not operable. > > > > > > This is not a problem, necessarily, but I thought it worth noting as > > > it may not be intended.

> > > > Someone, some time ago requested that test/reserved entries be added > > as well If memory serves correctly.

> > Will work on adding these in next day or so, sorry been quite busy lately. >

Ok 1.71 release uploaded to PAUSE containing support for 176 new gTLDs. These can also be extracted via gtld_new category. gTLDs are always parsed from IANA source below as it should be authorative. https://www.iana.org/domains/root/db

Thu Mar 20 23:38:14 2014 alexp [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Mar 20 23:58:35 2014 alexp [...] cpan.org - Correspondence added

On Thu Mar 20 23:38:14 2014, ALEXP wrote: Show quoted text

> On Thu Mar 20 23:02:13 2014, ALEXP wrote:

> > On Thu Mar 20 22:59:09 2014, ALEXP wrote:

> > > On Fri Feb 07 15:37:13 2014, RJBS wrote:

> > > > I sorted this problem at work by parsing the root zone file with > > > > this > > > > code: > > > > > > > > my %tld; > > > > while (<>) { > > > > my ($domain) = split /\t/; > > > > next if $domain =~ tr/././ > 1; > > > > $domain =~ s/\.\z//; > > > > $tld{ lc $domain} = 1; > > > > } > > > > > > > > print "$_\n" for sort keys %tld; > > > > > > > > I wanted to check whether this would miss any data that are found > > > > in > > > > Net::Domain::TLD, so I added this: > > > > > > > > use Net::Domain::TLD 'tlds'; > > > > use Encode qw(from_to); > > > > use Encode::Punycode; > > > > > > > > sub dom { > > > > from_to($_, 'Punycode', 'utf-8') if s/^xn--//; > > > > return $_; > > > > } > > > > > > > > warn "missing " . dom . "\n" for grep { ! $tld{$_} } tlds; > > > > > > > > What I found was interesting: > > > > > > > > missing பரிட்சை > > > > missing বাংলা > > > > missing გე > > > > missing إختبار > > > > missing δοκιμή > > > > missing سودان > > > > missing טעסט > > > > missing परीक्षा > > > > missing 测试 > > > > missing испытание > > > > missing テスト > > > > missing آزمایشی > > > > missing 測試 > > > > missing پاکستان > > > > missing 테스트 > > > > missing um > > > > missing bq > > > > missing mf > > > > missing eh > > > > missing yu > > > > missing ss > > > > missing bl > > > > > > > > Net::Domain::TLD includes the "test" domains (see > > > > https://en.wikipedia.org/wiki/List_of_Internet_top- > > > > level_domains#Test_TLDs ) and a number of domains that are > > > > reserved > > > > but not operable. > > > > > > > > This is not a problem, necessarily, but I thought it worth noting > > > > as > > > > it may not be intended.

> > > > > > Someone, some time ago requested that test/reserved entries be > > > added > > > as well If memory serves correctly.

> > > > Will work on adding these in next day or so, sorry been quite busy > > lately. > >

> > Ok 1.71 release uploaded to PAUSE containing support for 176 new > gTLDs. These can also be extracted via gtld_new category. > > gTLDs are always parsed from IANA source below as it should be > authorative. > > https://www.iana.org/domains/root/db

1.72 release uploaded as well containing support for "restricted" RFC2606 names

Fri Mar 21 03:29:29 2014 BARBIE [...] cpan.org - Correspondence added

Over the weekend I will get you a list of all the nGTLDs and their release dates (as far as I know).

Fri Mar 21 10:00:55 2014 OSCHWALD [...] cpan.org - Correspondence added

Thanks!

Fri Mar 21 11:04:19 2014 alexp [...] cpan.org - Correspondence added

On Fri Mar 21 03:29:29 2014, BARBIE wrote: Show quoted text

> Over the weekend I will get you a list of all the nGTLDs and their > release dates (as far as I know).

Thanks, but I would prefer to wait until these are officially released before adding them in. Once they are I will re-parse the IANA rootdb file. In the future this process may be automated.

Fri Mar 21 11:24:16 2014 barbie [...] missbarbell.co.uk - Correspondence added

CC:	BARBIE [...] cpan.org
Subject:	Re: [rt.cpan.org #92783] support for new GTLDs
Date:	Fri, 21 Mar 2014 15:24:04 +0000
To:	bug-Net-Domain-TLD [...] rt.cpan.org
From:	Barbie <barbie [...] missbarbell.co.uk>

Hi Alex, They're being added daily. There were around 10 releases this week, and roughly the same over the next week. They're being released a few at a time, which will mean your module is continually out of date, unless you plan to release on a daily basis. If you want to release via automated tools, the list I can provide might help, but it does then mean users are going to have continually update daily too, which is not likely to happen. Unless you allow them all, then you will likely find your userbase diminish, as they find alternative solutions. Thanks, Barbie. -- Birmingham.pm - http://birmingham.pm.org CPAN Testers - http://cpantesters.org YAPC Surveys - http://yapc-surveys.org Perl Jam - http://perljam.info On Fri, Mar 21, 2014 at 3:04 PM, Alex Pavlovic via RT < bug-Net-Domain-TLD@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=92783 > > > On Fri Mar 21 03:29:29 2014, BARBIE wrote:

> > Over the weekend I will get you a list of all the nGTLDs and their > > release dates (as far as I know).

> > Thanks, but I would prefer to wait until these are officially released > before adding them in. > > Once they are I will re-parse the IANA rootdb file. > > In the future this process may be automated. > > >

Fri Mar 21 12:00:52 2014 alexp [...] cpan.org - Correspondence added

Hi Barbie I understand the entries are being released every day. The idea was to have module do a pull on IANA root db file when module is being used and cache the information. I mayadd this in next release as structure also needs to change. We can add these entries now the only concern was that some of them may not make it in as they have not been finalized and approved. Then we would need to resync again. Is there a chance of this happening?

Fri Mar 21 15:35:36 2014 BARBIE [...] cpan.org - Correspondence added

Hi Alex, Ah I see what you mean, yes that would be nice, then you can update locally periodically when things settle. As far as I know there are 1400+ extensions that have been agreed and signed off, and will be released throughout this year. Many have scheduled release dates, due to the pre-planning for Sunrise/Landrush etc. I've attached the ones I know about, which should give you at least 4 months worth. If you want the full 1400+ approved extensions I can get you those too. Thanks, Barbie

Subject:

gtld_launch_dates.csv

Extension,Release Date BIKE,05-Feb-14 CLOTHING,05-Feb-14 GURU,05-Feb-14 HOLDINGS,05-Feb-14 PLUMBING,05-Feb-14 SINGLES,05-Feb-14 VENTURES,05-Feb-14 CAMERA,12-Feb-14 EQUIPMENT,12-Feb-14 ESTATE,12-Feb-14 GALLERY,12-Feb-14 GRAPHICS,12-Feb-14 LIGHTING,12-Feb-14 PHOTOGRAPHY,12-Feb-14 CONSTRUCTION,19-Feb-14 CONTRACTORS,19-Feb-14 DIRECTORY,19-Feb-14 KITCHEN,19-Feb-14 LAND,19-Feb-14 TECHNOLOGY,19-Feb-14 TODAY,19-Feb-14 SEXY,25-Feb-14 TATTOO,25-Feb-14 DIAMONDS,26-Feb-14 ENTERPRISES,26-Feb-14 TIPS,26-Feb-14 VOYAGE,26-Feb-14 CAREERS,05-Mar-14 PHOTOS,05-Mar-14 RECIPES,05-Mar-14 SHOES,05-Mar-14 CAB,12-Mar-14 COMPANY,19-Mar-14 DOMAINS,12-Mar-14 LIMO,12-Mar-14 MENU,17-Mar-14 BERLIN,18-Mar-14 UNO,19-Mar-14 ACADEMY,19-Mar-14 CENTER,19-Mar-14 COMPUTER,19-Mar-14 MANAGEMENT,19-Mar-14 SYSTEMS,19-Mar-14 BUILDERS,26-Mar-14 EMAIL,26-Mar-14 SOLUTIONS,26-Mar-14 SUPPORT,26-Mar-14 TRAINING,26-Mar-14 RUHR,27-Mar-14 CEO,28-Mar-14 CAMP,02-Apr-14 EDUCATION,02-Apr-14 GLASS,02-Apr-14 INSTITUTE,02-Apr-14 REPAIR,02-Apr-14 COFFEE,09-Apr-14 FLORIST,09-Apr-14 HOUSE,09-Apr-14 INTERNATIONAL,09-Apr-14 SOLAR,09-Apr-14 BUZZ,15-Apr-14 PICS,15-Apr-14 PHOTO,15-Apr-14 GIFT,15-Apr-14 LINK,15-Apr-14 GUITARS,15-Apr-14 HOLIDAY,16-Apr-14 MARKETING,16-Apr-14 BLUE,17-Apr-14 KIM,17-Apr-14 PINK,17-Apr-14 RED,17-Apr-14 SHIKSHA,17-Apr-14 ONL,17-Apr-14 CODES,23-Apr-14 FARM,23-Apr-14 VIAJES,23-Apr-14 BUILD,28-Apr-14 BOUTIQUE,30-Apr-14 BARGAINS,30-Apr-14 ZONE,30-Apr-14 CHEAP,30-Apr-14 AGENCY,30-Apr-14 KIWI,01-May-14 WATCH,07-May-14 COOL,07-May-14 CLUB,07-May-14 TIENDA,07-May-14 WORKS,07-May-14 EXPERT,07-May-14 EXPOSED,14-May-14 DEMOCRAT,14-May-14 DANCE,14-May-14 FOUNDATION,14-May-14 LUXURY,19-May-14 FLIGHTS,21-May-14 CRUISES,21-May-14 RENTALS,21-May-14 VILLAS,21-May-14 VACATIONS,21-May-14 WIKI,26-May-14 IMMOBILIEN,28-May-14 NINJA,28-May-14 PROPERTIES,28-May-14 CONDOS,28-May-14 MAISON,28-May-14 XYZ,01-Jun-14 EVENTS,04-Jun-14 PARTNERS,04-Jun-14 DATING,04-Jun-14 SOCIAL,04-Jun-14 REVIEWS,04-Jun-14 FUTBOL,04-Jun-14 PRODUCTIONS,04-Jun-14 CLEANING,11-Jun-14 CATERING,11-Jun-14 COMMUNITY,11-Jun-14 TOOLS,11-Jun-14 CARDS,11-Jun-14 UK,16-Jun-14 PARTS,18-Jun-14 INDUSTRIES,18-Jun-14 SUPPLY,18-Jun-14 SUPPLIES,18-Jun-14 NAGOYA,23-Jun-14 REPORT,25-Jun-14 VISION,25-Jun-14 FISH,25-Jun-14 PUB,09-Jul-14 WIEN,15-Jul-14 MODA,16-Jul-14 TOKYO,22-Jul-14 VOTING,22-Jul-14 KAUFEN,23-Jul-14 ACTOR,06-Aug-14

Fri Mar 21 15:46:52 2014 alexp [...] cpan.org - Correspondence added

On Fri Mar 21 15:35:36 2014, BARBIE wrote: Show quoted text

> Hi Alex, > > Ah I see what you mean, yes that would be nice, then you can update > locally periodically when things settle. > > As far as I know there are 1400+ extensions that have been agreed and > signed off, and will be released throughout this year. Many have > scheduled release dates, due to the pre-planning for Sunrise/Landrush > etc. I've attached the ones I know about, which should give you at > least 4 months worth. > > If you want the full 1400+ approved extensions I can get you those > too. > > Thanks, > Barbie

Hi Barbie, Can you send me a list of all 1400 extensions in the following format "extension","extension description" for example "bike","bike extension description" lowercase is fine for gTLD. If you don't have description, maybe I could auto generate it. Then I will push this into next release and schedule the auto update mechanism for release after that one.