Bug #124344 for Text-CSV: Text::CSV problem/question

Tue Feb 06 16:38:52 2018 lorienstice [...] gmail.com - Ticket created

Subject:	Text::CSV problem/question
Date:	Tue, 6 Feb 2018 13:38:36 -0800
To:	bug-Text-CSV [...] rt.cpan.org
From:	Lorien Stice-Lawrence <lorienstice [...] gmail.com>

Hi, Thanks for putting together such a useful module. I've used it in many programs in the past. In the past it used to be the case that when I was printing to a CSV file throughout my program (i.e. once at the beginning to print the labels, and then looping through items and printing a line at a time) the output csv file was continuously being edited as my program ran and I could always peak inside the file to make sure the output looked right. Similarly, if the program ran into a problem and failed (for a reason unrelated to Text::CSV) I would still have all of the output I'd printed from the previous loops. However, now no content is printed to the csv file until the entire program has ended running, and if the program stops prematurely (for a non-Text::CSV-related reason) none of the output has been printed to the file at all. I'm at a loss as to why this is happening because I'm using the same structure that I always have. This is the basic structure of the loop I'm using. Is my issue due to a change to Text::CSV or have I somehow been lucking out before? Lorien use Text::CSV; my @rows; my $csv = Text::CSV->new ( { binary => 1, eol => "\n", auto_diag => 1 } ) or die "Cannot use CSV: ".Text::CSV->error_diag (); open(my $mycsv, ">", $outputfile) or die "$outputfile: $!"; my @columns=('var1', 'var1','var3'); $csv->print ($mycsv, [@columns]); #loop opendir (DIR, $folder) or die $!; while (my $now_file = readdir (DIR)){ #various calculations which create $var, $var2, and $var3 #PRINT ALL OUTPUT TO A CSV FILE# @my_array= ($var1, $var2,$var3); $csv-> print ($mycsv, [@my_array]); } closedir DIR; close $mycsv or die "$!";

Wed Feb 07 00:43:43 2018 ishigaki [...] cpan.org - Correspondence added

On Wed Feb 07 06:38:52 2018, lorienstice@gmail.com wrote: Show quoted text

> Hi, > > Thanks for putting together such a useful module. I've used it in many > programs in the past. In the past it used to be the case that when I was > printing to a CSV file throughout my program (i.e. once at the beginning to > print the labels, and then looping through items and printing a line at a > time) the output csv file was continuously being edited as my program ran > and I could always peak inside the file to make sure the output looked > right. Similarly, if the program ran into a problem and failed (for a > reason unrelated to Text::CSV) I would still have all of the output I'd > printed from the previous loops. However, now no content is printed to the > csv file until the entire program has ended running, and if the program > stops prematurely (for a non-Text::CSV-related reason) none of the output > has been printed to the file at all. I'm at a loss as to why this is > happening because I'm using the same structure that I always have.

Could you show us a bit more basic info? - Which version of Text::CSV you use (and the one of Text::CSV_XS if you have)? - Around when is the "past"? (ie. do you remember about which version you used before?) - Do you have a sample CSV file that can reproduce your issue with the latest Text::CSV(_XS)? Show quoted text

> > This is the basic structure of the loop I'm using. Is my issue due to a > change to Text::CSV or have I somehow been lucking out before? > > Lorien > > use Text::CSV; > > my @rows; > my $csv = Text::CSV->new ( { binary => 1, eol => "\n", auto_diag => 1 } ) > or die "Cannot use CSV: ".Text::CSV->error_diag (); > open(my $mycsv, ">", $outputfile) or die "$outputfile: $!"; > > my @columns=('var1', 'var1','var3'); > $csv->print ($mycsv, [@columns]); > > > #loop > opendir (DIR, $folder) or die $!; > while (my $now_file = readdir (DIR)){ > > #various calculations which create $var, $var2, and $var3 > > > #PRINT ALL OUTPUT TO A CSV FILE# > @my_array= ($var1, $var2,$var3); > $csv-> print ($mycsv, [@my_array]); > } > > closedir DIR; > close $mycsv or die "$!";

Wed Feb 07 00:43:43 2018 The RT System itself - Status changed from 'new' to 'open'

Fri Feb 09 17:52:15 2018 lorienstice [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #124344] Text::CSV problem/question
Date:	Fri, 9 Feb 2018 14:51:53 -0800
To:	bug-Text-CSV [...] rt.cpan.org
From:	Lorien Stice-Lawrence <lorienstice [...] gmail.com>

Hi Kenichi, Thanks for getting back to me! In response to your questions: - I'm using version 1.95; both on Windows and also in a Linux environment (I have the same issue in both places). - I used Text::CSV in a program about a month ago and did not have this issue. - I've attached a file that I created with Text::CSV that suffered from this problem; i.e., while the Perl program was still running this file had a filesize of 0 and would be completely blank when you opened it. Not even labels printed in the first row, which I always do before looping through operations, were printed until the entire program had finished running. So there's nothing wrong with this file; it's just that this file is not populated at all until Perl has completely finished running. I can't attached any of the 0 byte files to this e-mail because Gmail won't let me (because they are completely empty) but there is definitely nothing in them. Thanks! Lorien On Tue, Feb 6, 2018 at 9:43 PM, Kenichi Ishigaki via RT < bug-Text-CSV@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=124344 > > > On Wed Feb 07 06:38:52 2018, lorienstice@gmail.com wrote:

> > Hi, > > > > Thanks for putting together such a useful module. I've used it in many > > programs in the past. In the past it used to be the case that when I was > > printing to a CSV file throughout my program (i.e. once at the beginning

> to

> > print the labels, and then looping through items and printing a line at a > > time) the output csv file was continuously being edited as my program ran > > and I could always peak inside the file to make sure the output looked > > right. Similarly, if the program ran into a problem and failed (for a > > reason unrelated to Text::CSV) I would still have all of the output I'd > > printed from the previous loops. However, now no content is printed to

> the

> > csv file until the entire program has ended running, and if the program > > stops prematurely (for a non-Text::CSV-related reason) none of the output > > has been printed to the file at all. I'm at a loss as to why this is > > happening because I'm using the same structure that I always have.

> > Could you show us a bit more basic info? > > - Which version of Text::CSV you use (and the one of Text::CSV_XS if you > have)? > - Around when is the "past"? (ie. do you remember about which version you > used before?) > - Do you have a sample CSV file that can reproduce your issue with the > latest Text::CSV(_XS)? >

> > > > This is the basic structure of the loop I'm using. Is my issue due to a > > change to Text::CSV or have I somehow been lucking out before? > > > > Lorien > > > > use Text::CSV; > > > > my @rows; > > my $csv = Text::CSV->new ( { binary => 1, eol => "\n", auto_diag => 1 } ) > > or die "Cannot use CSV: ".Text::CSV->error_diag (); > > open(my $mycsv, ">", $outputfile) or die "$outputfile: $!"; > > > > my @columns=('var1', 'var1','var3'); > > $csv->print ($mycsv, [@columns]); > > > > > > #loop > > opendir (DIR, $folder) or die $!; > > while (my $now_file = readdir (DIR)){ > > > > #various calculations which create $var, $var2, and $var3 > > > > > > #PRINT ALL OUTPUT TO A CSV FILE# > > @my_array= ($var1, $var2,$var3); > > $csv-> print ($mycsv, [@my_array]); > > } > > > > closedir DIR; > > close $mycsv or die "$!";

> > > >

Message body is not shown because sender requested not to inline it.

Sat Feb 10 08:11:16 2018 ishigaki [...] cpan.org - Correspondence added

On Sat Feb 10 07:52:15 2018, lorienstice@gmail.com wrote: Show quoted text

> Hi Kenichi, > > Thanks for getting back to me! In response to your questions: > > > - I'm using version 1.95; both on Windows and also in a Linux > environment (I have the same issue in both places). > - I used Text::CSV in a program about a month ago and did not have > this > issue.

1. Because Text::CSV has not been released since last April and you said you didn't see this issue about a month ago, it most probably is your problem (unless you updated Text::CSV_XS quite recently which I doubt). 2. Considering your explanation and the code example you showed above, it looks like you happen to have forgotten to enable autoflush for your output handle. Add $mycsv->autoflush(1); or something equivalent after you open a file to output and see. Show quoted text

> - I've attached a file that I created with Text::CSV that suffered > from > this problem; i.e., while the Perl program was still running this file > had > a filesize of 0 and would be completely blank when you opened it. Not > even > labels printed in the first row, which I always do before looping > through > operations, were printed until the entire program had finished > running. So > there's nothing wrong with this file; it's just that this file is not > populated at all until Perl has completely finished running. I can't > attached any of the 0 byte files to this e-mail because Gmail won't > let me > (because they are completely empty) but there is definitely nothing in > them. > > Thanks! > > Lorien > > On Tue, Feb 6, 2018 at 9:43 PM, Kenichi Ishigaki via RT < > bug-Text-CSV@rt.cpan.org> wrote: >

> > <URL: https://rt.cpan.org/Ticket/Display.html?id=124344 > > > > > On Wed Feb 07 06:38:52 2018, lorienstice@gmail.com wrote:

> > > Hi, > > > > > > Thanks for putting together such a useful module. I've used it in > > > many > > > programs in the past. In the past it used to be the case that when > > > I was > > > printing to a CSV file throughout my program (i.e. once at the > > > beginning

> > to

> > > print the labels, and then looping through items and printing a > > > line at a > > > time) the output csv file was continuously being edited as my > > > program ran > > > and I could always peak inside the file to make sure the output > > > looked > > > right. Similarly, if the program ran into a problem and failed (for > > > a > > > reason unrelated to Text::CSV) I would still have all of the output > > > I'd > > > printed from the previous loops. However, now no content is printed > > > to

> > the

> > > csv file until the entire program has ended running, and if the > > > program > > > stops prematurely (for a non-Text::CSV-related reason) none of the > > > output > > > has been printed to the file at all. I'm at a loss as to why this > > > is > > > happening because I'm using the same structure that I always have.

> > > > Could you show us a bit more basic info? > > > > - Which version of Text::CSV you use (and the one of Text::CSV_XS if > > you > > have)? > > - Around when is the "past"? (ie. do you remember about which version > > you > > used before?) > > - Do you have a sample CSV file that can reproduce your issue with > > the > > latest Text::CSV(_XS)? > >

> > > > > > This is the basic structure of the loop I'm using. Is my issue due > > > to a > > > change to Text::CSV or have I somehow been lucking out before? > > > > > > Lorien > > > > > > use Text::CSV; > > > > > > my @rows; > > > my $csv = Text::CSV->new ( { binary => 1, eol => "\n", auto_diag => > > > 1 } ) > > > or die "Cannot use CSV: ".Text::CSV->error_diag > > > (); > > > open(my $mycsv, ">", $outputfile) or die "$outputfile: $!"; > > > > > > my @columns=('var1', 'var1','var3'); > > > $csv->print ($mycsv, [@columns]); > > > > > > > > > #loop > > > opendir (DIR, $folder) or die $!; > > > while (my $now_file = readdir (DIR)){ > > > > > > #various calculations which create $var, $var2, and $var3 > > > > > > > > > #PRINT ALL OUTPUT TO A CSV FILE# > > > @my_array= ($var1, $var2,$var3); > > > $csv-> print ($mycsv, [@my_array]); > > > } > > > > > > closedir DIR; > > > close $mycsv or die "$!";

> > > > > > > >