Bug #100977 for Image-ExifTool: error in moving tiff files

Wed Dec 17 15:41:40 2014 rgb [...] ellerbach.com - Ticket created

Subject:	error in moving tiff files
Date:	Wed, 17 Dec 2014 15:41:01 -0500
To:	bug-Image-ExifTool [...] rt.cpan.org
From:	Richard Ball <rgb [...] ellerbach.com>

Version 9.76 on OSX 10.9.5 This command was used on a bunch of TIFF files: exiftool '-Directory<DateTimeOriginal' -if'$datetimeoriginal =~ /1984/' -d %Y . This ended up changing a lot of the metadata and in some cases resulted in a loss of the color information. I used exiftool -all against an original and a new file (that went from colour to monochrome) and the attached text file is the diff output. The size of the file has gone from 18MB to 9.5MB.

Version 9.76 on OSX 10.9.5

This command was used on a bunch of TIFF files:

exiftool '-Directory<DateTimeOriginal' -if'$datetimeoriginal =~ /1984/' -d %Y .

This ended up changing a lot of the metadata and in some cases resulted in a loss of the color information.

I used exiftool -all against an original and a new file (that went from colour to monochrome) and the attached text file is the diff output. The size of the file has gone from 18MB to 9.5MB.

Message body is not shown because sender requested not to inline it.

Wed Dec 17 17:53:04 2014 EXIFTOOL [...] cpan.org - Correspondence added

Thanks for this report, however I don't believe that your conclusions are valid for a few reasons: 1) The command in question will not modify the file. (even with the missing space after between "-if" and its argument). 2) Even if you did somehow manage to execute an exiftool command to change the files, the byte ordering would never be changed. ExifTool will not change the byte order of a TIFF file. 3) Even if you did somehow manage to execute an exiftool command to change the files, then exiftool would preserve the originals by adding "_original" to the file name, so you could restore from these files and nothing would be lost. I suspect you ran some other utility unknowinly. Microsoft utilities are notorious for changing byte order and running when you don't realize it. - Phil

Wed Dec 17 17:53:04 2014 The RT System itself - Status changed from 'new' to 'open'

Wed Dec 17 18:35:33 2014 EXIFTOOL [...] cpan.org - Correspondence added

Looking further down the difference file I see that the image was compressed (which ExifTool can not do), and also that it was Apple Aperture that is the culprit. - Phil

Wed Dec 17 20:14:18 2014 rgb [...] ellerbach.com - Correspondence added

Subject:	Re: [rt.cpan.org #100977] error in moving tiff files
Date:	Wed, 17 Dec 2014 20:13:52 -0500
To:	bug-Image-ExifTool [...] rt.cpan.org
From:	Richard Ball <rgb [...] ellerbach.com>

Thanks for the reply Phil. The files were in Aperture and I exported them from there so I can archive them separately from the remainder of the Aperture library. They were originally scanned as PhotoCD images several decades ago and then read off those CDs as tiff images which were what was brought into Aperture. The command I gave to exiftool, according to the help file, does not create the _original file since it is writing a new tiff to a new directory so the input file stays intact in the originating directory. I’ll describe in some more detail what went on, maybe that will help sort out what happened. I have 882 files of these PCD images (~16GB) and I wanted to separate those that had relevant date info from those that didn’t so I could modify the latter manually. The files all have the extension .tif. I ran this command on the directory they are in exiftool '-Directory<DateTimeOriginal' -if'$datetimeoriginal =~ /1984/' -d %Y . I get this output to the terminal: Warning: Tag 'if$datetimeoriginal' does not exist Warning: No writable tags set from ./pcddisk_0649_001.tif Warning: No writable tags set from ./pcddisk_0649_002.tif Warning: No writable tags set from ./pcddisk_0649_003.tif Warning: No writable tags set from ./pcddisk_0649_004.tif , . (continues for all the files) . 1 directories scanned 3 directories created 182 image files updated 699 image files unchanged The three directories created are “1984”, “1995” and “2010”. There are some files with datetimeoriginal data of 1984 or 1995. There are none with the date of 2010 that I know of but some do have a Modifydate of that year. If I run “exiftool -s -all” on the file pcddisk_2045_012.tiff (note the new extension) created by exiftool in the directory 2010 I get the info in the attached file out1.txt. If I run the same command on the original file (taken from a backup directory) I get the minimal info in the attached out2.txt file. The really odd thing is that if I run that “create_new_directories” command on just the original pcddisk_2045_012.tif file I get the message: Warning: Tag 'if$datetimeoriginal' does not exist Warning: No writable tags set from pcddisk_2045_012.tif 0 image files updated 1 image files unchanged and no directory is created and no .tiff file written. I’d be happy to run some tests if you have anything to suggest. —Richard On Dec 17, 2014, at 18:35, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=100977 > > > Looking further down the difference file I see that the image was compressed (which ExifTool can not do), and also that it was Apple Aperture that is the culprit. > > - Phil

Thanks for the reply Phil.

The files were in Aperture and I exported them from there so I can archive them separately from the remainder of the Aperture library. They were originally scanned as PhotoCD images several decades ago and then read off those CDs as tiff images which were what was brought into Aperture.

The command I gave to exiftool, according to the help file, does not create the _original file since it is writing a new tiff to a new directory so the input file stays intact in the originating directory.

I’ll describe in some more detail what went on, maybe that will help sort out what happened. I have 882 files of these PCD images (~16GB) and I wanted to separate those that had relevant date info from those that didn’t so I could modify the latter manually. The files all have the extension .tif. I ran this command on the directory they are in

exiftool '-Directory<DateTimeOriginal' -if'$datetimeoriginal =~ /1984/' -d %Y .

I get this output to the terminal:

Warning: Tag 'if$datetimeoriginal' does not exist

Warning: No writable tags set from ./pcddisk_0649_001.tif

Warning: No writable tags set from ./pcddisk_0649_002.tif

Warning: No writable tags set from ./pcddisk_0649_003.tif

Warning: No writable tags set from ./pcddisk_0649_004.tif

,

. (continues for all the files)

.

1 directories scanned

3 directories created

182 image files updated

699 image files unchanged

The three directories created are “1984”, “1995” and “2010”. There are some files with datetimeoriginal data of 1984 or 1995. There are none with the date of 2010 that I know of but some do have a Modifydate of that year.

If I run “exiftool -s -all” on the file pcddisk_2045_012.tiff (note the new extension) created by exiftool in the directory 2010 I get the info in the attached file out1.txt. If I run the same command on the original file (taken from a backup directory) I get the minimal info in the attached out2.txt file.

The really odd thing is that if I run that “create_new_directories” command on just the original pcddisk_2045_012.tif file I get the message:

Warning: Tag 'if$datetimeoriginal' does not exist

Warning: No writable tags set from pcddisk_2045_012.tif

0 image files updated

1 image files unchanged

and no directory is created and no .tiff file written. I’d be happy to run some tests if you have anything to suggest.

Message body is not shown because sender requested not to inline it.

—Richard

On Dec 17, 2014, at 18:35, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote:

Show quoted text

<URL: https://rt.cpan.org/Ticket/Display.html?id=100977 >

Looking further down the difference file I see that the image was compressed (which ExifTool can not do), and also that it was Apple Aperture that is the culprit.

- Phil

Thu Dec 18 07:25:59 2014 EXIFTOOL [...] cpan.org - Correspondence added

You're correct about the "_original" file. It won't be created if the original file is not touched. The warning about 'if$datetimeoriginal' is because you need a space after the -if option. So your -if condition was effectively ignored due to this. However, there is still something else going on. Also, ExifTool will not change the file extension unless you tell it to. I suggest you try this: 1) re-copy the PCD images into a fresh (initially empty) directory. 2) re-run your exiftool command. 3) look at the images in the sub-directories. If you are only writing the "Directory" tag in exiftool, then the files are only moved, and never actually written. If you omit the space after "-if", then all files containing DateTimeOriginal will be moved. If you add the space, then only files with a DateTimeOriginal containing 1984 will be moved. - Phil

Thu Dec 18 10:47:31 2014 rgb [...] ellerbach.com - Correspondence added

Subject:	Re: [rt.cpan.org #100977] error in moving tiff files
Date:	Thu, 18 Dec 2014 10:47:15 -0500
To:	bug-Image-ExifTool [...] rt.cpan.org
From:	Richard Ball <rgb [...] ellerbach.com>

OK, here is the test I ran (work on the files without 1984 in the DateTimeOriginal): ball$ mkdir PCDimages ball$ cp PCD\ images\ originals/* PCDimages/ ball$ cd PCDimages/ ball$ exiftool '-Directory<DateTimeOriginal' -if '$datetimeoriginal =! /1984/' -d %Y . Warning: No writable tags set from ./pcddisk_0649_001.tif Warning: No writable tags set from ./pcddisk_0649_002.tif Warning: No writable tags set from ./pcddisk_0649_003.tif . . . 1 directories scanned 3 directories created 182 image files updated 699 image files unchanged ball$ ls -1 | head -5 1984 1995 2010 pcddisk_0649_001.tif pcddisk_0649_002.tif In the two directories 1984 and 1995 the files have their original sizes and the same .tif extension so it appears they are moved without any modification. However the files “moved” into the 2010 directory are modified in the process. ball$ ls -l 2010 | head total 599376 -rw-r--r--@ 1 ball staff 10535374 Dec 18 10:05 pcddisk_0649_040.tiff -rw-r--r--@ 1 ball staff 18877126 Dec 18 10:06 pcddisk_0650_048.tiff -rw-r--r--@ 1 ball staff 9923805 Dec 18 10:07 pcddisk_2045_012.tiff -rw-r--r--@ 1 ball staff 11264423 Dec 18 10:07 pcddisk_2045_013.tiff -rw-r--r--@ 1 ball staff 9893265 Dec 18 10:07 pcddisk_2045_017.tiff -rw-r--r--@ 1 ball staff 9981043 Dec 18 10:07 pcddisk_2045_018.tiff -rw-r--r--@ 1 ball staff 9461657 Dec 18 10:07 pcddisk_2045_019.tiff -rw-r--r--@ 1 ball staff 10376917 Dec 18 10:07 pcddisk_2045_022.tiff -rw-r--r--@ 1 ball staff 11854743 Dec 18 10:07 pcddisk_2045_024.tiff For comparison here are the ls results for the same first four files from the 2010 directory from the originals: -rw-r--r--@ 1 ball staff 18875756 Feb 3 2010 pcddisk_0649_040.tif -rw-r--r--@ 1 ball staff 18875756 Feb 3 2010 pcddisk_0650_048.tif -rw-r--r--@ 1 ball staff 18875756 Feb 3 2010 pcddisk_2045_012.tif -rw-r--r--@ 1 ball staff 18875756 Feb 3 2010 pcddisk_2045_013.tif I don’t yet know why exiftool is picking the 29 files it is writing into the 2010 directory Richard On Dec 18, 2014, at 7:25, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=100977 > > > You're correct about the "_original" file. It won't be created if the original file is not touched. > > The warning about 'if$datetimeoriginal' is because you need a space after the -if option. So your -if condition was effectively ignored due to this. > > However, there is still something else going on. Also, ExifTool will not change the file extension unless you tell it to. I suggest you try this: > > 1) re-copy the PCD images into a fresh (initially empty) directory. > > 2) re-run your exiftool command. > > 3) look at the images in the sub-directories. > > If you are only writing the "Directory" tag in exiftool, then the files are only moved, and never actually written. If you omit the space after "-if", then all files containing DateTimeOriginal will be moved. If you add the space, then only files with a DateTimeOriginal containing 1984 will be moved. > > - Phil

Thu Dec 18 12:13:55 2014 EXIFTOOL [...] cpan.org - Correspondence added

I know why. There were a number of edited files with extension ".tiff" in the "originals" directory. What is the result of this command?: ls PCD\ images\ originals/*.tiff - Phil

Thu Dec 18 12:23:35 2014 rgb [...] ellerbach.com - Correspondence added

Subject:	Re: [rt.cpan.org #100977] error in moving tiff files
Date:	Thu, 18 Dec 2014 12:23:17 -0500
To:	bug-Image-ExifTool [...] rt.cpan.org
From:	Richard Ball <rgb [...] ellerbach.com>

You are right, there are .tiff files corresponding to each of the .tif files that get put into the 2010 directory. Thanks! On Dec 18, 2014, at 12:13, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=100977 > > > I know why. There were a number of edited files with extension ".tiff" in the "originals" directory. > > What is the result of this command?: > > ls PCD\ images\ originals/*.tiff > > - Phil

Thu Dec 18 12:29:58 2014 EXIFTOOL [...] cpan.org - Correspondence added 20 min

Great. Glad to resolve this. I'm putting this ticket in the rejected pile because it turned out not to be an exiftool bug after all. - Phil

Thu Dec 18 12:29:58 2014 EXIFTOOL [...] cpan.org - Status changed from 'open' to 'rejected'

Thu Dec 18 12:37:15 2014 rgb [...] ellerbach.com - Correspondence added

Subject:	Re: [rt.cpan.org #100977] error in moving tiff files
Date:	Thu, 18 Dec 2014 12:36:39 -0500
To:	bug-Image-ExifTool [...] rt.cpan.org
From:	Richard Ball <rgb [...] ellerbach.com>

Yes, user error. On Dec 18, 2014, at 12:29, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=100977 > > > Great. Glad to resolve this. > > I'm putting this ticket in the rejected pile because it turned out not to be an exiftool bug after all. > > - Phil