Skip Menu |

This queue is for tickets about the Image-ExifTool CPAN distribution.

Report information
The Basics
Id: 56487
Status: resolved
Worked: 1 hour (60 min)
Priority: 0/
Queue: Image-ExifTool

People
Owner: EXIFTOOL [...] cpan.org
Requestors: rrt [...] sc3d.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 8.15
Fixed in: 8.40



Subject: Does not preserve file permissions when altering file
WriteInfo seems to mask the current permissions with my umask, rather than preserving the file permissions, when updating EXIF tags.
On Sun Apr 11 14:33:53 2010, rrt wrote: Show quoted text
> WriteInfo seems to mask the current permissions with my umask, rather > than preserving the file permissions, when updating EXIF tags.
I'm not sure whether I should consider this a bug or not. ExifTool creates a new file when any tag is written. (Although if you are using the application -overwrite_original option or passing a single file as an argument to WriteInfo() then the new file is renamed to replace the original.) The application has a -P option to preserve the file modify date, but I have made no provision for preserving any other file attributes. (Other than the - overwrite_original_in_place option which has a big performance impact so I don't consider this a real solution.) If you are using the API, it is up to you to preserve any attributes you want. I don't think this is unreasonable, although I am open to other opinions. - Phil
From: rrt [...] sc3d.org
On Mon Apr 12 08:55:56 2010, EXIFTOOL wrote: Show quoted text
> On Sun Apr 11 14:33:53 2010, rrt wrote:
> > WriteInfo seems to mask the current permissions with my umask,
> rather
> > than preserving the file permissions, when updating EXIF tags.
> > I'm not sure whether I should consider this a bug or not.
I'm going to try to convince you you should. Show quoted text
> ExifTool creates a new file when any tag is written. (Although if you > are using the application > -overwrite_original option or passing a single file as an argument to > WriteInfo() then the new > file is renamed to replace the original.)
Image::ExifTool is for extracting and modifying metadata. I think there's a reasonable expectation that when you see something like: $exifTool->WriteInfo($file); one expects the semantics to be that the file is updated, not recreated. (Of course, creating a new file and then copying it over the old one is an established and respectable way of implementing this.) This is similar to when one edits a file in an editor. One expects the timestamp to be updated, but not the owner or permissions. Show quoted text
> The application has a -P option to preserve the file modify date,
This I find odd, actually: the date of taking the picture should be in the EXIF tags, and when one updates the tags, the file is modified, so why would one want to preserve the mtime? Show quoted text
> If you are using the API, it is up to you to preserve any attributes > you want. > > I don't think this is unreasonable, although I am open to other > opinions.
Are there any good reasons NOT to preserve file metadata? We haven't even talked about extended attributes, which are increasingly used on images (e.g. by search daemons), which annoyingly many programs happily ignore. I agree that it's a bit of a pain that applications currently have to reinvent the wheel when all they want is something quite simple, but having said that it's not much of a wheel to reinvent. I am in two minds about this myself. I can happily code what I want in my script, but it's a bit of a pain if I have to do this every time I use Image::ExifTool. Fixing it in Image::ExifTool would have the good effect of automatically making a lot of scripts written using it DTRT. On the other hand, were I the module author, I'd prefer to have standard calls to do this (by "this" I mean "create a temporary file, then later use it to replace an existing file"), rather than risk doing it wrong. I see there are some notes here: http://sial.org/howto/perl/tmp-files/ In particular: "Use modules such as File::MMap or IO::String to keep the data in memory. This may not be an option in all cases, but should be faster than touching the filesystem, and will avoid any filesystem related security problems. In recent versions of Perl, the open function now has options to open files in memory or to anonymous files on the filesystem. For more information, see perlopentut and perlfunc." Could any of this apply to you? In particular, are you avoiding all those security holes? The source code is sufficiently involved that I can't easily tell... For now I guess I'll do my own attribute copying.
On Mon Apr 12 10:05:39 2010, rrt wrote: Show quoted text
> the file is modified, so > why would one want to preserve the mtime?
I know I use mtime myself for sorting images, and they sort wrong if this is changed. Show quoted text
> Are there any good reasons NOT to preserve file metadata?
I would call this filesystem metadata, as opposed to metadata contained within the file itself. The only reason not to preserve this is that I don't know how to do this in a system independent way without a big performance impact. Remember, this must work on Windows and Mac systems too. Show quoted text
> We haven't even talked about extended attributes, > which are increasingly used on images (e.g. by search > daemons), which annoyingly many programs happily > ignore.
I looked into how I could preserve the permissions, and this is quite easy using the standard Perl functions fstat and chmod. But then there are other things like the user/group ownership that I haven't looked into yet, and I don't know anything about these extended attributes that you mentioned. Also on Mac systems there are things like file creation date (ouch! -- no standard Perl functions to handle this one!), file comments, icon position, etc, etc. Show quoted text
> "Use modules such as File::MMap or IO::String to keep the > data in memory. This may not be an option in all cases, but > should be faster than touching the filesystem, and will avoid > any filesystem related security problems. In recent versions > of Perl, the open function now has options to open files in > memory or to anonymous files on the filesystem. > For more information, see perlopentut and perlfunc." > > Could any of this apply to you? In particular, are you avoiding > all those security holes? The source code is sufficiently > involved that I can't easily tell...
I don't know what security problems they are talking about. I would recommend specifying an output file for WriteInfo() and handling this yourself. It is for convenience that I added the ability for WriteInfo() to overwrite the original file, but I don't really recommend this use. FYI, my algorithm is the following if no output file is specified: 1) Create new file with name "INFILE_exiftool_tmp". 2) Rewrite the input file to the temporary file, changing any specified information in the process. 3) If the temporary file was written with no errors, then use Perl's rename function to replace the original file with the temporary file. ie) rename("${infile}_exiftool_tmp", $infile); - Phil
On Mon Apr 12 10:41:17 2010, EXIFTOOL wrote: Show quoted text
> FYI, my algorithm is the following if no output > file is specified:
Actually, I just checked the documentation and hopefully this was fairly clear: "The destination file name may be undefined to edit the source file in place (make sure you have backups!). In this case, if a source file name is provided, a temporary file is created and renamed to replace the source file if no errors occurred while writing." You're right that the semantics of WriteInfo() may suggest otherwise, but if I can make the documentation more clear, please let me know. (Also, please let me know if there is a platform independent technique which allows the file attributes to be preserved!) - Phil
Subject: Re: [rt.cpan.org #56487] Does not preserve file permissions when altering file
Date: Mon, 12 Apr 2010 16:26:50 +0100
To: bug-Image-ExifTool <bug-Image-ExifTool [...] rt.cpan.org>
From: Reuben Thomas <rrt [...] sc3d.org>
On 12 April 2010 15:41:18 UTC+1, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=56487 > > > On Mon Apr 12 10:05:39 2010, rrt wrote:
>> the file is modified, so >> why would one want to preserve the mtime?
> > I know I use mtime myself for sorting images, and they sort wrong if this is changed.
Fair enough. Show quoted text
>> Are there any good reasons NOT to preserve file metadata?
> > I would call this filesystem metadata,
Sorry, my sloppiness. Show quoted text
> The only reason not to preserve this is that I don't know how to do this in a system > independent way without a big performance impact.  Remember, this must work on Windows > and Mac systems too.
OK, but supporting some of them, as you say below: Show quoted text
> I looked into how I could preserve the permissions, and this is quite easy using the standard > Perl functions fstat and chmod.  But then there are other things like the user/group > ownership that I haven't looked into yet,
This is easy with standard perl stat/chown. Show quoted text
> and I don't know anything about these extended > attributes that you mentioned.
File::ExtAttr handles these. One could easily support them only on systems that have them. Show quoted text
>  Also on Mac systems there are things like file creation date > (ouch! -- no standard Perl functions to handle this one!), file comments, icon position, etc, > etc.
So, this needs extra, system-dependent support. But it's no disaster if at least at first things like this aren't supported. I came up with a simple extensible API for handling this: # Get attributes of a file # N.B. Treat return values as a list, don't rely on the actual values here # i.e. call attrs_set($file1, attrs_get($file2)), that way the # attributes saved can be extended without breaking the API. # FIXME: Support extended attributes. sub attrs_get { my ($file) = @_; my ($mode, $uid, $gid) = (stat($file))[2, 4, 5]; $mode = $mode & 07777; return $mode, $uid, $gid; } # Set attributes of a file previously saved with attrs_save # file: file to set attributes of # ...: attributes list returned by attrs_get sub attrs_set { my ($file, $mode, $uid, $gid) = @_; chmod $mode, $file; chown $uid, $gid, $file; } Show quoted text
> I don't know what security problems they are talking about.
Is there something unclear on that page? For starters: Show quoted text
> 1) Create new file with name "INFILE_exiftool_tmp".
Oops. This is attackable. Use File::Temp or similar for secure temporary file creation. -- http://rrt.sc3d.org
Subject: Re: [rt.cpan.org #56487] Does not preserve file permissions when altering file
Date: Mon, 12 Apr 2010 17:11:14 +0100
To: bug-Image-ExifTool <bug-Image-ExifTool [...] rt.cpan.org>
From: Reuben Thomas <rrt [...] sc3d.org>
On 12 April 2010 16:16, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=56487 > > > On Mon Apr 12 10:41:17 2010, EXIFTOOL wrote:
>> FYI, my algorithm is the following if no output >> file is specified:
> > Actually, I just checked the documentation and hopefully this was fairly clear: > > "The destination file name may be undefined to edit the source file in place (make sure you > have backups!).  In this case, if a source file name is provided, a temporary file is created and > renamed to replace the source file if no errors occurred while writing."
You're right about that. I think you could improve things by changing the comment: # edit file in place (you do have backups, right?) to # overwrites file (you do have backups, right?) Show quoted text
>  (Also, please let me know if there is a > platform independent technique which allows the file attributes to be preserved!)
I think what I sent earlier is a good starting point: at least the code I sent uses only standard Perl functions. -- http://rrt.sc3d.org
On Mon Apr 12 11:27:08 2010, rrt wrote: Show quoted text
> I came up with a simple extensible API for handling this:
Thanks, your example is very useful. I will consider adding this to the code. Show quoted text
> > I don't know what security problems they are talking about.
> > Is there something unclear on that page? For starters: >
> > 1) Create new file with name "INFILE_exiftool_tmp".
> > Oops. This is attackable. Use File::Temp or similar for secure > temporary file creation.
Sorry I didn't read your reference before, but I have browsed through it now and I don't see that what I am doing is a problem. I don't want to use File::Temp because I want to create the file in the same directory as the original (otherwise the subsequent rename may fail on some systems). This in itself avoids the security problems of using a common /tmp directory. Also, arbitrary file overwrite isn't a problem because I always check to see if the file (temporary or otherwise) exists before attempting to create it, and fail if the file already exists. - Phil
On Mon Apr 12 12:11:31 2010, rrt wrote: Show quoted text
> You're right about that. I think you could improve things by changing > the comment: > > # edit file in place (you do have backups, right?) > > to > > # overwrites file (you do have backups, right?)
Will do. Thanks.
Subject: Re: [rt.cpan.org #56487] Does not preserve file permissions when altering file
Date: Mon, 12 Apr 2010 17:46:17 +0100
To: bug-Image-ExifTool <bug-Image-ExifTool [...] rt.cpan.org>
From: Reuben Thomas <rrt [...] sc3d.org>
On 12 April 2010 17:21, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text
> Sorry I didn't read your reference before, but I have browsed through it now and I don't see > that what I am doing is a problem.  I don't want to use File::Temp because I want to create > the file in the same directory as the original (otherwise the subsequent rename may fail on > some systems).
I failed to notice this aspect of your scheme. -- http://rrt.sc3d.org
I am testing a version of the code that preserves the permissions and ownership when WriteInfo() is called without a destination file. So far it looks good. If all goes well this will be included in ExifTool 8.18. I think it is appropriate to keep the original behaviour when a new destination file is created (ie. the new file gets the default ownership/permissions). Let me know if you disagree. - PHil
Subject: Re: [rt.cpan.org #56487] Does not preserve file permissions when altering file
Date: Mon, 12 Apr 2010 18:34:51 +0100
To: bug-Image-ExifTool <bug-Image-ExifTool [...] rt.cpan.org>
From: Reuben Thomas <rrt [...] sc3d.org>
On 12 April 2010 18:29, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=56487 > > > I am testing a version of the code that preserves the permissions and ownership when > WriteInfo() is called without a destination file.  So far it looks good.  If all goes well this will be > included in ExifTool 8.18. > > I think it is appropriate to keep the original behaviour when a new destination file is created (ie. > the new file gets the default ownership/permissions).  Let me know if you disagree.
I agree absolutely: the semantics is (and should be) like a simple "cp file1 file2". -- http://rrt.sc3d.org
ExifTool 8.18 is now available from the exiftool home page, and should now preserve file permissions and ownership as discussed. Please let me know if you have any problems with this version. - Phil
Subject: Re: [rt.cpan.org #56487] Does not preserve file permissions when altering file
Date: Fri, 16 Apr 2010 20:57:00 +0100
To: bug-Image-ExifTool <bug-Image-ExifTool [...] rt.cpan.org>
From: Reuben Thomas <rrt [...] sc3d.org>
On 16 April 2010 16:48, Phil Harvey via RT <bug-Image-ExifTool@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=56487 > > > ExifTool 8.18 is now available from the exiftool home page, and should now preserve file > permissions and ownership as discussed. > > Please let me know if you have any problems with this version.
I will. Thanks very much for this version, and I look forward to its landing in distros! -- http://rrt.sc3d.org
Production version 8.25 was just released with this feature, however the chown() call causes a failure on systems which don't implement this function (ie. OS/2), so this function call must be placed inside an eval. I will fix this in the next release. - Phil
From: rrt [...] sc3d.org
I'm using Image::ExifTool 8.60, and it still updates the timestamp of files when I change the info using foo->WriteInfo($file), i.e. a single argument. Looking at the code, I can see some code to call SetFileModifyDate, but that's modifying tags in the file, not the file metadata. Did I miss something?
From: rrt [...] sc3d.org
On Fri Jun 29 19:38:21 2012, rrt wrote: Show quoted text
> I'm using Image::ExifTool 8.60, and it still updates the timestamp of > files when I change the info using foo->WriteInfo($file), i.e. a single > argument. Looking at the code, I can see some code to call > SetFileModifyDate, but that's modifying tags in the file, not the file > metadata. Did I miss something?
OK, it seems I missed that SetFileModifyDate is indeed supposed to modify the file. But when I simply update a tag (e.g. the thumbnail), the timestamp of the file is also updated, and I can't work out how to pass an option to preserve it.
From: rrt [...] sc3d.org
I'm so sorry, I completely misunderstood this bug. Rereading the entire thread makes it all clear again. Please ignore my recent comments and reclose the bug!