Bug #109660 for File-ChangeNotify: 'wrong' event being caught for file creation in File::ChangeNotify::Watcher::Inotify

Wed Nov 25 08:35:49 2015 ltolhurst [...] inview.tv - Ticket created

Subject:	'wrong' event being caught for file creation in File::ChangeNotify::Watcher::Inotify
Date:	Wed, 25 Nov 2015 13:35:14 +0000
To:	<bug-File-ChangeNotify [...] rt.cpan.org>
From:	Ludovic Tolhurst <ltolhurst [...] inview.tv>

Hi there Mr Rolsky, I have been using the File::ChangeNotify module in some recent work for my employer. I have tried using both File::ChangeNotify::Watcher::Default and File::ChangeNotify::Watcher::Inotify. This problem affects both in a sense, but is really an issue with File::ChangeNotify::Watcher::Inotify. Some of the files I was working with were on the bulky side but not humunguous - ca. 220MB. But I was finding that when I wrote code to check the file size after the watcher had detected it, these files were failing a validity tests that they should be over 66 lines. We came to realise that the watcher was firing the code before its contents had finished being copied. Our lead dev figured out that the event you are picking up in ::Watcher::Inotify via Linux::Inotify2, IN_CREATE, was the issue. This event gets fired at the beginning of file creation. Hence, before the contents start being written. I tried going back to just using ::Watcher::Default, but the same issues were occurring there, too. Of course, this is unrelated to the inotify event, as inotify is not involved. In order to get the desired functionality, I simply replaced all instances of 'IN_CREATE' in the ::Watcher::Inotify code with 'IN_CLOSE_WRITE'. (I also made some POD changes and renamed the changed module). Now that I am watching for this event, I don't seem to be having any more problems with large files appearing to have zero lines. This should also address some of the issues mentioned in ticket #52325 regarding being able to run code at the end of a file transfer process. However, IN_CLOSE_WRITE also gets triggered when an existing file is overwritten, but not when renamed or moved. For my purposes, this was appropriate. But it does stretch the concept of file creation a little. Would you consider simply changing ::Watcher::Inotify to watch for this event in place of IN_CREATE? Thanks & regards, Ludovic Tolhurst-Cleaver Perl Engineer, Inview Technology Ltd., Northwich, England.

Fri Nov 27 01:51:41 2015 DROLSKY [...] cpan.org - Correspondence added

On Wed Nov 25 08:35:49 2015, ltolhurst@inview.tv wrote: Show quoted text

> Would you consider simply changing ::Watcher::Inotify to watch for this > event in place of IN_CREATE?

I don't think that'd be quite right. Reading the inotify man page, it looks like IN_CLOSE_WRITE will fire on both create & modify events. I think what I need to do is catch create & modify and _then_ wait for a corresponding IN_CLOSE_WRITE against the same file before passing the event on.

Fri Nov 27 01:51:42 2015 The RT System itself - Status changed from 'new' to 'open'

Mon Nov 30 05:15:47 2015 ltolhurst [...] inview.tv - Correspondence added

Subject:	Re: [rt.cpan.org #109660] 'wrong' event being caught for file creation in File::ChangeNotify::Watcher::Inotify
Date:	Mon, 30 Nov 2015 10:15:05 +0000
To:	<bug-File-ChangeNotify [...] rt.cpan.org>
From:	Ludovic Tolhurst <ltolhurst [...] inview.tv>

I agree - slightly more work, but that would of course allow you to accurately check that a new file has been created and fully written. Thanks for getting back to me. (BTW, thanks to you and the DateTime contributors for writing and maintaining DateTime - probably my favourite CPAN module!) Ludo On 27/11/15 06:51, Dave Rolsky via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=109660 > > > On Wed Nov 25 08:35:49 2015, ltolhurst@inview.tv wrote: >

>> Would you consider simply changing ::Watcher::Inotify to watch for this >> event in place of IN_CREATE?

> I don't think that'd be quite right. Reading the inotify man page, it looks like IN_CLOSE_WRITE will fire on both create & modify events. I think what I need to do is catch create & modify and _then_ wait for a corresponding IN_CLOSE_WRITE against the same file before passing the event on.