Skip Menu |

This queue is for tickets about the File-Path CPAN distribution.

Report information
The Basics
Id: 56509
Status: resolved
Priority: 0/
Queue: File-Path

People
Owner: jkeenan [...] cpan.org
Requestors: dnucera [...] ti.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 2.00_07
Fixed in: (no value)



Subject: Race condition
Hi I'm currently using File::Path with perl5.8.3 on a 64 bits linux machine. I have some questions about mkpath function in the perl package File::path you support. I would like to know if you take into account race conditions for it. Let me give you more details. I have several scripts that are run in parallel and try to create a directory (kind of “mkdir –p” command). I have some issues when using unix command “mkdir –p” at the same time For example: %mkdir –p /a/b/c %mkdir –p /a/b/d If b doesn’t exist at run time, irst script will try to create it, while second script will try to write in it whereas it is not completely created. Indeed I found this article to explain the behavior: Second, is that `mkdir -p` does not work for parallel builds. This occurs when make tries to invoke, for example, `mkdir -p /a/b` and `mkdir -p /a/c` in parallel, with directory /a being non-existent. What `mkdir -p` does is that first it stats /a directory and if it does not exist it tries to create that. There is a race condition between stat'ing the directory and creating it, during which another parallel invocation of mkdir creates this same directory, which causes creating /a directory in the former mkdir to return EEXIST and fail. It must be mkdir bug (both gnu and Solaris), but I not sure if it can be fixed in mkdir, since if creating a directory returns EEXIST, that directory needs to be stat'ed again, to make sure it is a directory and not a file, leading to a race condition again. (no POSIX interface to create a directory and if that name already exists in the filesystem return its type). I wanted to know if mkpath function can solve this issue. Thanks for your answer. Didier
On 2010-04-12 03:44:01, dnucera wrote: Show quoted text
> Hi > I'm currently using File::Path with perl5.8.3 on a 64 bits linux machine. > I have some questions about mkpath function in the perl package > File::path you support. > > > > I would like to know if you take into account race conditions for it. > Let me give you more details. > > I have several scripts that are run in parallel and try to create a > directory (kind of “mkdir –p” command). > > > > I have some issues when using unix command “mkdir –p” at the same time > > For example: > > %mkdir –p /a/b/c > > %mkdir –p /a/b/d > > > > If b doesn’t exist at run time, irst script will try to create it, while > second script will try to write in it whereas it is not completely created. > > Indeed I found this article to explain the behavior: > > > > Second, is that `mkdir -p` does not work for parallel builds. This > occurs when make tries to invoke, for example, `mkdir -p /a/b` and > `mkdir -p /a/c` in parallel, with directory /a being non-existent. What > `mkdir -p` does is that first it stats /a directory and if it does not > exist it tries to create that. There is a race condition between > stat'ing the directory and creating it, during which another parallel > invocation of mkdir creates this same directory, which causes creating > /a directory in the former mkdir to return EEXIST and fail. It must be > mkdir bug (both gnu and Solaris), but I not sure if it can be fixed in > mkdir, since if creating a directory returns EEXIST, that directory > needs to be stat'ed again, to make sure it is a directory and not a > file, leading to a race condition again. (no POSIX interface to create a > directory and if that name already exists in the filesystem return its > type). > > > > > > I wanted to know if mkpath function can solve this issue. >
Looking at the source code of File::Path::_mkpath it seems that this case (multiple processes are doing mkpath at the same time) is handled. There's a comment explicitly saying "allow for another process to have created it meanwhile". I think this issue may be set to resolved. Regards, Slaven
On Tue Dec 17 05:58:52 2013, SREZIC wrote: Show quoted text
> On 2010-04-12 03:44:01, dnucera wrote:
> > Hi > > I'm currently using File::Path with perl5.8.3 on a 64 bits linux > > machine. > > I have some questions about mkpath function in the perl package > > File::path you support. > > > > > > > > I would like to know if you take into account race conditions for it. > > Let me give you more details. > > > > I have several scripts that are run in parallel and try to create a > > directory (kind of “mkdir –p” command). > > > > > > > > I have some issues when using unix command “mkdir –p” at the same > > time > > > > For example: > > > > %mkdir –p /a/b/c > > > > %mkdir –p /a/b/d > > > > > > > > If b doesn’t exist at run time, irst script will try to create it, > > while > > second script will try to write in it whereas it is not completely > > created. > > > > Indeed I found this article to explain the behavior: > > > > > > > > Second, is that `mkdir -p` does not work for parallel builds. This > > occurs when make tries to invoke, for example, `mkdir -p /a/b` and > > `mkdir -p /a/c` in parallel, with directory /a being non-existent. > > What > > `mkdir -p` does is that first it stats /a directory and if it does > > not > > exist it tries to create that. There is a race condition between > > stat'ing the directory and creating it, during which another parallel > > invocation of mkdir creates this same directory, which causes > > creating > > /a directory in the former mkdir to return EEXIST and fail. It must > > be > > mkdir bug (both gnu and Solaris), but I not sure if it can be fixed > > in > > mkdir, since if creating a directory returns EEXIST, that directory > > needs to be stat'ed again, to make sure it is a directory and not a > > file, leading to a race condition again. (no POSIX interface to > > create a > > directory and if that name already exists in the filesystem return > > its > > type). > > > > > > > > > > > > I wanted to know if mkpath function can solve this issue. > >
> > Looking at the source code of File::Path::_mkpath it seems that this > case (multiple processes are doing mkpath at the same time) is > handled. There's a comment explicitly saying "allow for another > process to have created it meanwhile". > > I think this issue may be set to resolved. > > Regards, > Slaven
Since there has been no further communication from the original poster, and since I trust Slaven's judgement on this, I'm marking this ticket Resolved. Thank you very much. Jim Keenan