1

I have a part of a bash script, which I am executing as root:

rm -f ../../include/profile.h; \
cp profile.h ../../include/profile.h

the file ../../include/profile.h in question has permissions:

-rw-r--r-- 1 root root 12178 Nov  5 02:00 ../../include/profile.h

Most of the time, the snippet executes fine, but very rarely, impossible to reproduce, it fails:

cp: cannot create regular file `../../include/profile.h': File exists

I checked, there is no other part of the program, that would be executed in parallel, which could write the file in between the two statements, thus creating a race condition which would explain the behaviour. There is no other place than the above, which is executed only once, which would write this file.

The system is

kernel:

Linux dev64 2.6.32.63+drm33.26-64.128-a10 #6 SMP Fri Jul 25 15:21:56 PDT 2014 x86_64 x86_64 x86_64 GNU/Linux

distro:

CentOS release 6.3 (Final)

What could possibly cause this failure to appear??

Giacomo1968
  • 53,069
  • 19
  • 162
  • 212
user322908
  • 769
  • 2
  • 13
  • 24
  • 1
    The only thing that springs to mind is disc write caching, which may not have completed after the `rm` before the `cp` is executed. Perhaps calling `sync` between the two commands will cure it. – AFH Dec 13 '15 at 03:44
  • @AFH can you tell me, do I have a bug in the script, where `sync` is necessary for the script to be correct? In other words, is `sync` always necessary to call between two shell commands that depends on each others effect on the filesystem? I want to know is this my error, or is this a bug in Linux kernel and/or implementation of `cp`, which can be mitigated with the use of `sync`. – user322908 Dec 13 '15 at 04:00
  • Update to newest CentOS release 6.7. Then check whether the error still occurs. – Cyrus Dec 13 '15 at 08:08
  • @Cyrus thank you I can do that, but, please read my question carefully - it is meant literally. I am not so much interested in "try x and see if the problem disappears" because that will not bring me understanding of what is happening. I want to understand what is happening first and foremost, and based on that, I can then "do the right thing". – user322908 Dec 13 '15 at 08:35
  • @user322908 - It is certainly not a script bug and it is only a guess that it may be a caching bug: inserting `sync` will provide some evidence of the latter. If so, the cache in question may be in the disc driver or in the disc controller itself (I experienced this some years ago and the disc drives had to be replaced by the manufacturer). Unfortunately, I run Ubuntu, which is Debian-based, whereas CentOS is RedHat-based, so I cannot run any meaningful tests. I have given you a possible cause, as you asked: I'm not sure I can do more. – AFH Dec 13 '15 at 13:56
  • Normally `cp foo bar` will overwrite `bar` if it exists. The error `cp: cannot create regular file … File exists` indicates something deeper than only the old file not having been removed yet. Is FUSE involved? I believe it can introduce race conditions in some circumstances (see [this answer](https://superuser.com/a/1242432/432690)). Or maybe it's a remote filesystem and the root of the problem is on the server. I notice the question is old, the setup in question may no longer exist. – Kamil Maciorowski Mar 09 '21 at 08:44

3 Answers3

0

Use

strace cp profile.h ../../include/profile.h 2>logfile

and look for:

stat("../../include/profile.h", 0x7ffdca347950) = -1 ENOENT (No such file or directory)
open("../../include/profile.h", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4

That will tell you what's going on. In this example the file did not exist before the cp (which it determined), hence the O_CREAT|O_EXCL.

ctrl-d
  • 136
  • 5
0

Why not just run cp with the -f flag? Why are you doing an rm and then a cp?

Late answer but why are you doing this rm then cp dance?

rm -f ../../include/profile.h; \
cp profile.h ../../include/profile.h

Why not just run cp with the -f flag to force an overwrite like this:

cp -f profile.h ../../include/profile.h.

Seems like your script is doing double the work for no benefit.

Giacomo1968
  • 53,069
  • 19
  • 162
  • 212
-2

It may be a race condition when different processes are writing to a same file location.

cp process A:

  • action1: found the file not exist
  • action2: create the file

cp process B:

  • action1: found the file not exist
  • action2: create the file

When B.action2 happen between A.action1 and A.action2, A will report "cp: cannot create regular file '???': File exists"

Chunlin Zhang
  • 1
  • 2
    A quote from the question: _"I checked, there is no other part of the program, that would be executed in parallel, which could write the file in between the two statements, thus creating a race condition which would explain the behaviour. There is no other place than the above, which is executed only once, which would write this file"_ – gronostaj Mar 09 '21 at 08:08