Weird behavior while moving a directory

Weird behavior while moving a directory - Servers Administration, Networking, & Virtualization

Users browsing this thread: 2 Guest(s)

	venam Offline \| 29-09-2016, 05:47 AM \| #1

Hello fellow nixers,
In this thread I'll describe a behavior I've noticed by mistake.
Maybe someone can enlighten me about it.

Here's the scenario:

Try to follow with me.
We have two users, user1 and user2.

user1 creates a directory, and check it's stats:

Code:
~ > mkdir will_disappear

 ~ > stat will_disappear

  File: ‘will_disappear’

  Size: 4096          Blocks: 8          IO Block: 4096   directory

Device: 801h/2049d    Inode: 6947274     Links: 2

Access: (0775/drwxrwxr-x)  Uid: ( 1000/ patrick)   Gid: ( 1000/ patrick)

Access: 2016-09-29 11:21:16.086851928 +0300

Modify: 2016-09-29 11:21:14.426851879 +0300

Change: 2016-09-29 11:21:14.426851879 +0300

 Birth: -

user1 creates a file the directory and also check the stats:

Code:
~/will_disappear > touch will_disappear/magic

 ~/will_disappear > stat will_disappear/magic

  File: ‘magic’

  Size: 0             Blocks: 0          IO Block: 4096   regular empty file

Device: 801h/2049d    Inode: 6947276     Links: 1

Access: (0664/-rw-rw-r--)  Uid: ( 1000/ patrick)   Gid: ( 1000/ patrick)

Access: 2016-09-29 11:21:28.070852284 +0300

Modify: 2016-09-29 11:21:28.070852284 +0300

Change: 2016-09-29 11:21:28.070852284 +0300

 Birth: -

Now user2 enters the directory...

Code:
~/will_disappear > cd will_disappear

And user1 moves the directory while user2 is still inside of it.

Code:
~ > mv will_disappear{,.bak}

 ~ > stat will_disappear.bak

  File: ‘will_disappear.bak’

  Size: 4096          Blocks: 8          IO Block: 4096   directory

Device: 801h/2049d    Inode: 6947274     Links: 2

Access: (0775/drwxrwxr-x)  Uid: ( 1000/ patrick)   Gid: ( 1000/ patrick)

Access: 2016-09-29 11:21:51.010852964 +0300

Modify: 2016-09-29 11:21:28.070852284 +0300

Change: 2016-09-29 11:22:04.338853360 +0300

 Birth: -

The inode of the directory hasn't changed but the inode of the parent directory has, this makes sense.
Clarifications:

To user2 there doesn't seem to be any issue so far, he can't notice until he does something like this:

Code:
~/will_disappear > touch $(pwd)/test

touch: cannot touch ‘/home/patrick/will_disappear/test’: No such file or directory

OK...

Code:
~/will_disappear > pwd

/home/patrick/will_disappear

Somehow it thinks it's still inside "will_disappear".

Here's my investigation:
This is very weird, I first thought that the shell might be highjacking the `pwd` command and I wanted to make sure.
With user2 I ran:

Code:
~/will_disappear > strace pwd                                                                                <

execve("/bin/pwd", ["pwd"], [/* 56 vars */]) = 0

brk(0)                                  = 0x1d08000

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5c75169000

open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3

fstat(3, {st_mode=S_IFREG|0644, st_size=173895, ...}) = 0

mmap(NULL, 173895, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5c7513e000

close(3)                                = 0

open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3

read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P \2\0\0\0\0\0"..., 832) = 832

fstat(3, {st_mode=S_IFREG|0755, st_size=1840928, ...}) = 0

mmap(NULL, 3949248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5c74b84000

mprotect(0x7f5c74d3e000, 2097152, PROT_NONE) = 0

mmap(0x7f5c74f3e000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ba000) = 0x7f5c74f3e000

mmap(0x7f5c74f44000, 17088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f5c74f44000

close(3)                                = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5c7513d000

mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5c7513b000

arch_prctl(ARCH_SET_FS, 0x7f5c7513b740) = 0

mprotect(0x7f5c74f3e000, 16384, PROT_READ) = 0

mprotect(0x606000, 4096, PROT_READ)     = 0

mprotect(0x7f5c7516b000, 4096, PROT_READ) = 0

munmap(0x7f5c7513e000, 173895)          = 0

brk(0)                                  = 0x1d08000

brk(0x1d29000)                          = 0x1d29000

open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3

fstat(3, {st_mode=S_IFREG|0644, st_size=7221056, ...}) = 0

mmap(NULL, 7221056, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5c744a1000

close(3)                                = 0

getcwd("/home/patrick/will_disappear.bak", 4096) = 33

fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 32), ...}) = 0

mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5c75168000

write(1, "/home/patrick/will_disappear.bak"..., 33/home/patrick/will_disappear.bak

) = 33

close(1)                                = 0

munmap(0x7f5c75168000, 4096)            = 0

close(2)                                = 0

exit_group(0)                           = ?

+++ exited with 0 +++

Discard everything and focus on this part:

Quote:getcwd("/home/patrick/will_disappear.bak", 4096) = 33
fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 32), ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5c75168000
write(1, "/home/patrick/will_disappear.bak"..., 33/home/patrick/will_disappear.bak
) = 33

It's doing the system call correctly and even writing on the screen the right current directory.
Now why do I see wrong results, I'm not sure...

I did the test with bash and zsh so it might not be shell dependent.

The behavior is more or less the same when user1 removes the directory user2 is in.

I think I've found the answer:

Code:
~/will_disappear > which pwd

pwd: shell built-in command

 ~/will_disappear > type pwd

pwd is a shell builtin

 ~/will_disappear > /bin/pwd

/bin/pwd: couldn't find directory entry in ‘..’ with matching i-node

Lesson, alway use the absolute path for your commands, don't trust the shell.

Some info on pwd
More info on inode
More info on virtual filesystem

What do you think?

	TheAnachron Offline \| 29-09-2016, 06:25 AM \| #2

One also has to keep in mind that if user2 moves the directory to a new disk, user1 can not do anything in this directory.

Read also about this (from wikipedia):

Quote:A file's inode number stays the same when it is moved to another directory on the same device, or when the disk is defragmented which may change its physical location. This also implies that completely conforming inode behavior is impossible to implement with many non-Unix file systems, such as FAT and its descendants, which don't have a way of storing this invariance when both a file's directory entry and its data are moved around.

Now if we have a FAT fs mounted and someone was to rename it, it might cause other links to break and/or the user currently being in it to not be able to access it anymore.

	movq Offline \| 30-09-2016, 09:59 AM \| #3

Nice question, I love little riddles like this. :-)

So, the point is, Bash itself is holding a reference to the current working directory -- but that reference is a string. Let's have a look at Bash's source code.

"builtins/cd.def": Both "cd" and "pwd" are defined here. IIUC, the function pwd_builtin() is what's being executed when you type "pwd". It uses a shortcut:

Code:
#define tcwd the_current_working_directory

  directory = tcwd ? (verbatim_pwd ? sh_physpath (tcwd, 0) : tcwd)

           : get_working_directory ("pwd");

It uses the value of "the_current_working_directory", if set. Aha. Below that, there's this piece of code:

Code:
/* Try again using getcwd() if canonicalization fails (for instance, if

     the file system has changed state underneath bash). */

  if ((tcwd && directory == 0) ||

      (posixly_correct && same_file (".", tcwd, (struct stat *)0, (struct stat *)0) == 0))

    {

      if (directory && directory != tcwd)

        free (directory);

      directory = resetpwd ("pwd");

    }

So, Bash tries to detect whether its current value of "tcwd" is still valid. It uses same_file() to do that, defined in "general.c". This checks whether "." and "tcwd" point to the same inode on the same device. Yes, of course they do. But that doesn't mean that the cwd's directory name is still valid!

-- edit: No, wait, I'm confused. "tcwd" is the old directory name, it's invalid. But strace shows that there are no calls to stat() anyway, so I'd assume the code doesn't even execute the call to same_file(). Probably because posixly_correct is not set.

Yeah, if I start a Bash with "POSIXLY_CORRECT=1 bash", then "pwd" shows the new directory name (and I see calls to stat() in strace).

This is indeed weird. Why not just call the syscall getcwd() and be done with it? Why the hazzle with holding a string (!) reference to the cwd? I suspect this is for historical reasons ...

	venam Offline \| 30-09-2016, 10:20 AM \| #4

(30-09-2016, 09:59 AM)vain Wrote: This is indeed weird. Why not just call the syscall getcwd() and be done with it? Why the hazzle with holding a string (!) reference to the cwd? I suspect this is for historical reasons ...

Probably a mean of caching.
Also, where is the environment variable PWD mentioned in the source, when you are at it.
PWD holds the current directory and is used by many shells.
Is it set on change but not used or is it actually used and the environment variables are the "database" of those shells?

	movq Offline \| 30-09-2016, 01:22 PM \| #5

(30-09-2016, 10:20 AM)venam Wrote: the environment variables are the "database" of those shells

It would appear that, sometimes, this is indeed the case. For example, in eval.c of Bash:

Code:
/* Send an escape sequence to emacs term mode to tell it the

   current working directory. */

static void

send_pwd_to_eterm ()

{

  char *pwd, *f;

  f = 0;

  pwd = get_string_value ("PWD");

  if (pwd == 0)

    f = pwd = get_working_directory ("eterm");

  fprintf (stderr, "\032/%s\n", pwd);

  free (f);

}

Or jobs.c:

Code:
/* Return the working directory for the current process.  Unlike

   job_working_directory, this does not call malloc (), nor do any

   of the functions it calls.  This is so that it can safely be called

   from a signal handler. */

static char *

current_working_directory ()

{

  char *dir;

  static char d[PATH_MAX];

  dir = get_string_value ("PWD");

  if (dir == 0 && the_current_working_directory && no_symbolic_links)

    dir = the_current_working_directory;

  if (dir == 0)

    {

      dir = getcwd (d, sizeof(d));

      if (dir)

    dir = d;

    }

  return (dir == 0) ? "<unknown>" : dir;

}

Funny thing is, I don't see $PWD being used in the "pwd" shell builtin. Instead, it calls get_working_directory() as mentioned in my previous post, which, in turn, calls the syscall getcwd().

In German, there's the nice expression, "das ist organisch gewachsen" (literal translation something like "grown in an organic manner"), which basically is a euphemism for "this is a very old code base and 1000 people have hacked on it and nobody ever cared to refactor things". :-)

	TheAnachron Offline \| 01-10-2016, 06:41 AM \| #6

You mean "Das ist historisch gewachsen." or?

	movq Offline \| 01-10-2016, 07:23 AM \| #7

That might be more correct, yes. I think we use both phrases at work. %)

View a Printable Version