bug#19616: dist tarball contains hardlinks

[Dimitrios Apostolou]

(Cross posting to both bug-automake and bug-tar as I can see both projects 
being affected.)
(Please keep me CC'd as I'm not subscribed.)

Hello,

this bug report is because of the following messages when extracting a 
source tarball generated with "make dist". You can read the full bug 
report at https://dev.cfengine.com/issues/6925.

# tar -xzf core/cfengine-3.7.0a1.5ffcc54.tar.gz
tar: 
cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_modify_user_with_same_password.cf:
 Cannot hard link to 
`cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_with_same_password_hpux_tru':
 No such file or directory
tar: 
cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_modify_user_with_many_attributes_warn.cf:
 Cannot hard link to 
`cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_with_many_attributes_warn_h':
 No such file or directory
tar: 
cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_add_user_with_many_attributes_warn.cf:
 Cannot hard link to 
`cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_add_user_with_many_attributes_warn_hpux':
 No such file or directory
tar: 
cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_modify_user_lock_with_password.cf:
 Cannot hard link to 
`cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_lock_with_password_hpux_tru':
 No such file or directory
tar: 
cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_newly_created_account_should_not_count_as_locked_hpux_trusted.cf:
 Cannot hard link to 
`cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_newly_created_account_should_not_count_':
 No such file or directory
tar: 
cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_modify_user_unlock_with_password.cf:
 Cannot hard link to 
`cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_modify_user_unlock_with_password_hpux_t':
 No such file or directory
tar: 
cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/10_add_user_with_hashed_password.cf:
 Cannot hard link to 
`cfengine-3.7.0a1.5ffcc54/tests/acceptance/17_users/unsafe/20_add_user_with_hashed_password_hpux_trus':
 No such file or directory
tar: Error exit delayed from previous errors


Regading automake, it implements "make dist" using "tar -chf" where -h 
stands for --dereference, i.e. it dereferences all symlinks. I believe 
I've read that it is for portability, because symlinks are handled in a 
variety of manners in different tar implementations. However as seen 
above, what actually happens is that GNU tar stores the symlinks as 
hardlinks. Because the total path length of the hardlinks is more than 100 
characters, they get truncated due to "ustar" format limitations (ustar 
format is chosen using the "tar-ustar" automake option).

Please notice that removing "-h" fixes the problem. Symlinks are stored 
properly as their length is much smaller than 100 chars (because they are 
relative symlinks). Another workaround (the one I chose to use) is to pass 
the --hard-dereference to GNU tar in order to store the content of the 
files itself, avoiding all kind of links, and maintaining best 
portability. So I've added the following lines in my Makefile.am:

TAR_OPTIONS = --hard-dereference
export TAR_OPTIONS


Regarding GNU tar, I've read at [1] and [2] that this behaviour was 
introduced recently, but is the intended behaviour.

[1] https://bugs.launchpad.net/ubuntu/+source/tar/+bug/887107
[2] http://lists.gnu.org/archive/html/bug-tar/2011-11/msg00025.html

My opinion is that it's really deceiving, given what the man page 
mentions: "follow symlinks; archive and dump the files they point to". It 
took me a while to figure out what those "hard link" messages mean, 
looking if I introduced hard links by accident. And since the hard links 
have no direction, some messages show the linking direction reversed, 
which made the issue even stranger.

Why is such behaviour desirable? Are the reasons important enough to break 
expectations of users?


Thank you in advance,
Dimitris