Issue 17375 - colliding modules detected with binutils 2.28 linker and shared libraries
Summary: colliding modules detected with binutils 2.28 linker and shared libraries
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: druntime (show other issues)
Version: D2
Hardware: x86_64 Linux
: P1 critical
Assignee: No Owner
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-05-06 17:49 UTC by Martin Nowak
Modified: 2017-12-16 15:09 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this issue.
Description Martin Nowak 2017-05-06 17:49:56 UTC
Happens with shared libraries on Arch Linux w/ binutils 2.28.
Apparently copy relocated moduleinfos now end up as data symbols instead of bss.
Comment 1 Vladimir Panteleev 2017-05-06 20:04:06 UTC
On Arch Linux, this is reproducible with binutils 2.28.0, but not reproducible with 2.27.
Comment 2 Vladimir Panteleev 2017-05-07 00:02:45 UTC
Bisecting binutils points to:

9acc85a62eb76c270724bba15c889d2d05567b6a is the first bad commit
commit 9acc85a62eb76c270724bba15c889d2d05567b6a
Author: Alan Modra <amodra@gmail.com>
Date:   Wed Dec 28 17:04:15 2016 +1030

    Use dynrelro for symbols in relro sections too
    
            PR ld/20995
    bfd/
            * elflink.c (elf_link_add_object_symbols): Mark relro sections
            in dynamic objects SEC_READONLY.
    ld/
            * testsuite/ld-elf/pr20995c.s: New test file.
            * testsuite/ld-elf/pr20995-2so.r: Likewise.
            * testsuite/ld-elf/elf.exp: Run it.

:040000 040000 decd212336814caf0562835700a42a8f6d3e5e34 8f7d306bf5df00e27fa208f6dc66e345ca5064e8 M      bfd
:040000 040000 c101f8812c1c65893d81807818e3bfb8acd3ae1e decd54f18574d1b7d39c8888ff551cbde3d3cbe0 M      ld
Comment 3 Martin Nowak 2017-05-08 23:07:03 UTC
Also see https://sourceware.org/bugzilla/show_bug.cgi?id=20995 and http://sourceware-org.1504.n7.nabble.com/dynrelro-section-for-read-only-dynamic-symbols-copied-into-executable-td429858.html for more info on the binutils change.

As a workaround, also compile the executable with -fPIC (PIE). This will avoid the necessity for copy relocation. We made PIE the default in a recent release.
Comment 4 Vladimir Panteleev 2017-05-09 11:22:32 UTC
Workaround for Arch Linux:

Grab the 2.27 package from https://archive.archlinux.org/packages/b/binutils/binutils-2.27-1-x86_64.pkg.tar.xz, unpack it somewhere, then prefix $PATH with that package's "bin" directory and $LD_LIBRARY_PATH with that package's "lib" directory.
Comment 5 Martin Nowak 2017-05-18 18:05:01 UTC
The PIE (-fPIC) workaround seems a lot simpler. It should be made default on Arch anyhow after we've switched with 2.072.2 http://forum.dlang.org/post/o3sr68$2shj$1@digitalmars.com.
Comment 6 Vladimir Panteleev 2017-05-18 18:35:50 UTC
(In reply to Martin Nowak from comment #5)
> The PIE (-fPIC) workaround seems a lot simpler.

Sorry, yeah, I meant when modifying the build command is not feasible (e.g. when bisecting / doing regression testing).

Do you think you could submit a PR to fix the Druntime and Phobos Makefiles? I'd also like to make Digger apply that patch automatically so it's possible to run tests of older versions of D with newer binutils.
Comment 7 Martin Nowak 2017-06-16 18:47:12 UTC
Maybe you had a different git repo, my bisect ended at commit https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=9acc85a62eb76c270724bba15c889d2d05567b6a.

We could solve this by emitting the ModuleInfos themselves (not only a pointer to them) to their own sections and using the old section bracket trick (__start_minfo and __stop_minfo see https://github.com/dlang/dmd/blob/eb3bc8d8902a0b0c3b3b02058265d811ec310eb9/src/ddmd/backend/elfobj.c#L3319).

Would need 2 sections if some of them are readonly and some are relro though.
Comment 8 Vladimir Panteleev 2017-06-16 23:24:24 UTC
(In reply to Martin Nowak from comment #7)
> Maybe you had a different git repo, my bisect ended at commit
> https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;
> h=9acc85a62eb76c270724bba15c889d2d05567b6a.

No idea what you mean; that's the same commit I mentioned in comment 2 above.
Comment 9 Seb 2017-07-09 20:58:04 UTC
> It should be made default on Arch anyhow after we've switched with 2.072.2

Sadly it's not the default on Arch yet, see e.g.

http://forum.dlang.org/post/gpbyhmdsudlmapsvvqmq@forum.dlang.org

and my proposal to make it default for all Posix distros:

https://github.com/dlang/phobos/pull/5586
Comment 10 Thomas Mader 2017-07-31 07:48:27 UTC
Had the same problem with binutils 2.28 and dmd 2.075.0 on NixOS with druntime and phobos.

Working around the problem as suggested with

      make -f posix.mak PIC=-fPIC INSTALL_DIR=$out DMD=$DMD

works for phobos.

But for druntime I needed to patch a Makefile, 

    substituteInPlace druntime/test/common.mak \
        --replace "DFLAGS:=" "DFLAGS:=-fPIC "

otherwise the link test in the druntime unittests would fail when running with

      make -f posix.mak unittest PIC=-fPIC DMD=$DMD BUILD=release
Comment 11 github-bugzilla 2017-09-18 16:16:11 UTC
Commits pushed to master at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/3d36e434b84cf6ebd7341aaffe2b3b0b588a0262
Issue 17375 - use -fPIC by default for Phobos on Posix

This also fixes:

Fix Issue 17107 - Running phobos unittests do not work with PIE

https://github.com/dlang/phobos/commit/d400af081aad1dcb735170a99f9f8d1562788db9
Merge pull request #5586 from wilzbach/fpic-by-default

Issue 17375 - use -fPIC by default for Phobos on Posix
Comment 12 github-bugzilla 2017-10-16 09:58:00 UTC
Commits pushed to stable at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/3d36e434b84cf6ebd7341aaffe2b3b0b588a0262
Issue 17375 - use -fPIC by default for Phobos on Posix

https://github.com/dlang/phobos/commit/d400af081aad1dcb735170a99f9f8d1562788db9
Merge pull request #5586 from wilzbach/fpic-by-default
Comment 13 Seb 2017-12-13 08:57:24 UTC
The colliding module detection has been removed: https://github.com/dlang/druntime/pull/1921
Comment 14 Marco Leise 2017-12-16 15:00:12 UTC
If that detection has been removed, does this mean we can revert to compiling without -fPIC again? It's slows down code on x86. I put it in dmd.conf for Gentoo by default, because shared libraries would not load due to this bug. So can we revert to the pre binutils 2.28 procedures now for druntime, Phobos, dmd.conf?
Comment 15 Seb 2017-12-16 15:09:47 UTC
-fPIC is only the default for x86_64 on Linux and required due to PIE hardening on many distributions.