ld and GNU/Linux ld.so disagree on dependent library search order (?)

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

ld and GNU/Linux ld.so disagree on dependent library search order (?)

Jeremy Brown-3
We were bitten today by how GNU ld searches for dependent shared
libraries, and I was wondering if anyone could explain why it is done
the way it is.

The thing that got us is that when ld is looking up dependent
libraries, it will search a variety of locations, with the final two
sets being first, the system directories /lib and /usr/lib; and second
and finally, in directories listed in /etc/ld.so.conf [1]

To me, it seems backward for ld to first look in the system
directoreies, and only then look in custom-specified directories.  Why
is it done this way?

The way we got bit is that on GNU/Linux (Fedora), ld.so will look for
shared libraries in a variety of places, with the final two places
searched being, first, entries cached in /etc/ld.so.cache; and second
(and finally), /lib and then /usr/lib.  [2]

Because ld and ld.so disagree, we were forced to use -rpath-link just
to get ld to use the same lookup strategy at link-time as ld.so will
use at runtime --- and until we figured out the conflicting
search-path problem, we were having a really hard time understanding
why our links were failling.

Are we missing something that would simplify headache this away?

Many thanks,
Jeremy


------------------------------------------------------------
[1]:  From the GNU ld info page documentation on -rpath-link:

       7. The default directories, normally `/lib' and `/usr/lib'.

       8. For a native linker on an ELF system, if the file
          `/etc/ld.so.conf' exists, the list of directories found in
          that file.

------------------------------------------------------------
[2]:  From the GNU/Linux man page on ld.so (Fedora):

       o      From  the  cache file /etc/ld.so.cache which contains a compiled
              list of candidate libraries previously found  in  the  augmented
              library  path.  If, however, the binary was linked with -z node-
              flib linker option, libraries in the default library  paths  are
              skipped.

       o      In  the default path /lib, and then /usr/lib.  If the binary was
              linked with -z nodeflib linker option, this step is skipped.
Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

Ian Lance Taylor
Jeremy Brown <[hidden email]> writes:

> The thing that got us is that when ld is looking up dependent
> libraries, it will search a variety of locations, with the final two
> sets being first, the system directories /lib and /usr/lib; and second
> and finally, in directories listed in /etc/ld.so.conf [1]

ld and ld.so should use the same search path for shared libraries.
Anything else is a bug.  It sounds like you have found a bug, or else
the ld.so behaviour has changed.

Ian
Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

Jeremy Brown-3
Ian Lance Taylor <[hidden email]> writes:
> ld and ld.so should use the same search path for shared libraries.
> Anything else is a bug.  It sounds like you have found a bug, or else
> the ld.so behaviour has changed.

Certainly both seem to be behaving as they are documented to; it's
just that they are documented as behaving differently.

At the bottom of this email is a sequence of commands that illustrates
the problem on my Fedora core 3 x86_64 box.  It relies on the implicit
dependency of libgnome-2 on libxml2.  It needs to be run as root,
because it adjusts (and then resets) ld.so.conf.

Here's the most interesting part of the output.  At this point,
libxml2 has been copied into /usr/lib/test, and /usr/lib/test is the
first entry in /etc/ld.so.conf.

1 sh-3.00#  gcc -Xlinker -verbose foo.c -o foo -lgnome-2 2>&1 | grep libxml2
2 libxml2.so.2 needed by /usr/lib64/libgnomevfs-2.so.0
3 found libxml2.so.2 at /usr/lib64/libxml2.so.2  
4 sh-3.00#  ldd foo | grep libxml2
5         libxml2.so.2 => /usr/lib/test/libxml2.so.2 (0x00002aaaac16b000)
6 sh-3.00#  LD_DEBUG=libs ./foo 2>&1 | grep libxml2
7      24661:     find library=libxml2.so.2 [0]; searching
8      24661:       trying file=/usr/lib/test/libxml2.so.2
9      24661:     calling init: /usr/lib/test/libxml2.so.2
10     24661:     calling fini: /usr/lib/test/libxml2.so.2 [0]

Note that at line 3, the linker reports finding libxml2 at
/usr/lib64/libxml2, while at line 5, ldd finds it in /usr/lib/test/ As
another check, lines 8-10 show that ld.so also finds it in
/usr/lib/test  

As I said, this is consistent with the way both ld and ld.so are
documented, but very, very annoying.  My opinion is that ld is doing
the wrong thing here: it should favor paths in /etc/ld.so.conf ahead
of hard-coded system paths.

Should I report this as a bug?

Thanks for your time,
Jeremy



------------------------------------------------------------
 mkdir -p /usr/lib/test
 echo '/usr/lib/test' | cat - /etc/ld.so.conf > /etc/ld.so.conf.new
 mv /etc/ld.so.conf /etc/ld.so.conf.old
 mv /etc/ld.so.conf.new /etc/ld.so.conf
 cd /usr/lib/test
 cp -s /usr/lib64/libxml2.* .
 ldconfig
 cd /tmp
 echo 'int main(int argc, char **argv) { return 0;}' > foo.c
 gcc -Xlinker -verbose foo.c -o foo -lgnome-2 2>&1 | grep libxml2
 ldd foo | grep libxml2
 LD_DEBUG=libs ./foo 2>&1 | grep libxml2
 gcc -Xlinker -rpath-link /usr/lib/test -Xlinker -verbose foo.c -o foo -lgnome-2 2>&1 | grep libxml2
 ldd foo | grep libxml2
 mv /etc/ld.so.conf.old /etc/ld.so.conf
 rm -r /usr/lib/test
 ldconfig

Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

H.J. Lu-27
On Mon, Feb 06, 2006 at 12:42:52PM -0500, Jeremy Brown wrote:

> Ian Lance Taylor <[hidden email]> writes:
> > ld and ld.so should use the same search path for shared libraries.
> > Anything else is a bug.  It sounds like you have found a bug, or else
> > the ld.so behaviour has changed.
>
> Certainly both seem to be behaving as they are documented to; it's
> just that they are documented as behaving differently.
>
> At the bottom of this email is a sequence of commands that illustrates
> the problem on my Fedora core 3 x86_64 box.  It relies on the implicit
> dependency of libgnome-2 on libxml2.  It needs to be run as root,
> because it adjusts (and then resets) ld.so.conf.
>
> Here's the most interesting part of the output.  At this point,
> libxml2 has been copied into /usr/lib/test, and /usr/lib/test is the
> first entry in /etc/ld.so.conf.
>
> 1 sh-3.00#  gcc -Xlinker -verbose foo.c -o foo -lgnome-2 2>&1 | grep libxml2
> 2 libxml2.so.2 needed by /usr/lib64/libgnomevfs-2.so.0
> 3 found libxml2.so.2 at /usr/lib64/libxml2.so.2  
> 4 sh-3.00#  ldd foo | grep libxml2
> 5         libxml2.so.2 => /usr/lib/test/libxml2.so.2 (0x00002aaaac16b000)
> 6 sh-3.00#  LD_DEBUG=libs ./foo 2>&1 | grep libxml2
> 7      24661:     find library=libxml2.so.2 [0]; searching
> 8      24661:       trying file=/usr/lib/test/libxml2.so.2
> 9      24661:     calling init: /usr/lib/test/libxml2.so.2
> 10     24661:     calling fini: /usr/lib/test/libxml2.so.2 [0]
>
> Note that at line 3, the linker reports finding libxml2 at
> /usr/lib64/libxml2, while at line 5, ldd finds it in /usr/lib/test/ As
> another check, lines 8-10 show that ld.so also finds it in
> /usr/lib/test  
>
> As I said, this is consistent with the way both ld and ld.so are
> documented, but very, very annoying.  My opinion is that ld is doing
> the wrong thing here: it should favor paths in /etc/ld.so.conf ahead
> of hard-coded system paths.
>
> Should I report this as a bug?
>
> Thanks for your time,
> Jeremy
>

Please open a bug report with a testcase. I will look into it.

Thanks.


H.J.
Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

Jeremy Brown-3
Done.  Bug 2290 is now open.  Please let me know if you need more
information.

Thanks,
Jeremy

Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

H.J. Lu-27
On Mon, Feb 06, 2006 at 06:25:24PM -0500, Jeremy Brown wrote:
> Done.  Bug 2290 is now open.  Please let me know if you need more
> information.
>

Can you try this patch?

Thanks.


H.J.
---
2006-02-07  H.J. Lu  <[hidden email]>

        PR ld/2290
        * emultempl/elf32.em (gld${EMULATION_NAME}_after_open): Call
        gld${EMULATION_NAME}_check_ld_so_conf before checking default
        search directories for DT_NEEDED entries.

--- ld/emultempl/elf32.em.ld.so 2005-12-08 07:06:38.000000000 -0800
+++ ld/emultempl/elf32.em 2006-02-07 08:33:46.000000000 -0800
@@ -919,6 +919,18 @@ cat >>e${EMULATION_NAME}.c <<EOF
 
 EOF
 fi
+if [ "x${USE_LIBPATH}" = xyes ] ; then
+  case ${target} in
+    *-*-linux-* | *-*-k*bsd*-*)
+    # Linux
+      cat >>e${EMULATION_NAME}.c <<EOF
+  if (gld${EMULATION_NAME}_check_ld_so_conf (l->name, force))
+    break;
+
+EOF
+    ;;
+  esac
+fi
 cat >>e${EMULATION_NAME}.c <<EOF
   len = strlen (l->name);
   for (search = search_head; search != NULL; search = search->next)
@@ -937,17 +949,6 @@ cat >>e${EMULATION_NAME}.c <<EOF
   if (search != NULL)
     break;
 EOF
-if [ "x${USE_LIBPATH}" = xyes ] ; then
-  case ${target} in
-    *-*-linux-* | *-*-k*bsd*-*)
-      cat >>e${EMULATION_NAME}.c <<EOF
-  if (gld${EMULATION_NAME}_check_ld_so_conf (l->name, force))
-    break;
-EOF
-    # Linux
-    ;;
-  esac
-fi
 cat >>e${EMULATION_NAME}.c <<EOF
  }
 
Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

Alan Modra
On Tue, Feb 07, 2006 at 08:42:22AM -0800, H. J. Lu wrote:
> PR ld/2290
> * emultempl/elf32.em (gld${EMULATION_NAME}_after_open): Call
> gld${EMULATION_NAME}_check_ld_so_conf before checking default
> search directories for DT_NEEDED entries.

OK.  A NEWS entry would be nice too.

--
Alan Modra
IBM OzLabs - Linux Technology Centre
Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

Jeremy Brown-3
In reply to this post by H.J. Lu-27
I'll give it a shot.  Do I have a prayer working from the SRPMs, or do
I need to use the CVS head?

Thanks,
Jeremy


"H. J. Lu" <[hidden email]> writes:

> On Mon, Feb 06, 2006 at 06:25:24PM -0500, Jeremy Brown wrote:
>> Done.  Bug 2290 is now open.  Please let me know if you need more
>> information.
>>
>
> Can you try this patch?
>
> Thanks.
>
>
> H.J.
> ---
> 2006-02-07  H.J. Lu  <[hidden email]>
>
> PR ld/2290
> * emultempl/elf32.em (gld${EMULATION_NAME}_after_open): Call
> gld${EMULATION_NAME}_check_ld_so_conf before checking default
> search directories for DT_NEEDED entries.
>
> --- ld/emultempl/elf32.em.ld.so 2005-12-08 07:06:38.000000000 -0800
> +++ ld/emultempl/elf32.em 2006-02-07 08:33:46.000000000 -0800
> @@ -919,6 +919,18 @@ cat >>e${EMULATION_NAME}.c <<EOF
>  
>  EOF
>  fi
> +if [ "x${USE_LIBPATH}" = xyes ] ; then
> +  case ${target} in
> +    *-*-linux-* | *-*-k*bsd*-*)
> +    # Linux
> +      cat >>e${EMULATION_NAME}.c <<EOF
> +  if (gld${EMULATION_NAME}_check_ld_so_conf (l->name, force))
> +    break;
> +
> +EOF
> +    ;;
> +  esac
> +fi
>  cat >>e${EMULATION_NAME}.c <<EOF
>    len = strlen (l->name);
>    for (search = search_head; search != NULL; search = search->next)
> @@ -937,17 +949,6 @@ cat >>e${EMULATION_NAME}.c <<EOF
>    if (search != NULL)
>      break;
>  EOF
> -if [ "x${USE_LIBPATH}" = xyes ] ; then
> -  case ${target} in
> -    *-*-linux-* | *-*-k*bsd*-*)
> -      cat >>e${EMULATION_NAME}.c <<EOF
> -  if (gld${EMULATION_NAME}_check_ld_so_conf (l->name, force))
> -    break;
> -EOF
> -    # Linux
> -    ;;
> -  esac
> -fi
>  cat >>e${EMULATION_NAME}.c <<EOF
>   }
>  

Reply | Threaded
Open this post in threaded view
|

Re: ld and GNU/Linux ld.so disagree on dependent library search order (?)

H.J. Lu-27
In reply to this post by Alan Modra
On Thu, Feb 09, 2006 at 09:51:20AM +1030, Alan Modra wrote:
> On Tue, Feb 07, 2006 at 08:42:22AM -0800, H. J. Lu wrote:
> > PR ld/2290
> > * emultempl/elf32.em (gld${EMULATION_NAME}_after_open): Call
> > gld${EMULATION_NAME}_check_ld_so_conf before checking default
> > search directories for DT_NEEDED entries.
>
> OK.  A NEWS entry would be nice too.
>

I am going to check this in shortly.

Thanks.


H.J.
----
2006-02-07  H.J. Lu  <[hidden email]>

        PR ld/2290
        * NEWS: Updated for the Linux linker search order change.

        * emultempl/elf32.em (gld${EMULATION_NAME}_after_open): Call
        gld${EMULATION_NAME}_check_ld_so_conf before checking default
        search directories for DT_NEEDED entries.

--- ld/NEWS.ld.so 2006-02-01 09:37:06.000000000 -0800
+++ ld/NEWS 2006-02-08 16:10:10.000000000 -0800
@@ -1,5 +1,8 @@
 -*- text -*-
 
+* Modify the Linux linker to seach /etc/ld.so.conf first before
+  checking default search directories for DT_NEEDED entries.
+
 * PE-COFF: Forward exports from DLL's can now be specified in .def files
   passed directly to ld.
 
--- ld/emultempl/elf32.em.ld.so 2005-12-08 07:06:38.000000000 -0800
+++ ld/emultempl/elf32.em 2006-02-08 16:08:55.000000000 -0800
@@ -919,6 +919,18 @@ cat >>e${EMULATION_NAME}.c <<EOF
 
 EOF
 fi
+if [ "x${USE_LIBPATH}" = xyes ] ; then
+  case ${target} in
+    *-*-linux-* | *-*-k*bsd*-*)
+    # Linux
+      cat >>e${EMULATION_NAME}.c <<EOF
+  if (gld${EMULATION_NAME}_check_ld_so_conf (l->name, force))
+    break;
+
+EOF
+    ;;
+  esac
+fi
 cat >>e${EMULATION_NAME}.c <<EOF
   len = strlen (l->name);
   for (search = search_head; search != NULL; search = search->next)
@@ -937,17 +949,6 @@ cat >>e${EMULATION_NAME}.c <<EOF
   if (search != NULL)
     break;
 EOF
-if [ "x${USE_LIBPATH}" = xyes ] ; then
-  case ${target} in
-    *-*-linux-* | *-*-k*bsd*-*)
-      cat >>e${EMULATION_NAME}.c <<EOF
-  if (gld${EMULATION_NAME}_check_ld_so_conf (l->name, force))
-    break;
-EOF
-    # Linux
-    ;;
-  esac
-fi
 cat >>e${EMULATION_NAME}.c <<EOF
  }