[Bug libc/23249] New: Glibc is deliberately crippling non-Intel x64 CPUs

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] New: Glibc is deliberately crippling non-Intel x64 CPUs

glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

            Bug ID: 23249
           Summary: Glibc is deliberately crippling non-Intel x64 CPUs
           Product: glibc
           Version: 2.27
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: libc
          Assignee: unassigned at sourceware dot org
          Reporter: linux at carewolf dot com
                CC: drepper.fsp at gmail dot com
  Target Milestone: ---

Recently a "haswell" sub-arch was introduced to be similar to the old i686
subarch for x86. It is documented as requiring BMI1, BMI2, LZCNT, MOVBE,
POPCNT, AVX2 and FMA, but undocumented also checks the CPU is an Intel CPU
before using the faster paths.

Considering this is very similar to the old scandal of the Intel compiler, I
would suggest glibc fixes that before it becomes public knowledge.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Glibc is deliberately crippling non-Intel x64 CPUs

glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
                 CC|                            |fweimer at redhat dot com
         Resolution|---                         |WORKSFORME

--- Comment #1 from Florian Weimer <fweimer at redhat dot com> ---
We fix performance issues as they are identified, see bug 19467 for an example.

However, changes in this area require a deep understanding of CPU architecture
(and, preferably, future plans, so that deployed code does not take a
performance hit when switching to newer CPUs).  Just because something is
implemented at the instruction set level doesn't mean the implementation is
efficient.  For example, the first CPU generation with wider vector registers
often still uses old, narrower execution units, so not using the wider
registers can be more efficient.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Glibc is deliberately crippling non-Intel x64 CPUs

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #2 from Allan Jensen <linux at carewolf dot com> ---
What does that comment have to do with the fact that you are providing the
optimized path to all Intel CPUs including future ones if they implement the
necessary extensions, but deliberately do not give AMD processors the same fast
path?

Current gen AMD cpus are more likely to benefit from Haswell optimized binaries
than some future Intel chip, or mobile Intel chip.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Glibc is deliberately crippling non-Intel x64 CPUs

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #3 from Allan Jensen <linux at carewolf dot com> ---
And please note that I am talking about the picking up of 3rdparty libraries
would potentially to use this new sub-arch dir 'lib64/haswell', I am not
talking about glibc internal CPU scheduling which is completely separate and
very much optimized for every arch.

It is a specific line of code that is wrong here cpu_features.c:400.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Glibc is deliberately crippling non-Intel x64 CPUs

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #4 from Florian Weimer <fweimer at redhat dot com> ---
Okay, fell free to bring this up on libc-alpha.  Let's see what the Intel and
AMD maintainers think.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Glibc is deliberately crippling non-Intel x64 CPUs

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #5 from Florian Weimer <fweimer at redhat dot com> ---
FWIW, I verified that with glibc 2.28, the "haswell" subdirectories are not
searched on an AMD EPYC 7251 CPU.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Glibc is deliberately       |Epyc and other current AMD
                   |crippling non-Intel x64     |CPUs do not select the
                   |CPUs                        |"haswell" platform
                   |                            |subdirectory

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |arthur200126 at gmail dot com

--- Comment #6 from Florian Weimer <fweimer at redhat dot com> ---
*** Bug 24979 has been marked as a duplicate of this bug. ***

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Paul Menzel <pmenzel+sourceware.org-bugzilla at molgen dot mpg.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |pmenzel+sourceware.org-bugz
                   |                            |illa at molgen dot mpg.de

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Joey Riches <josephriches at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |josephriches at gmail dot com

--- Comment #7 from Joey Riches <josephriches at gmail dot com> ---
Sorry to necro an old thread but I wanted to know if there was any more
discussion around this?

I patched cpu-features.c to allow arch_kind_amd cpus to match the haswell
platform definition and have been getting good very good performance results
with the glibc-bench benchmarks as well as a haswell optimized openblas which
in turn improved results for R and octave (on a znver2 cpu).

It seems intel as already has done most of the hard work for amd here?

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

glaubitz at physik dot fu-berlin.de
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #8 from Florian Weimer <fweimer at redhat dot com> ---
(In reply to Joey Riches from comment #7)
> Sorry to necro an old thread but I wanted to know if there was any more
> discussion around this?

I'm still blocked until I get definitive feedback from AMD.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|WORKSFORME                  |---
           Assignee|unassigned at sourceware dot org   |fweimer at redhat dot com
   Last reconfirmed|                            |2020-04-22
             Status|RESOLVED                    |REOPENED
     Ever confirmed|0                           |1

--- Comment #9 from Florian Weimer <fweimer at redhat dot com> ---
I'm happy to report that I've been in contact with the right people at AMD for
a while.

I do not know yet what the exact outcome will be (if the “haswell” directory
will be used), but there will be a way to automatically load AVX2-optimized
libraries on AMD CPUs as well.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Severity|normal                      |enhancement

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

David Abdurachmanov <david.abdurachmanov at gmail dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |david.abdurachmanov at gmail dot c
                   |                            |om

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

Florian Weimer <fweimer at redhat dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |NEW

--- Comment #10 from Florian Weimer <fweimer at redhat dot com> ---
I surveyed the existing code and wrote a summary:

hwcaps subdirectory selection in the dynamic loader
<https://sourceware.org/pipermail/libc-alpha/2020-May/113757.html>

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #11 from Mingye Wang <arthur200126 at gmail dot com> ---
I agree that a "generational" revision scheme would be helpful in the long run.
For that information we can consult compiler databases
(gcc/gcc/common/config/i386/i386-common.c + gcc/config/i386/i386.h or
llvm/clang/include/clang/Basic/X86Target.def) and make some sort of table to
work with.

The LLVM project's version seems to already imply a generational scheme,
although they appear to be taking a lot of liberties by eliding a lot of the
stuff they don't use. They also don't seem to care about less-used stuff like
VIA.

The GCC database does include the VIA CPUs, but the part about eden-x4 not
having any sort of AVX is a bit dubious. Hell, their documentation mentions it
having AVX2...

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

tcl_de at gmx dot net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tcl_de at gmx dot net

--- Comment #12 from tcl_de at gmx dot net ---
Regarding VIA, the GCC onlinedocs say that eden-x4 supports AVX and AVX2, but
as you have pointed out, they are not enabled in GCC's i386-common.c. I have
had a look at CPUID dump from the instlatx64 project for an eden-x4 (see
http://users.atw.hu/instlatx64/CentaurHauls/CentaurHauls00006FE_CNR_Isaiah_CPUID.txt
) and if I decode it correctly, the GCC onlinedocs also miss some supported
instruction sets: MOVBE, POPCNT, AES, PCLMUL, FSGSBASE, RDRND, BMI, BMI2 and
F16C.
This means that all instruction sets listed for Haswell should be supported
except for FMA.
As far as I can tell, if RDRND is also removed from the list, this is the least
common denominator among all AVX2 cpus (technically bdver4 supports RDRND, but
Linux disables it by default due to buggy BIOS support, see
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c49a0a80137c7ca7d6ced4c812c9e07a949f6f24)

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #13 from Mingye Wang <arthur200126 at gmail dot com> ---
> Regarding VIA, the GCC onlinedocs say that eden-x4 supports AVX and AVX2, but as you have pointed out, they are not enabled in GCC's i386-common.c.

I have reported the problem to gcc as
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95030. Let me forward your cpuid
links.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #14 from tcl_de at gmx dot net ---
(In reply to Mingye Wang from comment #13)
> > Regarding VIA, the GCC onlinedocs say that eden-x4 supports AVX and AVX2, but as you have pointed out, they are not enabled in GCC's i386-common.c.
>
> I have reported the problem to gcc as
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95030. Let me forward your
> cpuid links.

Thanks for forwarding the information. I just realized that when I had a look
at the CPUID dump, I overlook something, as I only focussed on the instruction
sets list for haswell, but the eden-x4 actually supports a few more that are
not in that list: PREFETCHW, RDSEED, ADX, ABM, XSAVE, and maybe also XSAVEOPT
and XSAVEC, but I am not sure about the latter two.

--
You are receiving this mail because:
You are on the CC list for the bug.
Reply | Threaded
Open this post in threaded view
|

[Bug libc/23249] Epyc and other current AMD CPUs do not select the "haswell" platform subdirectory

Sourceware - glibc-bugs mailing list
In reply to this post by glaubitz at physik dot fu-berlin.de
https://sourceware.org/bugzilla/show_bug.cgi?id=23249

--- Comment #15 from Mingye Wang <arthur200126 at gmail dot com> ---
Created attachment 12811
  --> https://sourceware.org/bugzilla/attachment.cgi?id=12811&action=edit
What I think the discussion is going for

--
You are receiving this mail because:
You are on the CC list for the bug.
12