[PATCH] test-in-container: Install locales into the test container

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[PATCH] test-in-container: Install locales into the test container

Florian Weimer-5
The testing time impact of this change is significant: It increases
total test time from ~13 to ~15 minutes for me.  However, I do not
see any other way to run locale-dependent tests in a container.

2019-07-09  Florian Weimer  <[hidden email]>

        * Makefile (testroot.pristine/install.stamp): Install locales.

diff --git a/Makefile b/Makefile
index dc5de7aa6b..23c4326980 100644
--- a/Makefile
+++ b/Makefile
@@ -402,6 +402,7 @@ ifeq ($(run-built-tests),yes)
   done
 endif
  $(MAKE) install DESTDIR=$(objpfx)testroot.pristine
+ $(MAKE) localedata/install-locales DESTDIR=$(objpfx)testroot.pristine
  touch $(objpfx)testroot.pristine/install.stamp
 
 tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

DJ Delorie-2
Florian Weimer <[hidden email]> writes:
> The testing time impact of this change is significant: It increases
> total test time from ~13 to ~15 minutes for me.  However, I do not
> see any other way to run locale-dependent tests in a container.

This is also the only way to auto-test "make install-locales" too :-)

>   $(MAKE) install DESTDIR=$(objpfx)testroot.pristine
> + $(MAKE) localedata/install-locales DESTDIR=$(objpfx)testroot.pristine
>   touch $(objpfx)testroot.pristine/install.stamp

This matches what's documented in INSTALL so LGTM.

Reviewed-by: DJ Delorie <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

Florian Weimer-5
* DJ Delorie:

> Florian Weimer <[hidden email]> writes:
>> The testing time impact of this change is significant: It increases
>> total test time from ~13 to ~15 minutes for me.  However, I do not
>> see any other way to run locale-dependent tests in a container.
>
> This is also the only way to auto-test "make install-locales" too :-)
>
>>   $(MAKE) install DESTDIR=$(objpfx)testroot.pristine
>> + $(MAKE) localedata/install-locales DESTDIR=$(objpfx)testroot.pristine
>>   touch $(objpfx)testroot.pristine/install.stamp
>
> This matches what's documented in INSTALL so LGTM.
>
> Reviewed-by: DJ Delorie <[hidden email]>

The problem is that the delay also applies to partial runs of the test
suite:

$ time make -j8 subdirs=resolv check

goes from:

real    0m8.792s
user    0m15.486s
sys     0m6.017s

to:

real    2m5.588s
user    13m20.667s
sys     0m44.631s

Compare this to the time of a clean glibc build on this machine:

real    1m42.489s
user    8m20.556s
sys     2m6.281s

This is a bit embarrassing because the pristine container constructed in
this way is not even correct (bug 24794).

I think applying this patch as-is would make glibc development
substantially more difficult for me.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

DJ Delorie-2
Florian Weimer <[hidden email]> writes:
> The problem is that the delay also applies to partial runs of the test
> suite:

The first time, or each other time?

> $ time make -j8 subdirs=resolv check

This does (should :) create the pristine testroot, but only once.
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

Florian Weimer-5
* DJ Delorie:

> Florian Weimer <[hidden email]> writes:
>> The problem is that the delay also applies to partial runs of the test
>> suite:
>
> The first time, or each other time?

The first time.  It's still bad in some cases (depending on what I'm
working).

>> $ time make -j8 subdirs=resolv check
>
> This does (should :) create the pristine testroot, but only once.

Yes, and its contents stays incomplete.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

Carlos O'Donell-6
In reply to this post by Florian Weimer-5
On 7/9/19 4:52 PM, Florian Weimer wrote:
> I think applying this patch as-is would make glibc development
> substantially more difficult for me.

Where do you draw the line? Eventually the testing is going to
slow down your edit/build/test cycle. If this is a problem for
developers then we need some kind of objective "line" that we
try not to cross.

Should we have a staggered approach?

make check-sanity
  -- Run as many tests as fit into 10s.
  -- Run all tests which are not support_become_root,
     test-in-container, or xcheck tests, since these
     can be expensive.

make check
  -- No time limit for testing.
  -- Run all tests which are not xcheck tests.

make xhceck
  -- No time limit for testing.
  -- Run all tests including xcheck tests.

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

Partial test suite run builds corrupt test-in-container testroot

Carlos O'Donell-6
In reply to this post by Florian Weimer-5
On 7/9/19 5:02 PM, Florian Weimer wrote:
> * DJ Delorie:
>>> $ time make -j8 subdirs=resolv check
>>
>> This does (should :) create the pristine testroot, but only once.
>
> Yes, and its contents stays incomplete.

I'm changing the subject because this is a distinct htread.

Thanks for filling this:
https://sourceware.org/bugzilla/show_bug.cgi?id=24794

We should work to solve this, since it's a use case that many
developers expect to work.

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

DJ Delorie-2
In reply to this post by Carlos O'Donell-6
"Carlos O'Donell" <[hidden email]> writes:
> Where do you draw the line?

Originally I wanted the "make install" to trigger just before the first
test that actually needed it, but I found it very difficult to do that.
Eventually more and more tests will run inside the container, and that
optimization would be useless anyway.

Perhaps we could at least add $(PARALLELMFLAGS) to the "make install" ?
If it's not already propogated by some other means?  Or otherwise
optimize the install process, to limit the overhead during testing?
Reply | Threaded
Open this post in threaded view
|

Re: Partial test suite run builds corrupt test-in-container testroot

DJ Delorie-2
In reply to this post by Carlos O'Donell-6

I did a "make" followed by "make libio/tests" and compared with a "make;
make check" and they produced identical testroots (including iconvconfig).

I tried a "./configure; make libio/tests" and it failed pretty quickly
(shell-container not found) since support/ hadn't been built.

Please provide more detailed reproduction instructions, as I don't see a
"corrupt testroot" yet.
Reply | Threaded
Open this post in threaded view
|

Re: Partial test suite run builds corrupt test-in-container testroot

Florian Weimer-5
* DJ Delorie:

> I did a "make" followed by "make libio/tests" and compared with a "make;
> make check" and they produced identical testroots (including iconvconfig).
>
> I tried a "./configure; make libio/tests" and it failed pretty quickly
> (shell-container not found) since support/ hadn't been built.
>
> Please provide more detailed reproduction instructions, as I don't see a
> "corrupt testroot" yet.

You need to build the tree first before running the test suite subset.
For example:

  make -j8
  make -j8 check subdirs=libio

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: Partial test suite run builds corrupt test-in-container testroot

DJ Delorie-2
Florian Weimer <[hidden email]> writes:
> You need to build the tree first before running the test suite subset.
> For example:
>
>   make -j8
>   make -j8 check subdirs=libio

Yup, as I said:

>> I did a "make" followed by "make libio/tests" and compared with a "make;
>> make check" and they produced identical testroots (including iconvconfig).
Reply | Threaded
Open this post in threaded view
|

Re: Partial test suite run builds corrupt test-in-container testroot

Florian Weimer-5
* DJ Delorie:

> Florian Weimer <[hidden email]> writes:
>> You need to build the tree first before running the test suite subset.
>> For example:
>>
>>   make -j8
>>   make -j8 check subdirs=libio
>
> Yup, as I said:
>
>>> I did a "make" followed by "make libio/tests" and compared with a "make;
>>> make check" and they produced identical testroots (including iconvconfig).

Huh.  What's your make version?

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: Partial test suite run builds corrupt test-in-container testroot

DJ Delorie-2

Florian Weimer <[hidden email]> writes:
> Huh.  What's your make version?

4.2.1 - Fedora 30
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

Carlos O'Donell-6
In reply to this post by DJ Delorie-2
On 7/9/19 7:54 PM, DJ Delorie wrote:

> "Carlos O'Donell" <[hidden email]> writes:
>> Where do you draw the line?
>
> Originally I wanted the "make install" to trigger just before the first
> test that actually needed it, but I found it very difficult to do that.
> Eventually more and more tests will run inside the container, and that
> optimization would be useless anyway.
>
> Perhaps we could at least add $(PARALLELMFLAGS) to the "make install" ?
> If it's not already propogated by some other means?  Or otherwise
> optimize the install process, to limit the overhead during testing?

My point is that no matter how good you optimize, eventually there will
be enough tests that the time will creep up again.

I want Florian to be productive, and I want a build system that supports
a straight forward edit/build/test workflow.

My question is: How long is too long to wait for the results of testing
when you are doing a edit/build/test/debug cycle?

Should 'make check' be running enough tests to fill X seconds?

--
Cheers,
Carlos.
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-in-container: Install locales into the test container

Florian Weimer-5
In reply to this post by DJ Delorie-2
* DJ Delorie:

> Perhaps we could at least add $(PARALLELMFLAGS) to the "make install" ?
> If it's not already propogated by some other means?  Or otherwise
> optimize the install process, to limit the overhead during testing?

$(MAKE) already implies $(PARALLELMFLAGS).

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: Partial test suite run builds corrupt test-in-container testroot

Florian Weimer-5
In reply to this post by DJ Delorie-2
* DJ Delorie:

> Florian Weimer <[hidden email]> writes:
>> Huh.  What's your make version?
>
> 4.2.1 - Fedora 30

Here's what I did.  I got a box from Beaker with Fedora 30.

# dnf install screen
# dnf download --source glibc
# dnf builddep glibc-*.src.rpm
# screen
# dnf install git
# git clone --depth 1 git://sourceware.org/git/glibc.git git
# mkdir build
# cd build
# ../git/configure --prefix=/usr
# time nohup make -j24
# time nohup make -j24 subdirs=libio check

The output ends with:

FAIL: libio/tst-wfile-ascii
Summary of test results:
      1 FAIL
     87 PASS
make[1]: *** [Makefile:411: tests] Error 1
make[1]: Leaving directory '/root/git'
make: *** [Makefile:9: check] Error 2

The test log contains:

tst-wfile-ascii.c:37: numeric comparison failure
   left: 256 (0x100); from: system (iconvconfig)
  right: 0 (0x0); from: 0
error: 1 test failures

Locale settings (inherited via SSH):

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=en_US.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=en_US.UTF-8
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Root vs non-root doesn't make a difference.

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

[PATCH] test-container: Install with $(all-subdirs) [BZ #24794]

Tulio Magno Quites Machado Filho-2
Whenever a sub-make is created, it inherits the variable subdirs from its
parent.  This is also true when make check is called with a restricted
list of subdirs.  In this scenario, make install is executed "partially"
and testroot.pristine ends up with an incomplete installation.

2019-07-22  Tulio Magno Quites Machado Filho  <[hidden email]>

        [BZ #24794]
        * Makefile (testroot.pristine/install.stamp): Pass
        subdirs='$(all-subdirs)' to make install.
---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index a4ed747cef..9fbf705200 100644
--- a/Makefile
+++ b/Makefile
@@ -401,7 +401,8 @@ ifeq ($(run-built-tests),yes)
     $(test-wrapper) cp $$dso $(objpfx)testroot.pristine$$dso ;\
   done
 endif
- $(MAKE) install DESTDIR=$(objpfx)testroot.pristine
+ $(MAKE) install DESTDIR=$(objpfx)testroot.pristine \
+  subdirs='$(all-subdirs)'
  touch $(objpfx)testroot.pristine/install.stamp
 
 tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
--
2.14.5

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]

DJ Delorie-2

Tulio Magno Quites Machado Filho <[hidden email]> writes:
> Whenever a sub-make is created, it inherits the variable subdirs from its
> parent.  This is also true when make check is called with a restricted
> list of subdirs.  In this scenario, make install is executed "partially"
> and testroot.pristine ends up with an incomplete installation.

"obvious in retrospect"  Sigh.

> 2019-07-22  Tulio Magno Quites Machado Filho  <[hidden email]>
>
> [BZ #24794]
> * Makefile (testroot.pristine/install.stamp): Pass
> subdirs='$(all-subdirs)' to make install.

LGTM.  Thanks!

Reviewed-by: DJ Delorie <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]

Tulio Magno Quites Machado Filho-2
DJ Delorie <[hidden email]> writes:

> Tulio Magno Quites Machado Filho <[hidden email]> writes:
>> Whenever a sub-make is created, it inherits the variable subdirs from its
>> parent.  This is also true when make check is called with a restricted
>> list of subdirs.  In this scenario, make install is executed "partially"
>> and testroot.pristine ends up with an incomplete installation.
>
> "obvious in retrospect"  Sigh.
>
>> 2019-07-22  Tulio Magno Quites Machado Filho  <[hidden email]>
>>
>> [BZ #24794]
>> * Makefile (testroot.pristine/install.stamp): Pass
>> subdirs='$(all-subdirs)' to make install.
>
> LGTM.  Thanks!
>
> Reviewed-by: DJ Delorie <[hidden email]>

Pushed as 35e038c1d2ccb.

Thanks!

--
Tulio Magno
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] test-container: Install with $(all-subdirs) [BZ #24794]

Stefan Liebler-2
Hi,

starting with this commit, I see the following fail:
cat nss/tst-nss-files-hosts-long.out
error: tst-nss-files-hosts-long.c:35: ahostsv4 failed
error: 1 test failures

Does anybody else also see this fail?

While running "make check" I've recognized the output of
support/shell-container.c:
sh: execing getent failed: No such file or directory File
sh: execing /usr/bin/makedb failed: No such file or directory File

And indeed getent does not exist in <build-dir>/testroot.root or
<build-dir>/testroot.pristine. Without this commit, it was available.

In older logs, I've found these commands:
/usr/bin/install -c build/nss/getent
build/testroot.pristine/usr/bin/getent.new
/usr/bin/install -c build/nss/makedb
build/testroot.pristine/usr/bin/makedb.new
mv -f build/testroot.pristine/usr/bin/getent.new
build/testroot.pristine/usr/bin/getent
mv -f build/testroot.pristine/usr/bin/makedb.new
build/testroot.pristine/usr/bin/makedb

Can anybody help?

Bye
Stefan

On 7/23/19 6:29 PM, Tulio Magno Quites Machado Filho wrote:

> DJ Delorie <[hidden email]> writes:
>
>> Tulio Magno Quites Machado Filho <[hidden email]> writes:
>>> Whenever a sub-make is created, it inherits the variable subdirs from its
>>> parent.  This is also true when make check is called with a restricted
>>> list of subdirs.  In this scenario, make install is executed "partially"
>>> and testroot.pristine ends up with an incomplete installation.
>>
>> "obvious in retrospect"  Sigh.
>>
>>> 2019-07-22  Tulio Magno Quites Machado Filho  <[hidden email]>
>>>
>>> [BZ #24794]
>>> * Makefile (testroot.pristine/install.stamp): Pass
>>> subdirs='$(all-subdirs)' to make install.
>>
>> LGTM.  Thanks!
>>
>> Reviewed-by: DJ Delorie <[hidden email]>
>
> Pushed as 35e038c1d2ccb.
>
> Thanks!
>

12