Replace conformtest.pl with conformtest.py [committed]

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Replace conformtest.pl with conformtest.py [committed]

Joseph Myers
Continuing the consolidation on Python for various miscellaneous build
and test scripts, this patch moves conformtest from Perl to Python.

The substance of the tests run is intended to be the same as before,
except that the previous test for tags did not actually achieve the
intended purpose of verifying whether a tag was already declared, so
is changed to one that would actually fail for a tag that wasn't
declared, and a typo in the old test for variables being available
($xyzzy instead of xyzzy) would have made that test not use the
correct type (but it would have passed anyway with warnings).  No
attempt is made to keep the details of what the test output looks
like; instead, tests are given names which are made to follow PASS: /
FAIL: / XFAIL: / SKIP: / MISSING: as appropriate.

In the new version, there is more consistent parsing of test lines
(into a series of words, either surrounded by {} or separated by
spaces) that applies for all kinds of test lines, rather than the old
approach of different regular expressions for every kind of test.  A
few of the conform/data/ files are adjusted so their syntax works with
the new script (which now requires spaces in certain cases where the
old script tolerated them being missing, and does not allow stray
semicolons at the end of "function" lines).  Similarly, common logic
is used around what happens with a second subtest if a first one fails
(e.g., a test for a symbol's type if the test for availability fails),
rather than such logic being replicated separately for each kind of
test.  Common parsing also applies for test lines both when they are
lines for the header under test and when they are lines for another
header specified with allow-header, again unlike the old script.

Tested for x86_64, and with build-many-glibcs.py.  Committed.

2018-11-09  Joseph Myers  <[hidden email]>

        * conform/conformtest.py: New file.
        * conform/conformtest.pl: Remove.
        * conform/GlibcConform.pm: Likewise.
        * conform/glibcconform.py (KEYWORDS_C90): New constant.
        (KEYWORDS_C99): Likewise.
        (KEYWORDS): Likewise.
        * conform/Makefile ($(conformtest-header-tests)): Use
        conformtest.py instead of conformtest.pl.  Do not pass --tmpdir
        option.  Use --header instead of --headers.
        * conform/data/arpa/inet.h-data: Remove trailing semicolons on
        function entries.
        * conform/data/spawn.h-data: Likewise.
        * conform/data/fcntl.h-data (openat): Add space after function
        name.
        * conform/data/wchar.h-data (wcscasecmp): Likewise.
        (wcscasecmp_l): Likewise.
        * conform/data/termios.h-data (c_cc): Add space after element
        name.

diff --git a/conform/GlibcConform.pm b/conform/GlibcConform.pm
deleted file mode 100644
index ba9c7e822f..0000000000
--- a/conform/GlibcConform.pm
+++ /dev/null
@@ -1,74 +0,0 @@
-#!/usr/bin/perl
-
-# Shared code for glibc conformance tests.
-
-# Copyright (C) 2014-2018 Free Software Foundation, Inc.
-# This file is part of the GNU C Library.
-
-# The GNU C Library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-
-# The GNU C Library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-# Lesser General Public License for more details.
-
-# You should have received a copy of the GNU Lesser General Public
-# License along with the GNU C Library; if not, see
-# <http://www.gnu.org/licenses/>.
-
-package GlibcConform;
-require Exporter;
-@ISA = qw(Exporter);
-@EXPORT = qw(%CFLAGS list_exported_functions);
-
-# Compiler options for each standard.
-$CFLAGS{"ISO"} = "-ansi";
-$CFLAGS{"ISO99"} = "-std=c99";
-$CFLAGS{"ISO11"} = "-std=c11";
-$CFLAGS{"POSIX"} = "-D_POSIX_C_SOURCE=199506L -ansi";
-$CFLAGS{"XPG4"} = "-ansi -D_XOPEN_SOURCE";
-$CFLAGS{"XPG42"} = "-ansi -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED";
-$CFLAGS{"UNIX98"} = "-ansi -D_XOPEN_SOURCE=500";
-$CFLAGS{"XOPEN2K"} = "-std=c99 -D_XOPEN_SOURCE=600";
-$CFLAGS{"XOPEN2K8"} = "-std=c99 -D_XOPEN_SOURCE=700";
-$CFLAGS{"POSIX2008"} = "-std=c99 -D_POSIX_C_SOURCE=200809L";
-
-# Return a list of functions exported by a header, empty if an include
-# of the header does not compile.
-sub list_exported_functions {
-  my ($cc, $standard, $header, $tmpdir) = @_;
-  my ($cc_all) = "$cc -D_ISOMAC $CFLAGS{$standard}";
-  my ($tmpfile) = "$tmpdir/list-$$.c";
-  my ($auxfile) = "$tmpdir/list-$$.c.aux";
-  my ($ret);
-  my (%res) = ();
-  open (TMPFILE, ">$tmpfile") || die ("open $tmpfile: $!\n");
-  print TMPFILE "#include <$header>\n";
-  close (TMPFILE) || die ("close $tmpfile: $!\n");
-  $ret = system "$cc_all -c $tmpfile -o /dev/null -aux-info $auxfile > /dev/null";
-  unlink ($tmpfile) || die ("unlink $tmpfile: $!\n");
-  if ($ret != 0) {
-    return;
-  }
-  open (AUXFILE, "<$auxfile") || die ("open $auxfile: $!\n");
-  while (<AUXFILE>) {
-    s|/\*.*?\*/||g;
-    if (/^\s*$/) {
-      next;
-    }
-    # The word before a '(' that isn't '(*' is the function name
-    # before the argument list (not fully general, but sufficient for
-    # -aux-info output on standard headers).
-    if (/(\w+)\s*\([^*]/) {
-      $res{$1} = 1;
-    } else {
-      die ("couldn't parse -aux-info output: $_\n");
-    }
-  }
-  close (AUXFILE) || die ("close $auxfile: $!\n");
-  unlink ($auxfile) || die ("unlink $auxfile: $!\n");
-  return sort keys %res;
-}
diff --git a/conform/Makefile b/conform/Makefile
index fbc4110688..a428fbf937 100644
--- a/conform/Makefile
+++ b/conform/Makefile
@@ -172,13 +172,13 @@ else
 conformtest-cross = --cross
 endif
 $(conformtest-header-tests): $(objpfx)%/conform.out: \
-     conformtest.pl $(conformtest-headers-data)
+     conformtest.py $(conformtest-headers-data)
  (set -e; std_hdr=$*; std=$${std_hdr%%/*}; hdr=$${std_hdr#*/}; \
  mkdir -p $(@D)/scratch; \
- $(PERL) -I. conformtest.pl --tmpdir=$(@D)/scratch --cc='$(CC)' \
- --flags='$(conformtest-cc-flags)' --standard=$$std \
- --headers=$$hdr $(conformtest-xfail) $(conformtest-cross) \
- > $@); \
+ $(PYTHON) $< --cc='$(CC)' --flags='$(conformtest-cc-flags)' \
+   --standard=$$std --header=$$hdr $(conformtest-xfail) \
+   $(conformtest-cross) \
+   > $@ 2>&1); \
  $(evaluate-test)
 
 $(linknamespace-symlists-tests): $(objpfx)symlist-%: list-header-symbols.py
diff --git a/conform/conformtest.pl b/conform/conformtest.pl
deleted file mode 100644
index a4ef756105..0000000000
--- a/conform/conformtest.pl
+++ /dev/null
@@ -1,898 +0,0 @@
-#!/usr/bin/perl
-
-use GlibcConform;
-use Getopt::Long;
-use POSIX;
-
-$standard = "XOPEN2K8";
-$CC = "gcc";
-$tmpdir = "/tmp";
-$cross = "";
-$xfail_str = "";
-GetOptions ('headers=s' => \@headers, 'standard=s' => \$standard,
-    'flags=s' => \$flags, 'cc=s' => \$CC, 'tmpdir=s' => \$tmpdir,
-    'cross' => \$cross, 'xfail=s' => \$xfail_str);
-@headers = split(/,/,join(',',@headers));
-
-# List of the headers we are testing.
-if (@headers == ()) {
-  @headers = ("wordexp.h", "wctype.h", "wchar.h", "varargs.h", "utmpx.h",
-      "utime.h", "unistd.h", "ulimit.h", "ucontext.h", "uchar.h",
-      "time.h", "tgmath.h", "termios.h", "tar.h", "sys/wait.h",
-      "sys/utsname.h", "sys/un.h", "sys/uio.h", "sys/types.h",
-      "sys/times.h", "sys/timeb.h", "sys/time.h", "sys/statvfs.h",
-      "sys/stat.h", "sys/socket.h", "sys/shm.h", "sys/sem.h",
-      "sys/select.h", "sys/resource.h", "sys/msg.h", "sys/mman.h",
-      "sys/ipc.h", "syslog.h", "stropts.h", "strings.h", "string.h",
-      "stdnoreturn.h", "stdlib.h", "stdio.h", "stdint.h", "stddef.h",
-      "stdbool.h", "stdarg.h", "stdalign.h", "spawn.h", "signal.h",
-      "setjmp.h", "semaphore.h", "search.h", "sched.h", "regex.h",
-      "pwd.h", "pthread.h", "poll.h", "nl_types.h", "netinet/tcp.h",
-      "netinet/in.h", "net/if.h", "netdb.h", "ndbm.h", "mqueue.h",
-      "monetary.h", "math.h", "locale.h", "libgen.h", "limits.h",
-      "langinfo.h", "iso646.h", "inttypes.h", "iconv.h", "grp.h",
-      "glob.h", "ftw.h", "fnmatch.h", "fmtmsg.h", "float.h", "fenv.h",
-      "fcntl.h", "errno.h", "dlfcn.h", "dirent.h", "ctype.h", "cpio.h",
-      "complex.h", "assert.h", "arpa/inet.h", "aio.h");
-}
-
-$CFLAGS_namespace = "$flags -fno-builtin $CFLAGS{$standard} -D_ISOMAC";
-$CFLAGS = "$CFLAGS_namespace '-D__attribute__(x)='";
-
-# Check standard name for validity.
-die "unknown standard \"$standard\"" if ($CFLAGS{$standard} eq "");
-
-# if ($standard ne "XOPEN2K8" && $standard ne "POSIX2008") {
-#   # Some headers need a bit more attention.  At least with XPG7
-#   # all headers should be self-contained.
-#   $mustprepend{'inttypes.h'} = "#include <stddef.h>\n";
-#   $mustprepend{'glob.h'} = "#include <sys/types.h>\n";
-#   $mustprepend{'grp.h'} = "#include <sys/types.h>\n";
-#   $mustprepend{'regex.h'} = "#include <sys/types.h>\n";
-#   $mustprepend{'pwd.h'} = "#include <sys/types.h>\n";
-#   $mustprepend{'sched.h'} = "#include <sys/types.h>\n";
-#   $mustprepend{'signal.h'} = "#include <pthread.h>\n#include <sys/types.h>\n";
-#   $mustprepend{'stdio.h'} = "#include <sys/types.h>\n";
-#   $mustprepend{'sys/stat.h'} = "#include <sys/types.h>\n";
-#   $mustprepend{'wchar.h'} = "#include <stdarg.h>\n";
-#   $mustprepend{'wordexp.h'} = "#include <stddef.h>\n";
-# }
-
-# These are the ISO C90 keywords.
-@keywords = ('auto', 'break', 'case', 'char', 'const', 'continue', 'default',
-     'do', 'double', 'else', 'enum', 'extern', 'float', 'for', 'goto',
-     'if', 'int', 'long', 'register', 'return',
-     'short', 'signed', 'sizeof', 'static', 'struct', 'switch',
-     'typedef', 'union', 'unsigned', 'void', 'volatile', 'while');
-if ($CFLAGS{$standard} =~ /-std=(c99|c1x)/) {
-  push (@keywords, 'inline', 'restrict');
-}
-
-# Make a hash table from this information.
-while ($#keywords >= 0) {
-  $iskeyword{pop (@keywords)} = 1;
-}
-
-$verbose = 1;
-
-$total = 0;
-$skipped = 0;
-$errors = 0;
-$xerrors = 0;
-
-sub note_error {
-  my($xfail) = @_;
-  if ($xfail) {
-    $xerrors++;
-    printf ("Ignoring this failure.\n");
-  } else {
-    $errors++;
-  }
-}
-
-
-sub poorfnmatch {
-  my($pattern, $string) = @_;
-  my($strlen) = length ($string);
-  my($res);
-
-  if (substr ($pattern, 0, 1) eq '*') {
-    my($patlen) = length ($pattern) - 1;
-    $res = ($strlen >= $patlen
-    && substr ($pattern, -$patlen, $patlen) eq substr ($string, -$patlen, $patlen));
-  } elsif (substr ($pattern, -1, 1) eq '*') {
-    if (substr ($pattern, -2, 1) eq ']') {
-      my($patlen) = index ($pattern, '[');
-      my($range) = substr ($pattern, $patlen + 1, -2);
-      $res = ($strlen > $patlen
-      && substr ($pattern, 0, $patlen) eq substr ($string, 0, $patlen)
-      && index ($range, substr ($string, $patlen, 1)) != -1);
-    } else {
-      my($patlen) = length ($pattern) - 1;
-      $res = ($strlen >= $patlen
-      && substr ($pattern, 0, $patlen) eq substr ($string, 0, $patlen));
-    }
-  } else {
-    $res = $pattern eq $string;
-  }
-  return $res;
-}
-
-
-sub compiletest
-{
-  my($fnamebase, $msg, $errmsg, $skip, $optional, $xfail) = @_;
-  my($result) = $skip;
-  my($printlog) = 0;
-
-  ++$total;
-  printf ("  $msg...");
-
-  if ($skip != 0) {
-    ++$skipped;
-    printf (" SKIP\n");
-  } else {
-    $ret = system "$CC $CFLAGS -c $fnamebase.c -o $fnamebase.o > $fnamebase.out 2>&1";
-    if ($ret != 0) {
-      if ($optional != 0) {
- printf (" $errmsg\n");
- $result = 1;
-      } else {
- printf (" FAIL\n");
- if ($verbose != 0) {
-  printf ("    $errmsg  Compiler message:\n");
-  $printlog = 1;
- }
- note_error($xfail);
- $result = 1;
-      }
-    } else {
-      printf (" OK\n");
-      if ($verbose > 1 && -s "$fnamebase.out") {
- # We print all warnings issued.
- $printlog = 1;
-      }
-    }
-    if ($printlog != 0) {
-      printf ("    " . "-" x 71 . "\n");
-      open (MESSAGE, "< $fnamebase.out");
-      while (<MESSAGE>) {
- printf ("    %s", $_);
-      }
-      close (MESSAGE);
-      printf ("    " . "-" x 71 . "\n");
-    }
-  }
-  unlink "$fnamebase.c";
-  unlink "$fnamebase.o";
-  unlink "$fnamebase.out";
-
-  $result;
-}
-
-
-sub runtest
-{
-  my($fnamebase, $msg, $errmsg, $skip, $xfail) = @_;
-  my($result) = $skip;
-  my($printlog) = 0;
-
-  ++$total;
-  printf ("  $msg...");
-
-  if ($skip != 0) {
-    ++$skipped;
-    printf (" SKIP\n");
-  } else {
-    $ret = system "$CC $CFLAGS -o $fnamebase $fnamebase.c > $fnamebase.out 2>&1";
-    if ($ret != 0) {
-      printf (" FAIL\n");
-      if ($verbose != 0) {
- printf ("    $errmsg  Compiler message:\n");
- $printlog = 1;
-      }
-      note_error($xfail);
-      $result = 1;
-    } elsif ($cross) {
-      printf (" SKIP\n");
-    } else {
-      # Now run the program.  If the exit code is not zero something is wrong.
-      $result = system "$fnamebase > $fnamebase.out2 2>&1";
-      if ($result == 0) {
- printf (" OK\n");
- if ($verbose > 1 && -s "$fnamebase.out") {
-  # We print all warnings issued.
-  $printlog = 1;
-  system "cat $fnamebase.out2 >> $fnamebase.out";
- }
-      } else {
- printf (" FAIL\n");
- note_error($xfail);
- $printlog = 1;
- unlink "$fnamebase.out";
- rename "$fnamebase.out2", "$fnamebase.out";
-      }
-    }
-    if ($printlog != 0) {
-      printf ("    " . "-" x 71 . "\n");
-      open (MESSAGE, "< $fnamebase.out");
-      while (<MESSAGE>) {
- printf ("    %s", $_);
-      }
-      close (MESSAGE);
-      printf ("    " . "-" x 71 . "\n");
-    }
-  }
-  unlink "$fnamebase";
-  unlink "$fnamebase.c";
-  unlink "$fnamebase.o";
-  unlink "$fnamebase.out";
-  unlink "$fnamebase.out2";
-
-  $result;
-}
-
-
-sub newtoken {
-  my($token, @allow) = @_;
-  my($idx);
-
-  return if ($token =~ /^[0-9_]/ || $iskeyword{$token});
-
-  for ($idx = 0; $idx <= $#allow; ++$idx) {
-    return if (poorfnmatch ($allow[$idx], $token));
-  }
-
-  $errors{$token} = 1;
-}
-
-
-sub removetoken {
-  my($token) = @_;
-  my($idx);
-
-  return if ($token =~ /^[0-9_]/ || $iskeyword{$token});
-
-  if (exists $errors{$token}) {
-    undef $errors{$token};
-  }
-}
-
-
-sub checknamespace {
-  my($h, $fnamebase, @allow) = @_;
-
-  ++$total;
-
-  # Generate a program to get the contents of this header.
-  open (TESTFILE, ">$fnamebase.c");
-  print TESTFILE "#include <$h>\n";
-  close (TESTFILE);
-
-  undef %errors;
-  open (CONTENT, "$CC $CFLAGS_namespace -E $fnamebase.c -P -Wp,-dN | sed -e '/^# [1-9]/d' -e '/^[[:space:]]*\$/d' |");
-  loop: while (<CONTENT>) {
-    chop;
-    if (/^#define (.*)/) {
-      newtoken ($1, @allow);
-    } elsif (/^#undef (.*)/) {
-      removetoken ($1);
-    } else {
-      # We have to tokenize the line.
-      my($str) = $_;
-
-      $str =~ s/"[^"]*"//g;
-      foreach $token (split(/[^a-zA-Z0-9_]/, $str)) {
- if ($token ne "") {
-  newtoken ($token, @allow);
- }
-      }
-    }
-  }
-  close (CONTENT);
-  unlink "$fnamebase.c";
-  $realerror = 0;
-  if ($#errors != 0) {
-    # Sort the output list so it's easier to compare results with diff.
-    foreach $f (sort keys(%errors)) {
-      if ($errors{$f} == 1) {
- if ($realerror == 0) {
-  printf ("FAIL\n    " . "-" x 72 . "\n");
-  $realerror = 1;
-  ++$errors;
- }
- printf ("    Namespace violation: \"%s\"\n", $f);
-      }
-    }
-    printf ("    " . "-" x 72 . "\n") if ($realerror != 0);
-  }
-
-  if ($realerror == 0) {
-    printf ("OK\n");
-  }
-}
-
-
-while ($#headers >= 0) {
-  my($h) = pop (@headers);
-  my($hf) = $h;
-  $hf =~ s|/|-|;
-  my($fnamebase) = "$tmpdir/$hf-test";
-  my($missing) = 1;
-  my(@allow) = ();
-  my(@allowheader) = ();
-  my(%seenheader) = ();
-  my($prepend) = $mustprepend{$h};
-  my($test_exist) = 1;
-
-  printf ("Testing <$h>\n");
-  printf ("----------" . "-" x length ($h) . "\n");
-
-  open (CONTROL, "$CC -E -D$standard -std=c99 -x c data/$h-data |");
-  control: while (<CONTROL>) {
-    chop;
-    next control if (/^#/);
-    next control if (/^[ ]*$/);
-
-    if ($test_exist) {
-      $test_exist = 0;
-      # Generate a program to test for the availability of this header.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      close (TESTFILE);
-
-      $missing = compiletest ($fnamebase, "Checking whether <$h> is available",
-      "Header <$h> not available", 0, 0, 0);
-      printf ("\n");
-      last control if ($missing);
-    }
-
-    my($xfail) = 0;
-    if (/^xfail-/) {
-      s/^xfail-//;
-      $xfail = 1;
-    } elsif (/^xfail\[([^\]]*)\]-/) {
-      my($xfail_cond) = $1;
-      s/^xfail\[([^\]]*)\]-//;
-      # "xfail[cond]-" or "xfail[cond1|cond2|...]-" means a failure of
-      # the test is allowed if any of the listed conditions are in the
-      # --xfail command-line option argument.
-      if ($xfail_str =~ /\b($xfail_cond)\b/) {
- $xfail = 1;
-      }
-    }
-    my($optional) = 0;
-    if (/^optional-/) {
-      s/^optional-//;
-      $optional = 1;
-    }
-    if (/^element *(\{([^}]*)\}|([^{ ]*)) *(\{([^}]*)\}|([^{ ]*)) *([A-Za-z0-9_]*) *(.*)/) {
-      my($struct) = "$2$3";
-      my($type) = "$5$6";
-      my($member) = "$7";
-      my($rest) = "$8";
-      my($res) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $member;
-
-      # Generate a program to test for the availability of this member.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      print TESTFILE "$struct a;\n";
-      print TESTFILE "$struct b;\n";
-      print TESTFILE "extern void xyzzy (__typeof__ (&b.$member), __typeof__ (&a.$member), unsigned);\n";
-      print TESTFILE "void foobarbaz (void) {\n";
-      print TESTFILE "  xyzzy (&a.$member, &b.$member, sizeof (a.$member));\n";
-      print TESTFILE "}\n";
-      close (TESTFILE);
-
-      $res = compiletest ($fnamebase, "Testing for member $member",
-  ($optional
-   ? "NOT AVAILABLE."
-   : "Member \"$member\" not available."), $res,
-  $optional, $xfail);
-
-      if ($res == 0 || $missing != 0 || !$optional) {
- # Test the types of the members.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- print TESTFILE "$struct a;\n";
- print TESTFILE "extern $type b$rest;\n";
- print TESTFILE "extern __typeof__ (a.$member) b;\n";
- close (TESTFILE);
-
- compiletest ($fnamebase, "Testing for type of member $member",
-     "Member \"$member\" does not have the correct type.",
-     $res, 0, $xfail);
-      }
-    } elsif (/^(macro|constant|macro-constant|macro-int-constant) +([a-zA-Z0-9_]*) *(?:{([^}]*)} *)?(?:([>=<!]+) ([A-Za-z0-9_\\'-]*))?/) {
-      my($symbol_type) = $1;
-      my($symbol) = $2;
-      my($type) = $3;
-      my($op) = $4;
-      my($value) = $5;
-      my($res) = $missing;
-      my($mres) = $missing;
-      my($cres) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $symbol;
-
-      if ($symbol_type =~ /macro/) {
- # Generate a program to test for availability of this macro.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- print TESTFILE "#ifndef $symbol\n";
- print TESTFILE "# error \"Macro $symbol not defined\"\n";
- print TESTFILE "#endif\n";
- close (TESTFILE);
-
- $mres = compiletest ($fnamebase, "Test availability of macro $symbol",
-     ($optional
-      ? "NOT PRESENT"
-      : "Macro \"$symbol\" is not available."), $res,
-     $optional, $xfail);
-      }
-
-      if ($symbol_type =~ /constant/) {
- # Generate a program to test for the availability of this constant.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- print TESTFILE "__typeof__ ($symbol) a = $symbol;\n";
- close (TESTFILE);
-
- $cres = compiletest ($fnamebase, "Testing for constant $symbol",
-     ($optional
-      ? "NOT PRESENT"
-      : "Constant \"$symbol\" not available."), $res,
-     $optional, $xfail);
-      }
-
-      $res = $res || $mres || $cres;
-
-      if ($symbol_type eq "macro-int-constant" && ($res == 0 || !$optional)) {
- # Test that the symbol is usable in #if.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- print TESTFILE "#if $symbol < 0\n";
- print TESTFILE "# define conformtest_negative 1\n";
- my($s) = "0";
- for (my $i = 0; $i < 63; $i++) {
-  print TESTFILE "# if $symbol & (1LL << $i)\n";
-  print TESTFILE "#  define conformtest_bit_$i 0LL\n";
-  print TESTFILE "# else\n";
-  print TESTFILE "#  define conformtest_bit_$i (1LL << $i)\n";
-  print TESTFILE "# endif\n";
-  $s .= "|conformtest_bit_$i";
- }
- print TESTFILE "# define conformtest_value ~($s)\n";
- print TESTFILE "#else\n";
- print TESTFILE "# define conformtest_negative 0\n";
- $s = "0";
- for (my $i = 0; $i < 64; $i++) {
-  print TESTFILE "# if $symbol & (1ULL << $i)\n";
-  print TESTFILE "#  define conformtest_bit_$i (1ULL << $i)\n";
-  print TESTFILE "# else\n";
-  print TESTFILE "#  define conformtest_bit_$i 0ULL\n";
-  print TESTFILE "# endif\n";
-  $s .= "|conformtest_bit_$i";
- }
- print TESTFILE "# define conformtest_value ($s)\n";
- print TESTFILE "#endif\n";
- print TESTFILE "_Static_assert ((($symbol < 0) == conformtest_negative) && ($symbol == conformtest_value), \"value match inside and outside #if\");\n";
- close (TESTFILE);
-
- compiletest ($fnamebase, "Testing for #if usability of symbol $symbol",
-     "Symbol \"$symbol\" not usable in #if.", $res, 0, $xfail);
-      }
-
-      if (defined ($type) && ($res == 0 || !$optional)) {
- # Test the type of the symbol.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- if ($type =~ /^promoted:/) {
-  $type =~ s/^promoted://;
-  print TESTFILE "__typeof__ (($type) 0 + ($type) 0) a;\n";
- } else {
-  print TESTFILE "__typeof__ (($type) 0) a;\n";
- }
- print TESTFILE "extern __typeof__ ($symbol) a;\n";
- close (TESTFILE);
-
- compiletest ($fnamebase, "Testing for type of symbol $symbol",
-     "Symbol \"$symbol\" does not have the correct type.",
-     $res, 0, $xfail);
-      }
-
-      if (defined ($op) && ($res == 0 || !$optional)) {
- # Generate a program to test for the value of this symbol.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- print TESTFILE "_Static_assert ($symbol $op $value, \"value constraint\");\n";
- close (TESTFILE);
-
- $res = compiletest ($fnamebase, "Testing for value of symbol $symbol",
-    "Symbol \"$symbol\" has not the right value.",
-    $res, 0, $xfail);
-      }
-    } elsif (/^symbol *([a-zA-Z0-9_]*) *([A-Za-z0-9_-]*)?/) {
-      my($symbol) = $1;
-      my($value) = $2;
-      my($res) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $symbol;
-
-      # Generate a program to test for the availability of this constant.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      print TESTFILE "void foobarbaz (void) {\n";
-      print TESTFILE "__typeof__ ($symbol) a = $symbol;\n";
-      print TESTFILE "}\n";
-      close (TESTFILE);
-
-      $res = compiletest ($fnamebase, "Testing for symbol $symbol",
-  "Symbol \"$symbol\" not available.", $res, 0, $xfail);
-
-      if ($value ne "") {
- # Generate a program to test for the value of this constant.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- print TESTFILE "int main (void) { return $symbol != $value; }\n";
- close (TESTFILE);
-
- $res = runtest ($fnamebase, "Testing for value of symbol $symbol",
- "Symbol \"$symbol\" has not the right value.", $res,
- $xfail);
-      }
-    } elsif (/^type *(\{([^}]*)|([a-zA-Z0-9_]*))/) {
-      my($type) = "$2$3";
-      my($maybe_opaque) = 0;
-
-      # Remember that this name is allowed.
-      if ($type =~ /^struct *(.*)/) {
- push @allow, $1;
-      } elsif ($type =~ /^union *(.*)/) {
- push @allow, $1;
-      } else {
- push @allow, $type;
- $maybe_opaque = 1;
-      }
-
-      # Generate a program to test for the availability of this type.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      if ($maybe_opaque == 1) {
- print TESTFILE "$type *a;\n";
-      } else {
- print TESTFILE "$type a;\n";
-      }
-      close (TESTFILE);
-
-      compiletest ($fnamebase, "Testing for type $type",
-   ($optional
-    ? "NOT AVAILABLE"
-    : "Type \"$type\" not available."), $missing, $optional,
-   $xfail);
-    } elsif (/^tag *(\{([^}]*)|([a-zA-Z0-9_]*))/) {
-      my($type) = "$2$3";
-
-      # Remember that this name is allowed.
-      if ($type =~ /^struct *(.*)/) {
- push @allow, $1;
-      } elsif ($type =~ /^union *(.*)/) {
- push @allow, $1;
-      } else {
- push @allow, $type;
-      }
-
-      # Generate a program to test for the availability of this type.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      print TESTFILE "$type;\n";
-      close (TESTFILE);
-
-      compiletest ($fnamebase, "Testing for type $type",
-   "Type \"$type\" not available.", $missing, 0, $xfail);
-    } elsif (/^function *(\{([^}]*)\}|([a-zA-Z0-9_]*)) [(][*]([a-zA-Z0-9_]*) ([(].*[)])/) {
-      my($rettype) = "$2$3";
-      my($fname) = "$4";
-      my($args) = "$5";
-      my($res) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $fname;
-
-      # Generate a program to test for availability of this function.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      # print TESTFILE "#undef $fname\n";
-      print TESTFILE "$rettype (*(*foobarbaz) $args = $fname;\n";
-      close (TESTFILE);
-
-      $res = compiletest ($fnamebase, "Test availability of function $fname",
-  ($optional
-   ? "NOT AVAILABLE"
-   : "Function \"$fname\" is not available."), $res,
-  $optional, $xfail);
-
-      if ($res == 0 || $missing == 1 || !$optional) {
- # Generate a program to test for the type of this function.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- # print TESTFILE "#undef $fname\n";
- print TESTFILE "extern $rettype (*(*foobarbaz) $args;\n";
- print TESTFILE "extern __typeof__ (&$fname) foobarbaz;\n";
- close (TESTFILE);
-
- compiletest ($fnamebase, "Test for type of function $fname",
-     "Function \"$fname\" has incorrect type.", $res, 0,
-     $xfail);
-      }
-    } elsif (/^function *(\{([^}]*)\}|([a-zA-Z0-9_]*)) ([a-zA-Z0-9_]*) ([(].*[)])/) {
-      my($rettype) = "$2$3";
-      my($fname) = "$4";
-      my($args) = "$5";
-      my($res) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $fname;
-
-      # Generate a program to test for availability of this function.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      # print TESTFILE "#undef $fname\n";
-      print TESTFILE "$rettype (*foobarbaz) $args = $fname;\n";
-      close (TESTFILE);
-
-      $res = compiletest ($fnamebase, "Test availability of function $fname",
-  ($optional
-   ? "NOT AVAILABLE"
-   : "Function \"$fname\" is not available."), $res,
-  $optional, $xfail);
-
-      if ($res == 0 || $missing != 0 || !$optional) {
- # Generate a program to test for the type of this function.
- open (TESTFILE, ">$fnamebase.c");
- print TESTFILE "$prepend";
- print TESTFILE "#include <$h>\n";
- # print TESTFILE "#undef $fname\n";
- print TESTFILE "extern $rettype (*foobarbaz) $args;\n";
- print TESTFILE "extern __typeof__ (&$fname) foobarbaz;\n";
- close (TESTFILE);
-
- compiletest ($fnamebase, "Test for type of function $fname",
-     "Function \"$fname\" has incorrect type.", $res, 0,
-     $xfail);
-      }
-    } elsif (/^variable *(\{([^}]*)\}|([a-zA-Z0-9_]*)) ([a-zA-Z0-9_]*) *(.*)/) {
-      my($type) = "$2$3";
-      my($vname) = "$4";
-      my($rest) = "$5";
-      my($res) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $vname;
-
-      # Generate a program to test for availability of this function.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      # print TESTFILE "#undef $fname\n";
-      print TESTFILE "typedef $type xyzzy$rest;\n";
-      print TESTFILE "$xyzzy *foobarbaz = &$vname;\n";
-      close (TESTFILE);
-
-      $res = compiletest ($fnamebase, "Test availability of variable $vname",
-  "Variable \"$vname\" is not available.", $res, 0,
-  $xfail);
-
-      # Generate a program to test for the type of this function.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      # print TESTFILE "#undef $fname\n";
-      print TESTFILE "extern $type $vname$rest;\n";
-      close (TESTFILE);
-
-      compiletest ($fnamebase, "Test for type of variable $fname",
-   "Variable \"$vname\" has incorrect type.", $res, 0, $xfail);
-    } elsif (/^macro-function *(\{([^}]*)\}|([a-zA-Z0-9_]*)) ([a-zA-Z0-9_]*) ([(].*[)])/) {
-      my($rettype) = "$2$3";
-      my($fname) = "$4";
-      my($args) = "$5";
-      my($res) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $fname;
-
-      # Generate a program to test for availability of this function.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      print TESTFILE "#ifndef $fname\n";
-      print TESTFILE "$rettype (*foobarbaz) $args = $fname;\n";
-      print TESTFILE "#endif\n";
-      close (TESTFILE);
-
-      $res = compiletest ($fnamebase, "Test availability of macro $fname",
-  "Function \"$fname\" is not available.", $res, 0,
-  $xfail);
-
-      # Generate a program to test for the type of this function.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      print TESTFILE "#ifndef $fname\n";
-      print TESTFILE "extern $rettype (*foobarbaz) $args;\n";
-      print TESTFILE "extern __typeof__ (&$fname) foobarbaz;\n";
-      print TESTFILE "#endif\n";
-      close (TESTFILE);
-
-      compiletest ($fnamebase, "Test for type of macro $fname",
-   "Function \"$fname\" has incorrect type.", $res, 0, $xfail);
-    } elsif (/^macro-str *([^ ]*) *(\".*\")/) {
-      # The above regex doesn't handle a \" in a string.
-      my($macro) = "$1";
-      my($string) = "$2";
-      my($res) = $missing;
-
-      # Remember that this name is allowed.
-      push @allow, $macro;
-
-      # Generate a program to test for availability of this macro.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      print TESTFILE "#ifndef $macro\n";
-      print TESTFILE "# error \"Macro $macro not defined\"\n";
-      print TESTFILE "#endif\n";
-      close (TESTFILE);
-
-      compiletest ($fnamebase, "Test availability of macro $macro",
-   "Macro \"$macro\" is not available.", $missing, 0, $xfail);
-
-      # Generate a program to test for the value of this macro.
-      open (TESTFILE, ">$fnamebase.c");
-      print TESTFILE "$prepend";
-      print TESTFILE "#include <$h>\n";
-      # We can't include <string.h> here.
-      print TESTFILE "extern int (strcmp)(const char *, const char *);\n";
-      print TESTFILE "int main (void) { return (strcmp) ($macro, $string) != 0;}\n";
-      close (TESTFILE);
-
-      $res = runtest ($fnamebase, "Testing for value of macro $macro",
-      "Macro \"$macro\" has not the right value.", $res,
-      $xfail);
-    } elsif (/^allow-header *(.*)/) {
-      my($pattern) = $1;
-      if ($seenheader{$pattern} != 1) {
- push @allowheader, $pattern;
- $seenheader{$pattern} = 1;
-      }
-      next control;
-    } elsif (/^allow *(.*)/) {
-      my($pattern) = $1;
-      push @allow, $pattern;
-      next control;
-    } else {
-      # printf ("line is `%s'\n", $_);
-      next control;
-    }
-
-    printf ("\n");
-  }
-  close (CONTROL);
-
-  # Read the data files for the header files which are allowed to be included.
-  while ($#allowheader >= 0) {
-    my($ah) = pop @allowheader;
-
-    open (ALLOW, "$CC -E -D$standard -x c data/$ah-data |");
-    acontrol: while (<ALLOW>) {
-      chop;
-      next acontrol if (/^#/);
-      next acontrol if (/^[ ]*$/);
-
-      s/^xfail(\[([^\]]*)\])?-//;
-      s/^optional-//;
-      if (/^element *(\{([^}]*)\}|([^ ]*)) *(\{([^}]*)\}|([^ ]*)) *([A-Za-z0-9_]*) *(.*)/) {
- push @allow, $7;
-      } elsif (/^(macro|constant|macro-constant|macro-int-constant) +([a-zA-Z0-9_]*) *(?:{([^}]*)} *)?(?:([>=<!]+) ([A-Za-z0-9_-]*))?/) {
- push @allow, $2;
-      } elsif (/^(type|tag) *(\{([^}]*)|([a-zA-Z0-9_]*))/) {
- my($type) = "$3$4";
-
- # Remember that this name is allowed.
- if ($type =~ /^struct *(.*)/) {
-  push @allow, $1;
- } elsif ($type =~ /^union *(.*)/) {
-  push @allow, $1;
- } else {
-  push @allow, $type;
- }
-      } elsif (/^function *(\{([^}]*)\}|([a-zA-Z0-9_]*)) [(][*]([a-zA-Z0-9_]*) ([(].*[)])/) {
- push @allow, $4;
-      } elsif (/^function *(\{([^}]*)\}|([a-zA-Z0-9_]*)) ([a-zA-Z0-9_]*) ([(].*[)])/) {
- push @allow, $4;
-      } elsif (/^variable *(\{([^}]*)\}|([a-zA-Z0-9_]*)) ([a-zA-Z0-9_]*)/) {
- push @allow, $4;
-      } elsif (/^macro-function *(\{([^}]*)\}|([a-zA-Z0-9_]*)) ([a-zA-Z0-9_]*) ([(].*[)])/) {
- push @allow, $4;
-      } elsif (/^symbol *([a-zA-Z0-9_]*) *([A-Za-z0-9_-]*)?/) {
- push @allow, $1;
-      } elsif (/^allow-header *(.*)/) {
- if ($seenheader{$1} != 1) {
-  push @allowheader, $1;
-  $seenheader{$1} = 1;
- }
-      } elsif (/^allow *(.*)/) {
- push @allow, $1;
-      }
-    }
-    close (ALLOW);
-  }
-
-  if ($test_exist) {
-    printf ("  Not defined\n");
-  } else {
-    # Now check the namespace.
-    printf ("  Checking the namespace of \"%s\"... ", $h);
-    if ($missing) {
-      ++$skipped;
-      printf ("SKIP\n");
-    } else {
-      checknamespace ($h, $fnamebase, @allow);
-    }
-  }
-
-  printf ("\n\n");
-}
-
-printf "-" x 76 . "\n";
-printf ("  Total number of tests   : %4d\n", $total);
-
-printf ("  Number of failed tests  : %4d (", $errors);
-$percent = ($errors * 100) / $total;
-if ($errors > 0 && $percent < 1.0) {
-  printf (" <1%%)\n");
-} else {
-  printf ("%3d%%)\n", $percent);
-}
-
-printf ("  Number of xfailed tests : %4d (", $xerrors);
-$percent = ($xerrors * 100) / $total;
-if ($xerrors > 0 && $percent < 1.0) {
-  printf (" <1%%)\n");
-} else {
-  printf ("%3d%%)\n", $percent);
-}
-
-printf ("  Number of skipped tests : %4d (", $skipped);
-$percent = ($skipped * 100) / $total;
-if ($skipped > 0 && $percent < 1.0) {
-  printf (" <1%%)\n");
-} else {
-  printf ("%3d%%)\n", $percent);
-}
-
-exit $errors != 0;
-# Local Variables:
-#  perl-indent-level: 2
-# End:
diff --git a/conform/conformtest.py b/conform/conformtest.py
new file mode 100644
index 0000000000..46fdfde8a6
--- /dev/null
+++ b/conform/conformtest.py
@@ -0,0 +1,664 @@
+#!/usr/bin/python3
+# Check header contents against the given standard.
+# Copyright (C) 2018 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+#
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <http://www.gnu.org/licenses/>.
+
+import argparse
+import fnmatch
+import os.path
+import re
+import subprocess
+import sys
+import tempfile
+
+import glibcconform
+
+
+class ElementTest(object):
+    """Test for an element of a structure or union type."""
+
+    def __init__(self, dummy, type_name, member_type, member_name, *rest):
+        """Initialize an ElementTest object."""
+        self.type_name = type_name
+        self.member_type = member_type
+        self.member_name = member_name
+        self.rest = ' '.join(rest)
+        self.allow_name = self.member_name
+
+    def run(self, header_tests):
+        """Run an ElementTest."""
+        text = ('%s a;\n'
+                '%s b;\n'
+                'extern void xyzzy (__typeof__ (&b.%s), __typeof__ (&a.%s), '
+                'unsigned);\n'
+                'void foobarbaz (void) {\n'
+                'xyzzy (&a.%s, &b.%s, sizeof (a.%s));\n'
+                '}\n'
+                % (self.type_name, self.type_name,
+                   self.member_name, self.member_name,
+                   self.member_name, self.member_name, self.member_name))
+        header_tests.compile_test('Availability of member %s'
+                                  % self.member_name,
+                                  text)
+        text = ('%s a;\n'
+                'extern %s b%s;\n'
+                'extern __typeof__ (a.%s) b;\n'
+                % (self.type_name, self.member_type, self.rest,
+                   self.member_name))
+        header_tests.compile_test('Type of member %s' % self.member_name,
+                                  text)
+
+
+class ConstantTest(object):
+    """Test for a macro or constant."""
+
+    def __init__(self, symbol_type, symbol, extra1=None, extra2=None,
+                 extra3=None):
+        """Initialize a ConstantTest object."""
+        self.symbol_type = symbol_type
+        self.symbol = symbol
+        # A comparison operation may be specified without a type.
+        if extra2 is not None and extra3 is None:
+            self.c_type = None
+            self.op = extra1
+            self.value = extra2
+        else:
+            self.c_type = extra1
+            self.op = extra2
+            self.value = extra3
+        self.allow_name = self.symbol
+
+    def run(self, header_tests):
+        """Run a ConstantTest."""
+        if 'macro' in self.symbol_type:
+            text = ('#ifndef %s\n'
+                    '# error "Macro %s not defined"\n'
+                    '#endif\n'
+                    % (self.symbol, self.symbol))
+            header_tests.compile_test('Availability of macro %s'
+                                      % self.symbol,
+                                      text)
+        if 'constant' in self.symbol_type:
+            text = ('__typeof__ (%s) a = %s;\n'
+                    % (self.symbol, self.symbol))
+            header_tests.compile_test('Availability of constant %s'
+                                      % self.symbol,
+                                      text)
+        if self.symbol_type == 'macro-int-constant':
+            sym_bits_def_neg = ''.join(
+                '# if %s & (1LL << %d)\n'
+                '#  define conformtest_bit_%d 0LL\n'
+                '# else\n'
+                '#  define conformtest_bit_%d (1LL << %d)\n'
+                '# endif\n'
+                % (self.symbol, i, i, i, i) for i in range(63))
+            sym_bits_or_neg = '|'.join('conformtest_bit_%d' % i
+                                       for i in range(63))
+            sym_bits_def_pos = ''.join(
+                '# if %s & (1ULL << %d)\n'
+                '#  define conformtest_bit_%d (1ULL << %d)\n'
+                '# else\n'
+                '#  define conformtest_bit_%d 0ULL\n'
+                '# endif\n'
+                % (self.symbol, i, i, i, i) for i in range(64))
+            sym_bits_or_pos = '|'.join('conformtest_bit_%d' % i
+                                       for i in range(64))
+            text = ('#if %s < 0\n'
+                    '# define conformtest_negative 1\n'
+                    '%s'
+                    '# define conformtest_value ~(%s)\n'
+                    '#else\n'
+                    '# define conformtest_negative 0\n'
+                    '%s'
+                    '# define conformtest_value (%s)\n'
+                    '#endif\n'
+                    '_Static_assert (((%s < 0) == conformtest_negative) '
+                    '&& (%s == conformtest_value), '
+                    '"value match inside and outside #if");\n'
+                    % (self.symbol, sym_bits_def_neg, sym_bits_or_neg,
+                       sym_bits_def_pos, sym_bits_or_pos,
+                       self.symbol, self.symbol))
+            header_tests.compile_test('#if usability of symbol %s'
+                                      % self.symbol,
+                                      text)
+        if self.c_type is not None:
+            if self.c_type.startswith('promoted:'):
+                c_type = self.c_type[len('promoted:'):]
+                text = ('__typeof__ ((%s) 0 + (%s) 0) a;\n'
+                        % (c_type, c_type))
+            else:
+                text = '__typeof__ ((%s) 0) a;\n' % self.c_type
+            text += 'extern __typeof__ (%s) a;\n' % self.symbol
+            header_tests.compile_test('Type of symbol %s' % self.symbol,
+                                      text)
+        if self.op is not None:
+            text = ('_Static_assert (%s %s %s, "value constraint");\n'
+                    % (self.symbol, self.op, self.value))
+            header_tests.compile_test('Value of symbol %s' % self.symbol,
+                                      text)
+
+
+class SymbolTest(object):
+    """Test for a symbol (not a compile-time constant)."""
+
+    def __init__(self, dummy, symbol, value=None):
+        """Initialize a SymbolTest object."""
+        self.symbol = symbol
+        self.value = value
+        self.allow_name = self.symbol
+
+    def run(self, header_tests):
+        """Run a SymbolTest."""
+        text = ('void foobarbaz (void) {\n'
+                '__typeof__ (%s) a = %s;\n'
+                '}\n'
+                % (self.symbol, self.symbol))
+        header_tests.compile_test('Availability of symbol %s'
+                                  % self.symbol,
+                                  text)
+        if self.value is not None:
+            text = ('int main (void) { return %s != %s; }\n'
+                    % (self.symbol, self.value))
+            header_tests.execute_test('Value of symbol %s' % self.symbol,
+                                      text)
+
+
+class TypeTest(object):
+    """Test for a type name."""
+
+    def __init__(self, dummy, type_name):
+        """Initialize a TypeTest object."""
+        self.type_name = type_name
+        if type_name.startswith('struct '):
+            self.allow_name = type_name[len('struct '):]
+            self.maybe_opaque = False
+        elif type_name.startswith('union '):
+            self.allow_name = type_name[len('union '):]
+            self.maybe_opaque = False
+        else:
+            self.allow_name = type_name
+            self.maybe_opaque = True
+
+    def run(self, header_tests):
+        """Run a TypeTest."""
+        text = ('%s %sa;\n'
+                % (self.type_name, '*' if self.maybe_opaque else ''))
+        header_tests.compile_test('Availability of type %s' % self.type_name,
+                                  text)
+
+
+class TagTest(object):
+    """Test for a tag name."""
+
+    def __init__(self, dummy, type_name):
+        """Initialize a TagTest object."""
+        self.type_name = type_name
+        if type_name.startswith('struct '):
+            self.allow_name = type_name[len('struct '):]
+        elif type_name.startswith('union '):
+            self.allow_name = type_name[len('union '):]
+        else:
+            raise ValueError('unexpected kind of tag: %s' % type_name)
+
+    def run(self, header_tests):
+        """Run a TagTest."""
+        # If the tag is not declared, these function prototypes have
+        # incompatible types.
+        text = ('void foo (%s *);\n'
+                'void foo (%s *);\n'
+                % (self.type_name, self.type_name))
+        header_tests.compile_test('Availability of tag %s' % self.type_name,
+                                  text)
+
+
+class FunctionTest(object):
+    """Test for a function."""
+
+    def __init__(self, dummy, return_type, function_name, *args):
+        """Initialize a FunctionTest object."""
+        self.function_name_full = function_name
+        self.args = ' '.join(args)
+        if function_name.startswith('(*'):
+            # Function returning a pointer to function.
+            self.return_type = '%s (*' % return_type
+            self.function_name = function_name[len('(*'):]
+        else:
+            self.return_type = return_type
+            self.function_name = function_name
+        self.allow_name = self.function_name
+
+    def run(self, header_tests):
+        """Run a FunctionTest."""
+        text = ('%s (*foobarbaz) %s = %s;\n'
+                % (self.return_type, self.args, self.function_name))
+        header_tests.compile_test('Availability of function %s'
+                                  % self.function_name,
+                                  text)
+        text = ('extern %s (*foobarbaz) %s;\n'
+                'extern __typeof__ (&%s) foobarbaz;\n'
+                % (self.return_type, self.args, self.function_name))
+        header_tests.compile_test('Type of function %s' % self.function_name,
+                                  text)
+
+
+class VariableTest(object):
+    """Test for a variable."""
+
+    def __init__(self, dummy, var_type, var_name, *rest):
+        """Initialize a VariableTest object."""
+        self.var_type = var_type
+        self.var_name = var_name
+        self.rest = ' '.join(rest)
+        self.allow_name = var_name
+
+    def run(self, header_tests):
+        """Run a VariableTest."""
+        text = ('typedef %s xyzzy%s;\n'
+                'xyzzy *foobarbaz = &%s;\n'
+                % (self.var_type, self.rest, self.var_name))
+        header_tests.compile_test('Availability of variable %s'
+                                  % self.var_name,
+                                  text)
+        text = ('extern %s %s%s;\n'
+                % (self.var_type, self.var_name, self.rest))
+        header_tests.compile_test('Type of variable %s' % self.var_name,
+                                  text)
+
+
+class MacroFunctionTest(object):
+    """Test for a possibly macro-only function."""
+
+    def __init__(self, dummy, return_type, function_name, *args):
+        """Initialize a MacroFunctionTest object."""
+        self.return_type = return_type
+        self.function_name = function_name
+        self.args = ' '.join(args)
+        self.allow_name = function_name
+
+    def run(self, header_tests):
+        """Run a MacroFunctionTest."""
+        text = ('#ifndef %s\n'
+                '%s (*foobarbaz) %s = %s;\n'
+                '#endif\n'
+                % (self.function_name, self.return_type, self.args,
+                   self.function_name))
+        header_tests.compile_test('Availability of macro %s'
+                                  % self.function_name,
+                                  text)
+        text = ('#ifndef %s\n'
+                'extern %s (*foobarbaz) %s;\n'
+                'extern __typeof__ (&%s) foobarbaz;\n'
+                '#endif\n'
+                % (self.function_name, self.return_type, self.args,
+                   self.function_name))
+        header_tests.compile_test('Type of macro %s' % self.function_name,
+                                  text)
+
+
+class MacroStrTest(object):
+    """Test for a string-valued macro."""
+
+    def __init__(self, dummy, macro_name, value):
+        """Initialize a MacroStrTest object."""
+        self.macro_name = macro_name
+        self.value = value
+        self.allow_name = macro_name
+
+    def run(self, header_tests):
+        """Run a MacroStrTest."""
+        text = ('#ifndef %s\n'
+                '# error "Macro %s not defined"\n'
+                '#endif\n'
+                % (self.macro_name, self.macro_name))
+        header_tests.compile_test('Availability of macro %s' % self.macro_name,
+                                  text)
+        # We can't include <string.h> here.
+        text = ('extern int (strcmp)(const char *, const char *);\n'
+                'int main (void) { return (strcmp) (%s, %s) != 0; }\n'
+                % (self.macro_name, self.value))
+        header_tests.execute_test('Value of macro %s' % self.macro_name,
+                                  text)
+
+
+class HeaderTests(object):
+    """The set of tests run for a header."""
+
+    def __init__(self, header, standard, cc, flags, cross, xfail):
+        """Initialize a HeaderTests object."""
+        self.header = header
+        self.standard = standard
+        self.cc = cc
+        self.flags = flags
+        self.cross = cross
+        self.xfail_str = xfail
+        self.cflags_namespace = ('%s -fno-builtin %s -D_ISOMAC'
+                                 % (flags, glibcconform.CFLAGS[standard]))
+        # When compiling the conformance test programs, use of
+        # __attribute__ in headers is disabled because of attributes
+        # that affect the types of functions as seen by typeof.
+        self.cflags = "%s '-D__attribute__(x)='" % self.cflags_namespace
+        self.tests = []
+        self.allow = set()
+        self.allow_fnmatch = set()
+        self.headers_handled = set()
+        self.total = 0
+        self.skipped = 0
+        self.errors = 0
+        self.xerrors = 0
+
+    def add_allow(self, name, pattern_ok):
+        """Add an identifier as an allowed token for this header.
+
+        If pattern_ok, fnmatch patterns are OK as well as
+        identifiers.
+
+        """
+        if re.fullmatch(r'[A-Za-z_][A-Za-z0-9_]*', name):
+            self.allow.add(name)
+        elif pattern_ok:
+            self.allow_fnmatch.add(name)
+        else:
+            raise ValueError('bad identifier: %s' % name)
+
+    def check_token(self, bad_tokens, token):
+        """Check whether an identifier token is allowed, and record it in
+        bad_tokens if not.
+
+        """
+        if token.startswith('_'):
+            return
+        if token in glibcconform.KEYWORDS[self.standard]:
+            return
+        if token in self.allow:
+            return
+        for pattern in self.allow_fnmatch:
+            if fnmatch.fnmatch(token, pattern):
+                return
+        bad_tokens.add(token)
+
+    def handle_test_line(self, line, allow):
+        """Handle a single line in the test data.
+
+        If allow is true, the header is one specified in allow-header
+        and so tests are marked as allowed for namespace purposes but
+        otherwise ignored.
+
+        """
+        orig_line = line
+        xfail = False
+        if line.startswith('xfail-'):
+            xfail = True
+            line = line[len('xfail-'):]
+        else:
+            match = re.match(r'xfail\[(.*?)\]-(.*)', line)
+            if match:
+                xfail_cond = match.group(1)
+                line = match.group(2)
+                # "xfail[cond]-" or "xfail[cond1|cond2|...]-" means a
+                # failure of the test is allowed if any of the listed
+                # conditions are in the --xfail command-line option
+                # argument.
+                if self.xfail_str and re.search(r'\b(%s)\b' % xfail_cond,
+                                                self.xfail_str):
+                    xfail = True
+        optional = False
+        if line.startswith('optional-'):
+            optional = True
+            line = line[len('optional-'):]
+        # Tokens in test data are space-separated, except for {...}
+        # tokens that may contain spaces.
+        tokens = []
+        while line:
+            match = re.match(r'\{(.*?)\}(.*)', line)
+            if match:
+                tokens.append(match.group(1))
+                line = match.group(2)
+            else:
+                match = re.match(r'([^ ]*)(.*)', line)
+                tokens.append(match.group(1))
+                line = match.group(2)
+            line = line.strip()
+        if tokens[0] == 'allow-header':
+            if len(tokens) != 2 or xfail or optional:
+                raise ValueError('bad allow-header line: %s' % orig_line)
+            if tokens[1] not in self.headers_handled:
+                self.load_tests(tokens[1], True)
+            return
+        if tokens[0] == 'allow':
+            if len(tokens) != 2 or xfail or optional:
+                raise ValueError('bad allow line: %s' % orig_line)
+            self.add_allow(tokens[1], True)
+            return
+        test_classes = {'element': ElementTest,
+                        'macro': ConstantTest,
+                        'constant': ConstantTest,
+                        'macro-constant': ConstantTest,
+                        'macro-int-constant': ConstantTest,
+                        'symbol': SymbolTest,
+                        'type': TypeTest,
+                        'tag': TagTest,
+                        'function': FunctionTest,
+                        'variable': VariableTest,
+                        'macro-function': MacroFunctionTest,
+                        'macro-str': MacroStrTest}
+        test = test_classes[tokens[0]](*tokens)
+        test.xfail = xfail
+        test.optional = optional
+        self.add_allow(test.allow_name, False)
+        if not allow:
+            self.tests.append(test)
+
+    def load_tests(self, header, allow):
+        """Load tests of a header.
+
+        If allow is true, the header is one specified in allow-header
+        and so tests are marked as allowed for namespace purposes but
+        otherwise ignored.
+
+        """
+        self.headers_handled.add(header)
+        header_s = header.replace('/', '_')
+        temp_file = os.path.join(self.temp_dir, 'header-data-%s' % header_s)
+        cmd = ('%s -E -D%s -std=c99 -x c data/%s-data > %s'
+               % (self.cc, self.standard, header, temp_file))
+        subprocess.check_call(cmd, shell=True)
+        with open(temp_file, 'r') as tests:
+            for line in tests:
+                line = line.strip()
+                if line == '' or line.startswith('#'):
+                    continue
+                self.handle_test_line(line, allow)
+
+    def note_error(self, name, xfail):
+        """Note a failing test."""
+        if xfail:
+            print('XFAIL: %s' % name)
+            self.xerrors += 1
+        else:
+            print('FAIL: %s' % name)
+            self.errors += 1
+        sys.stdout.flush()
+
+    def note_skip(self, name):
+        """Note a skipped test."""
+        print('SKIP: %s' % name)
+        self.skipped += 1
+        sys.stdout.flush()
+
+    def compile_test(self, name, text):
+        """Run a compilation test; return True if it passes."""
+        self.total += 1
+        if self.group_ignore:
+            return False
+        optional = self.group_optional
+        self.group_optional = False
+        if self.group_skip:
+            self.note_skip(name)
+            return False
+        c_file = os.path.join(self.temp_dir, 'test.c')
+        o_file = os.path.join(self.temp_dir, 'test.o')
+        with open(c_file, 'w') as c_file_out:
+            c_file_out.write('#include <%s>\n%s' % (self.header, text))
+        cmd = ('%s %s -c %s -o %s' % (self.cc, self.cflags, c_file, o_file))
+        try:
+            subprocess.check_call(cmd, shell=True)
+        except subprocess.CalledProcessError:
+            if optional:
+                print('MISSING: %s' % name)
+                sys.stdout.flush()
+                self.group_ignore = True
+            else:
+                self.note_error(name, self.group_xfail)
+                self.group_skip = True
+            return False
+        print('PASS: %s' % name)
+        sys.stdout.flush()
+        return True
+
+    def execute_test(self, name, text):
+        """Run an execution test."""
+        self.total += 1
+        if self.group_ignore:
+            return False
+        if self.group_skip:
+            self.note_skip(name)
+            return
+        c_file = os.path.join(self.temp_dir, 'test.c')
+        exe_file = os.path.join(self.temp_dir, 'test')
+        with open(c_file, 'w') as c_file_out:
+            c_file_out.write('#include <%s>\n%s' % (self.header, text))
+        cmd = ('%s %s %s -o %s' % (self.cc, self.cflags, c_file, exe_file))
+        try:
+            subprocess.check_call(cmd, shell=True)
+        except subprocess.CalledProcessError:
+            self.note_error(name, self.group_xfail)
+            return
+        if self.cross:
+            self.note_skip(name)
+            return
+        try:
+            subprocess.check_call(exe_file, shell=True)
+        except subprocess.CalledProcessError:
+            self.note_error(name, self.group_xfail)
+            return
+        print('PASS: %s' % name)
+        sys.stdout.flush()
+
+    def check_namespace(self, name):
+        """Check the namespace of a header."""
+        c_file = os.path.join(self.temp_dir, 'namespace.c')
+        out_file = os.path.join(self.temp_dir, 'namespace-out')
+        with open(c_file, 'w') as c_file_out:
+            c_file_out.write('#include <%s>\n' % self.header)
+        cmd = ('%s %s -E %s -P -Wp,-dN > %s'
+               % (self.cc, self.cflags_namespace, c_file, out_file))
+        subprocess.check_call(cmd, shell=True)
+        bad_tokens = set()
+        with open(out_file, 'r') as content:
+            for line in content:
+                line = line.strip()
+                if not line:
+                    continue
+                if re.match(r'# [1-9]', line):
+                    continue
+                match = re.match(r'#define (.*)', line)
+                if match:
+                    self.check_token(bad_tokens, match.group(1))
+                    continue
+                match = re.match(r'#undef (.*)', line)
+                if match:
+                    bad_tokens.discard(match.group(1))
+                    continue
+                # Tokenize the line and check identifiers found.  The
+                # handling of strings does not allow for escaped
+                # quotes, no allowance is made for character
+                # constants, and hex floats may be wrongly split into
+                # tokens including identifiers, but this is sufficient
+                # in practice and matches the old perl script.
+                line = re.sub(r'"[^"]*"', '', line)
+                line = line.strip()
+                for token in re.split(r'[^A-Za-z0-9_]+', line):
+                    if re.match(r'[A-Za-z_]', token):
+                        self.check_token(bad_tokens, token)
+        if bad_tokens:
+            for token in sorted(bad_tokens):
+                print('    Namespace violation: "%s"' % token)
+            self.note_error(name, False)
+        else:
+            print('PASS: %s' % name)
+            sys.stdout.flush()
+
+    def run(self):
+        """Load and run tests of a header."""
+        with tempfile.TemporaryDirectory() as self.temp_dir:
+            self.load_tests(self.header, False)
+            self.group_optional = False
+            self.group_xfail = False
+            self.group_ignore = False
+            self.group_skip = False
+            available = self.compile_test('Availability of <%s>' % self.header,
+                                          '')
+            if available:
+                for test in self.tests:
+                    # A test may run more than one subtest.  If the
+                    # initial subtest for an optional symbol fails,
+                    # others are not run at all; if the initial
+                    # subtest for an optional symbol succeeds, others
+                    # are run and are not considered optional; if the
+                    # initial subtest for a required symbol fails,
+                    # others are skipped.
+                    self.group_optional = test.optional
+                    self.group_xfail = test.xfail
+                    self.group_ignore = False
+                    self.group_skip = False
+                    test.run(self)
+            namespace_name = 'Namespace of <%s>' % self.header
+            if available:
+                self.check_namespace(namespace_name)
+            else:
+                self.note_skip(namespace_name)
+        print('-' * 76)
+        print('  Total number of tests   : %4d' % self.total)
+        print('  Number of failed tests  : %4d' % self.errors)
+        print('  Number of xfailed tests : %4d' % self.xerrors)
+        print('  Number of skipped tests : %4d' % self.skipped)
+        sys.exit(1 if self.errors else 0)
+
+
+def main():
+    """The main entry point."""
+    parser = argparse.ArgumentParser(description='Check header contents.')
+    parser.add_argument('--header', metavar='HEADER',
+                        help='name of header')
+    parser.add_argument('--standard', metavar='STD',
+                        help='standard to use when processing header')
+    parser.add_argument('--cc', metavar='CC',
+                        help='C compiler to use')
+    parser.add_argument('--flags', metavar='CFLAGS',
+                        help='Compiler flags to use with CC')
+    parser.add_argument('--cross', action='store_true',
+                        help='Do not run compiled test programs')
+    parser.add_argument('--xfail', metavar='COND',
+                        help='Name of condition for XFAILs')
+    args = parser.parse_args()
+    tests = HeaderTests(args.header, args.standard, args.cc, args.flags,
+                        args.cross, args.xfail)
+    tests.run()
+
+
+if __name__ == '__main__':
+    main()
diff --git a/conform/data/arpa/inet.h-data b/conform/data/arpa/inet.h-data
index d4ab6bb72a..040b8212c7 100644
--- a/conform/data/arpa/inet.h-data
+++ b/conform/data/arpa/inet.h-data
@@ -11,19 +11,19 @@ macro INET_ADDRSTRLEN
 macro INET6_ADDRSTRLEN
 
 // The following can be declared as functions, defined as macros or both:
-function uint32_t htonl (uint32_t);
-function uint16_t htons (uint16_t);
-function uint32_t ntohl (uint32_t);
-function uint16_t htons (uint16_t);
+function uint32_t htonl (uint32_t)
+function uint16_t htons (uint16_t)
+function uint32_t ntohl (uint32_t)
+function uint16_t htons (uint16_t)
 
-function in_addr_t inet_addr (const char*);
-function in_addr_t inet_lnaof (struct in_addr);
-function {struct in_addr} inet_makeaddr (in_addr_t, in_addr_t);
-function in_addr_t inet_netof (struct in_addr);
-function in_addr_t inet_network (const char *);
-function {char*} inet_ntoa (struct in_addr);
-function {const char*} inet_ntop (int, const void*, char*, socklen_t);
-function int inet_pton (int, const char*, void*);
+function in_addr_t inet_addr (const char*)
+function in_addr_t inet_lnaof (struct in_addr)
+function {struct in_addr} inet_makeaddr (in_addr_t, in_addr_t)
+function in_addr_t inet_netof (struct in_addr)
+function in_addr_t inet_network (const char *)
+function {char*} inet_ntoa (struct in_addr)
+function {const char*} inet_ntop (int, const void*, char*, socklen_t)
+function int inet_pton (int, const char*, void*)
 
 allow-header netinet/in.h
 allow-header inttypes.h
diff --git a/conform/data/fcntl.h-data b/conform/data/fcntl.h-data
index 2d5827f6ec..ffa4a351fb 100644
--- a/conform/data/fcntl.h-data
+++ b/conform/data/fcntl.h-data
@@ -119,7 +119,7 @@ constant AT_SYMLINK_NOFOLLOW
 constant AT_SYMLINK_FOLLOW
 constant AT_REMOVEDIR
 
-function int openat(int, const char*, int, ...)
+function int openat (int, const char*, int, ...)
 #endif
 
 #if !defined POSIX
diff --git a/conform/data/spawn.h-data b/conform/data/spawn.h-data
index de4aaa7fe2..be69922669 100644
--- a/conform/data/spawn.h-data
+++ b/conform/data/spawn.h-data
@@ -34,8 +34,8 @@ function int posix_spawn_file_actions_adddup2 (posix_spawn_file_actions_t*, int,
 function int posix_spawn_file_actions_addopen (posix_spawn_file_actions_t*, int, const char *, int, mode_t)
 function int posix_spawn_file_actions_destroy (posix_spawn_file_actions_t*)
 function int posix_spawn_file_actions_init (posix_spawn_file_actions_t*)
-function int posix_spawn (pid_t*, const char*, const posix_spawn_file_actions_t*, const posix_spawnattr_t*, char *const[], char *const[]);
-function int posix_spawnp (pid_t*, const char*, const posix_spawn_file_actions_t*, const posix_spawnattr_t*, char *const[], char *const[]);
+function int posix_spawn (pid_t*, const char*, const posix_spawn_file_actions_t*, const posix_spawnattr_t*, char *const[], char *const[])
+function int posix_spawnp (pid_t*, const char*, const posix_spawn_file_actions_t*, const posix_spawnattr_t*, char *const[], char *const[])
 
 allow-header sched.h
 allow-header signal.h
diff --git a/conform/data/termios.h-data b/conform/data/termios.h-data
index 9aec2f5a17..bb2092bb6f 100644
--- a/conform/data/termios.h-data
+++ b/conform/data/termios.h-data
@@ -13,7 +13,7 @@ element {struct termios} tcflag_t c_iflag
 element {struct termios} tcflag_t c_oflag
 element {struct termios} tcflag_t c_cflag
 element {struct termios} tcflag_t c_lflag
-element {struct termios} cc_t c_cc[NCCS]
+element {struct termios} cc_t c_cc [NCCS]
 
 constant NCCS
 
diff --git a/conform/data/wchar.h-data b/conform/data/wchar.h-data
index 0beae8957d..e414651a33 100644
--- a/conform/data/wchar.h-data
+++ b/conform/data/wchar.h-data
@@ -77,8 +77,8 @@ function {wchar_t*} wcpncpy (wchar_t*, const wchar_t*, size_t)
 # endif
 function size_t wcrtomb (char*, wchar_t, mbstate_t*)
 # if defined XOPEN2K8 || defined POSIX2008
-function int wcscasecmp(const wchar_t*, const wchar_t*)
-function int wcscasecmp_l(const wchar_t*, const wchar_t*, locale_t)
+function int wcscasecmp (const wchar_t*, const wchar_t*)
+function int wcscasecmp_l (const wchar_t*, const wchar_t*, locale_t)
 # endif
 function {wchar_t*} wcscat (wchar_t*, const wchar_t*)
 function {wchar_t*} wcschr (const wchar_t*, wchar_t)
diff --git a/conform/glibcconform.py b/conform/glibcconform.py
index c1db46029d..c465f42fc2 100644
--- a/conform/glibcconform.py
+++ b/conform/glibcconform.py
@@ -35,6 +35,28 @@ CFLAGS = {'ISO': '-ansi',
           'XOPEN2K8': '-std=c99 -D_XOPEN_SOURCE=700',
           'POSIX2008': '-std=c99 -D_POSIX_C_SOURCE=200809L'}
 
+# ISO C90 keywords.
+KEYWORDS_C90 = {'auto', 'break', 'case', 'char', 'const', 'continue',
+                'default', 'do', 'double', 'else', 'enum', 'extern', 'float',
+                'for', 'goto', 'if', 'int', 'long', 'register', 'return',
+                'short', 'signed', 'sizeof', 'static', 'struct', 'switch',
+                'typedef', 'union', 'unsigned', 'void', 'volatile', 'while'}
+
+# ISO C99 keywords.
+KEYWORDS_C99 = KEYWORDS_C90 | {'inline', 'restrict'}
+
+# Keywords for each standard.
+KEYWORDS = {'ISO': KEYWORDS_C90,
+            'ISO99': KEYWORDS_C99,
+            'ISO11': KEYWORDS_C99,
+            'POSIX': KEYWORDS_C90,
+            'XPG4': KEYWORDS_C90,
+            'XPG42': KEYWORDS_C90,
+            'UNIX98': KEYWORDS_C90,
+            'XOPEN2K': KEYWORDS_C99,
+            'XOPEN2K8': KEYWORDS_C99,
+            'POSIX2008': KEYWORDS_C99}
+
 
 def list_exported_functions(cc, standard, header):
     """Return the set of functions exported by a header, empty if an

--
Joseph S. Myers
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replace conformtest.pl with conformtest.py [committed]

Florian Weimer-5
* Joseph Myers:

> Continuing the consolidation on Python for various miscellaneous build
> and test scripts, this patch moves conformtest from Perl to Python.

For me, this increases test time for conform/ from 97 seconds to
126 seconds, perhaps due to the increased startup overhead of Python
compared to Perl.

Is there a way we can recover some of the additional overhead?  Perhaps
by doing more work in a single Python invocation?

Thanks,
Florian
Reply | Threaded
Open this post in threaded view
|

Re: Replace conformtest.pl with conformtest.py [committed]

Joseph Myers
On Fri, 9 Nov 2018, Florian Weimer wrote:

> * Joseph Myers:
>
> > Continuing the consolidation on Python for various miscellaneous build
> > and test scripts, this patch moves conformtest from Perl to Python.
>
> For me, this increases test time for conform/ from 97 seconds to
> 126 seconds, perhaps due to the increased startup overhead of Python
> compared to Perl.
>
> Is there a way we can recover some of the additional overhead?  Perhaps
> by doing more work in a single Python invocation?

The division into separate processes for each (standard, header) pair
(some time ago) was to enable those tests to run in parallel (apart from
allowing more precise XFAILing).  So I don't think recombining tests into
a single Python invocation, and so losing that parallelism, is such a good
idea.

What ought to be plausible is to reduce the number of *compiler*
invocations - further separate generating the text of a test from actually
running it, arrange for tests to use automatically-generated unique
identifiers instead of hardcoded identifiers, and then concatenate all
tests that aren't "optional" or "xfail" into a single file, trying
compiling that and only compiling the individual tests separately if
building the combined file fails.  (The vast bulk of time spent running
these tests is in the compiler; you can get more specific figures for your
system by replacing the subprocess.check_call call in compile_test with
"pass".  I don't know how much of that is compiler startup overhead that
would be saved by using a combined compiler execution.)

--
Joseph S. Myers
[hidden email]