nightly checkout and testing script

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

nightly checkout and testing script

William Cohen
To make sure systemtap tests are run on a regular basis I have set up
cron jobs on my local machines (FC4/Rawhide and a couple RHEL4-U3 beta
machines) to check out the current systemtap sources and tests,
configure them, build them, and run the tests nightly at 4:30am Eastern
Time. The attached simple-minded script is what is run. The script
expects that there is a copy of the elfutils sources around in the
appropriate place. The script sets up a clean directory to do this work
in, systemtap_${date_time}

The script doesn't send report back the results yet. That is the next step.

Any comments on the script would be appreciated.

-Will

#! /bin/bash
#
# A simple script to automate checkup and running of the tests
#
#
# Will Cohen
# 20060116

#variable definitions

export DATE_TIME=`date +%Y%m%d%H%M`
export SYSTEMTAP=~/systemtap_${DATE_TIME}
export ELFUTILS=~/rh-rpms/BUILD/elfutils-0.118
export STAP_OBJ=${SYSTEMTAP}/obj
export STAP_SRC=${SYSTEMTAP}/src
export LD_LIBRARY_PATH=${STAP_OBJ}/lib-elfutils:${STAP_OBJ}/lib-elfutils/systemtap:$LD_LIBRARY_PATH
export PATH=${STAP_OBJ}:$PATH
export SYSTEMTAP_TAPSET=${STAP_SRC}/tapset
export SYSTEMTAP_RUNTIME=${STAP_SRC}/runtime
export STAP_TESTS=${SYSTEMTAP}/tests
export CVSROOT=:pserver:[hidden email]:/cvs/systemtap

check_error()
{
  if test $1 != 0; then
    echo $2
    exit $1
  fi
}


#check out src and test directory
mkdir -p ${STAP_OBJ}
cd ${SYSTEMTAP}
#echo "anonymous\n" > cvs login
cvs co src
check_error $? "problem checking out src"
cvs co tests
check_error $? "problem checking out tests"

#configure and build
cd ${STAP_OBJ}
${STAP_SRC}/configure --with-elfutils=${ELFUTILS}
check_error $? "problem configure"
make gcov >& ${DATE_TIME}.problems
check_error $? "problem building stap"

# do the runtime tests with test coverage
cd ${STAP_TESTS}/testsuite
mkdir -p ./coverage
#lcov --directory $STAP_OBJ --zerocounters
runtest --tool=systemtap
#lcov --directory $STAP_OBJ --capture --output-file ./coverage/stap.info
#genhtml -o ./coverage ./coverage/stap.info

# FIXME Extract the bits of information and report the results
# Pull out the FAILS and summary from the testsuite
# list architecture
# send the email
Reply | Threaded
Open this post in threaded view
|

Re: nightly checkout and testing script

William Cohen
I have made a number of revisions to the testing script. It nows sends a
summary of the test results. I just have it sending me the email, but it
would be trivial to have the results be mailed to the systemtap mailing
list. Do people want me to set this to automatically send the test
results to the systemtap mailing list? Could we set this up on ppc64 and
ia64 machines to keep watch of how well the translator is working for them.

I have a crontab job like the following set up to run the tests nightly
in the early morning:

$ crontab -l
30 04 * * * /home/wcohen/bin/stap_testing > /tmp/stap_testing.log

-Will

#! /bin/bash
#
# A simple script to automate checkup and running of the tests
#
#
# Will Cohen
# 20060116

#variable definitions

export DATE_TIME=`date -u +%Y%m%d%H%M`
export USER=`whoami`
export KERNEL=`uname -s -r -v -m -p -i -o`
export TEST_NAME=stap_testing_${DATE_TIME}
export SYSTEMTAP=~/${TEST_NAME}
export ELFUTILS=~/rh-rpms/BUILD/elfutils-0.119
export STAP_OBJ=${SYSTEMTAP}/obj
export STAP_SRC=${SYSTEMTAP}/src
export LD_LIBRARY_PATH=${STAP_OBJ}/lib-elfutils:${STAP_OBJ}/lib-elfutils/systemtap:$LD_LIBRARY_PATH
export PATH=${STAP_OBJ}:$PATH
export SYSTEMTAP_TAPSET=${STAP_SRC}/tapset
export SYSTEMTAP_RUNTIME=${STAP_SRC}/runtime
export STAP_TESTS=${SYSTEMTAP}/tests
export CVSROOT=:pserver:[hidden email]:/cvs/systemtap
export MAIL_TO_ADDR=[hidden email]
export BUILD_RESULTS=${STAP_OBJ}/${DATE_TIME}.problems
export TEST_RESULTS=/tmp/${TEST_NAME}.results

check_error()
{
  if test $1 != 0; then
    echo $2
    exit $1
  fi
}


#check out src and test directory
mkdir -p ${STAP_OBJ}
cd ${SYSTEMTAP}
echo "anonymous\n" > cvs login
cvs co src
check_error $? "problem checking out src"
cvs co tests
check_error $? "problem checking out tests"

#configure and build
cd ${STAP_OBJ}
${STAP_SRC}/configure --with-elfutils=${ELFUTILS}
check_error $? "problem configure"
make gcov >& ${BUILD_RESULTS}
check_error $? "problem building stap"

# do the runtime tests with test coverage
cd ${STAP_TESTS}/testsuite
mkdir -p ./coverage
#lcov --directory $STAP_OBJ --zerocounters
runtest --tool=systemtap
#lcov --directory $STAP_OBJ --capture --output-file ./coverage/stap.info
#genhtml -o ./coverage ./coverage/stap.info

# Extract the bits of information and report the results
# list architecture
# Pull out the FAILS for translator and summary from the testsuite

echo "Date: ${DATE_TIME}" > ${TEST_RESULTS}
echo "User: ${USER}" >> ${TEST_RESULTS}
echo "Kernel: ${KERNEL}" >> ${TEST_RESULTS}
echo -e "\nStap translator summary of failed tests" >> ${TEST_RESULTS}
cd ${STAP_OBJ}; find -name "*.err" -exec basename {} .err \;  >> ${TEST_RESULTS}
echo -e "\nTestsuite summary of failed tests" >> ${TEST_RESULTS}
cd ${STAP_TESTS}/testsuite
grep "^FAIL:" systemtap.sum >> ${TEST_RESULTS}
awk '/=== systemtap Summary ===/,EOF' systemtap.log >> ${TEST_RESULTS}

# send the email
cat ${TEST_RESULTS} | mail -s "${TEST_RESULTS}" $MAIL_TO_ADDR