testsuits on IA64

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

testsuits on IA64

Keshavamurthy, Anil S
I am trying to run testsuits (dejagnu) on IA64 and most of the tests
fails on Ia64 with FAIL : timeout.
How do I  increase the timeout value?

Thanks,

-Anil Keshavamurthy
Sr. Software Engineer
Open Source Technology Center/SSG
Intel Corp.
(w) 503-712-4476
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

Frank Ch. Eigler
Hi -

> I am trying to run testsuits (dejagnu) on IA64 and most of the tests
> fails on Ia64 with FAIL : timeout.
> How do I increase the timeout value?

The dejagnu ".exp" files contain timeout overrides.  However, they are
already fairly generous (30-60 seconds).  How long does it take on
your machine to get one of these probes started?

- FChE
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

Keshavamurthy, Anil S
On Fri, Nov 04, 2005 at 02:00:15PM -0500, Frank Ch. Eigler wrote:
> Hi -
>
> > I am trying to run testsuits (dejagnu) on IA64 and most of the tests
> > fails on Ia64 with FAIL : timeout.
> > How do I increase the timeout value?
>
> The dejagnu ".exp" files contain timeout overrides.  However, they are
> already fairly generous (30-60 seconds).  How long does it take on
> your machine to get one of these probes started?

Usually it is 6 to 10 seconds if I run the same test manually.
Currently I am just running "runtest" with no options.

Let me know if you need log file or any other details.

-Anil
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

Frank Ch. Eigler
Hi -

> > [...]  How long does it take on
> > your machine to get one of these probes started?
>
> Usually it is 6 to 10 seconds if I run the same test manually.
> Currently I am just running "runtest" with no options. [...]

OK, that suggests that it's not an ordinary timeout problem.  I have
encountered your symptoms on an ordinary machine, when I started the
test suite from a user unprivileged to do prompt-less sudo.  Then the
"sudo stpd" child of systemtap would wait for a password, and never
get it (since dejagnu/expect never sends one).

- FChE
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

William Cohen
In reply to this post by Keshavamurthy, Anil S
Keshavamurthy Anil S wrote:

> On Fri, Nov 04, 2005 at 02:00:15PM -0500, Frank Ch. Eigler wrote:
>
>>Hi -
>>
>>
>>>I am trying to run testsuits (dejagnu) on IA64 and most of the tests
>>>fails on Ia64 with FAIL : timeout.
>>>How do I increase the timeout value?
>>
>>The dejagnu ".exp" files contain timeout overrides.  However, they are
>>already fairly generous (30-60 seconds).  How long does it take on
>>your machine to get one of these probes started?
>
>
> Usually it is 6 to 10 seconds if I run the same test manually.
> Currently I am just running "runtest" with no options.
>
> Let me know if you need log file or any other details.
>
> -Anil

Could you send the systemtap.log file? Are you seeing the "systemtap
starting probe" and "systemtap test success" messages in the systemtap.log?

-Will
Reply | Threaded
Open this post in threaded view
|

RE: testsuits on IA64

Keshavamurthy, Anil S
In reply to this post by Keshavamurthy, Anil S
Will,
I am attaching log file. Your help is appreciated.

Thanks,
Anil Keshavamurthy

>-----Original Message-----
>From: [hidden email]
>[mailto:[hidden email]] On Behalf Of William Cohen
>Sent: Friday, November 04, 2005 3:04 PM
>To: Keshavamurthy, Anil S
>Cc: Frank Ch. Eigler; [hidden email]
>Subject: Re: testsuits on IA64
>
>Keshavamurthy Anil S wrote:
>> On Fri, Nov 04, 2005 at 02:00:15PM -0500, Frank Ch. Eigler wrote:
>>
>>>Hi -
>>>
>>>
>>>>I am trying to run testsuits (dejagnu) on IA64 and most of the tests
>>>>fails on Ia64 with FAIL : timeout.
>>>>How do I increase the timeout value?
>>>
>>>The dejagnu ".exp" files contain timeout overrides.  
>However, they are
>>>already fairly generous (30-60 seconds).  How long does it take on
>>>your machine to get one of these probes started?
>>
>>
>> Usually it is 6 to 10 seconds if I run the same test manually.
>> Currently I am just running "runtest" with no options.
>>
>> Let me know if you need log file or any other details.
>>
>> -Anil
>
>Could you send the systemtap.log file? Are you seeing the "systemtap
>starting probe" and "systemtap test success" messages in the
>systemtap.log?
>
>-Will
>

testrun.log (17K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

Hien Nguyen
It looks like memory alignment problem for some of the tests. Run
stap equal.stp

then look at your /var/log/messages
Hien.
Keshavamurthy, Anil S wrote:

>Will,
>I am attaching log file. Your help is appreciated.
>
>Thanks,
>Anil Keshavamurthy
>
>  
>
>>-----Original Message-----
>>From: [hidden email]
>>[mailto:[hidden email]] On Behalf Of William Cohen
>>Sent: Friday, November 04, 2005 3:04 PM
>>To: Keshavamurthy, Anil S
>>Cc: Frank Ch. Eigler; [hidden email]
>>Subject: Re: testsuits on IA64
>>
>>Keshavamurthy Anil S wrote:
>>    
>>
>>>On Fri, Nov 04, 2005 at 02:00:15PM -0500, Frank Ch. Eigler wrote:
>>>
>>>      
>>>
>>>>Hi -
>>>>
>>>>
>>>>        
>>>>
>>>>>I am trying to run testsuits (dejagnu) on IA64 and most of the tests
>>>>>fails on Ia64 with FAIL : timeout.
>>>>>How do I increase the timeout value?
>>>>>          
>>>>>
>>>>The dejagnu ".exp" files contain timeout overrides.  
>>>>        
>>>>
>>However, they are
>>    
>>
>>>>already fairly generous (30-60 seconds).  How long does it take on
>>>>your machine to get one of these probes started?
>>>>        
>>>>
>>>Usually it is 6 to 10 seconds if I run the same test manually.
>>>Currently I am just running "runtest" with no options.
>>>
>>>Let me know if you need log file or any other details.
>>>
>>>-Anil
>>>      
>>>
>>Could you send the systemtap.log file? Are you seeing the "systemtap
>>starting probe" and "systemtap test success" messages in the
>>systemtap.log?
>>
>>-Will
>>
>>    
>>

Reply | Threaded
Open this post in threaded view
|

RE: testsuits on IA64

bibo,mao-2
In reply to this post by Keshavamurthy, Anil S
The same phenomenon exists on EM64T, when I enter "runtest" command, and in another console enter "ps -aux", there will be many defunct processes like:
root     20648  0.3  0.0     0    0 ?      Zs   09:54   0:00 [stap] <defunct>
root     20755  0.3  0.0     0    0 ?      Zs   09:54   0:00 [stap] <defunct>
root     20861  0.3  0.0     0    0 ?      Zs   09:55   0:00 [stap] <defunct>
root     20972  0.3  0.0     0    0 ?      Zs   09:55   0:00 [stap] <defunct>
root     21075  0.3  0.0     0    0 ?      Zs   09:55   0:00 [stap] <defunct>
root     21181  0.3  0.0     0    0 ?      Zs   09:55   0:00 [stap] <defunct>

But if I divided systemtap.base directory into two parts: systemtap1 and systemtap2, and test it by "runtest -di systemtap1" and "runtest -di systemtap2". It will run well.


>-----Original Message-----
>From: [hidden email] [mailto:[hidden email]]
>On Behalf Of Hien Nguyen
>Sent: 2005年11月5日 7:47
>To: Keshavamurthy, Anil S
>Cc: William Cohen; Frank Ch. Eigler; [hidden email]
>Subject: Re: testsuits on IA64
>
>It looks like memory alignment problem for some of the tests. Run
>stap equal.stp
>
>then look at your /var/log/messages
>Hien.
>Keshavamurthy, Anil S wrote:
>
>>Will,
>>I am attaching log file. Your help is appreciated.
>>
>>Thanks,
>>Anil Keshavamurthy
>>
>>
>>
>>>-----Original Message-----
>>>From: [hidden email]
>>>[mailto:[hidden email]] On Behalf Of William Cohen
>>>Sent: Friday, November 04, 2005 3:04 PM
>>>To: Keshavamurthy, Anil S
>>>Cc: Frank Ch. Eigler; [hidden email]
>>>Subject: Re: testsuits on IA64
>>>
>>>Keshavamurthy Anil S wrote:
>>>
>>>
>>>>On Fri, Nov 04, 2005 at 02:00:15PM -0500, Frank Ch. Eigler wrote:
>>>>
>>>>
>>>>
>>>>>Hi -
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>I am trying to run testsuits (dejagnu) on IA64 and most of the tests
>>>>>>fails on Ia64 with FAIL : timeout.
>>>>>>How do I increase the timeout value?
>>>>>>
>>>>>>
>>>>>The dejagnu ".exp" files contain timeout overrides.
>>>>>
>>>>>
>>>However, they are
>>>
>>>
>>>>>already fairly generous (30-60 seconds).  How long does it take on
>>>>>your machine to get one of these probes started?
>>>>>
>>>>>
>>>>Usually it is 6 to 10 seconds if I run the same test manually.
>>>>Currently I am just running "runtest" with no options.
>>>>
>>>>Let me know if you need log file or any other details.
>>>>
>>>>-Anil
>>>>
>>>>
>>>Could you send the systemtap.log file? Are you seeing the "systemtap
>>>starting probe" and "systemtap test success" messages in the
>>>systemtap.log?
>>>
>>>-Will
>>>
>>>
>>>

Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

William Cohen
In reply to this post by Keshavamurthy, Anil S
Keshavamurthy, Anil S wrote:
> Will,
> I am attaching log file. Your help is appreciated.
>
> Thanks,
> Anil Keshavamurthy

I took a look at the resulting test log.

Seeing unaligned accesses on ia64 like the following (from
equal.stp)for a number of the tests:

stap(9651): unaligned access to 0x200000000311c016, ip=0x2000000000070710
stap(9651): unaligned access to 0x200000000311c016, ip=0x2000000000070710
stap(9651): unaligned access to 0x200000000311c035, ip=0x2000000000070c10
stap(9651): unaligned access to 0x200000000311c071, ip=0x2000000000073190

This appears to be in userspace. Which process and location does this
map back to? Is this the systemtap daemon that is started when the
systemtap kernel module is loaded?

Could you run "stap -V -k ./systemtap.base/kmodule.stp" and send the
output? The kmodule test failed and it would be good to know why.

Could you try to run ./systemtap.base/simple.stp by hand? It failed on
startup. Test after it also seem to fail startup. I am wondering if
that failure caused the other startup failures.



-Will
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

William Cohen
William Cohen wrote:

> Could you run "stap -V -k ./systemtap.base/kmodule.stp" and send the
> output? The kmodule test failed and it would be good to know why.

wrong case for verbose.

stap -v -k ./systemtap.base/kmodule.stp

-Will
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

Keshavamurthy, Anil S
In reply to this post by William Cohen
On Tue, Nov 08, 2005 at 11:14:46AM -0500, William Cohen wrote:

> Seeing unaligned accesses on ia64 like the following (from
> equal.stp)for a number of the tests:
>
> stap(9651): unaligned access to 0x200000000311c016, ip=0x2000000000070710
> stap(9651): unaligned access to 0x200000000311c016, ip=0x2000000000070710
> stap(9651): unaligned access to 0x200000000311c035, ip=0x2000000000070c10
> stap(9651): unaligned access to 0x200000000311c071, ip=0x2000000000073190
>
> This appears to be in userspace. Which process and location does this
> map back to? Is this the systemtap daemon that is started when the
> systemtap kernel module is loaded?
The above unaligned access got fixed when I upgraded elfutils to elfutils-0.116.
http://sourceware.org/bugzilla/show_bug.cgi?id=1340


>
> Could you run "stap -V -k ./systemtap.base/kmodule.stp" and send the
> output? The kmodule test failed and it would be good to know why.
Attaching the output.

>
> Could you try to run ./systemtap.base/simple.stp by hand? It failed on
> startup. Test after it also seem to fail startup. I am wondering if
> that failure caused the other startup failures.
Also when I run the individual test by hand i.e stap -v ./systemtap.base/test_name.stp
all the tests passes.

-Anil Keshavamurthy

kmodule_source.tgz (60K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

William Cohen
Keshavamurthy Anil S wrote:

>>Could you run "stap -V -k ./systemtap.base/kmodule.stp" and send the
>>output? The kmodule test failed and it would be good to know why.
>
> Attaching the output.

I don't have a ia64 machine handy to try things out on. Could you send
me the output that comes on the console from "stap -v -k
./systemtap.base/kmodule.stp"? I would like to see that error messages
the compiler is producing when it attempts to produce the module.

-Will
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

Keshavamurthy, Anil S
On Wed, Nov 09, 2005 at 10:07:52AM -0500, William Cohen wrote:
> Keshavamurthy Anil S wrote:
>
> I don't have a ia64 machine handy to try things out on. Could you send
> me the output that comes on the console from "stap -v -k
> ./systemtap.base/kmodule.stp"? I would like to see that error messages
> the compiler is producing when it attempts to produce the module.

Hi Will,
        Here goes the output you requested, as mentioned earlier I don't have any problems
running individual tap scripts manually. It is only when I run runtest I see timeouts.

Thanks,
Anil

[root@csdor-tiger1 testsuite]# stap -v -k systemtap.base/kmodule.stp
Created temporary directory "/tmp/stapLpQ3QC"
Searched '/usr/local/share/systemtap/tapset/2.6.9-22.10.EL/*.stp', match count 0
Searched '/usr/local/share/systemtap/tapset/2.6.9/*.stp', match count 0
Searched '/usr/local/share/systemtap/tapset/2.6/*.stp', match count 0
Searched '/usr/local/share/systemtap/tapset/*.stp', match count 9
Pass 1: parsed user script and 9 library script(s).
parsed 'ext3_sync_file' -> func 'ext3_sync_file'
pattern 'ext3' matches module 'ext3'
focused on module 'ext3' = [a00000020014c000-a000000200192db8, bias 0]
pattern 'ext3_sync_file' matches function 'ext3_sync_file'
selected function ext3_sync_file
prologue disagreement: ext3_sync_file heur0=a000000200153981 heur1=a000000200153982
querying entrypc of function 'ext3_sync_file'
Pass 2: analyzed user script.  3 probe(s), 10 function(s), 1 global(s).
Running grep " [tT] " /proc/kallsyms | sort -k 1,16 -s -o /tmp/stapLpQ3QC/symbols.sorted
Pass 3: translated to C into "/tmp/stapLpQ3QC/stap_15614.c"
Running make -C "/lib/modules/2.6.9-22.10.EL/build" M="/tmp/stapLpQ3QC" modules
make: Entering directory `/usr/src/kernels/2.6.9-22.10.EL-ia64'
  CC [M]  /tmp/stapLpQ3QC/stap_15614.o
  Building modules, stage 2.
  MODPOST
  CC      /tmp/stapLpQ3QC/stap_15614.mod.o
  LD [M]  /tmp/stapLpQ3QC/stap_15614.ko
make: Leaving directory `/usr/src/kernels/2.6.9-22.10.EL-ia64'
Pass 4: compiled into "stap_15614.ko"
Running sudo /usr/local/libexec/systemtap/stpd -r /tmp/stapLpQ3QC/stap_15614.ko
systemtap starting probe
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

William Cohen
Keshavamurthy Anil S wrote:

> On Wed, Nov 09, 2005 at 10:07:52AM -0500, William Cohen wrote:
>
>>Keshavamurthy Anil S wrote:
>>
>>I don't have a ia64 machine handy to try things out on. Could you send
>>me the output that comes on the console from "stap -v -k
>>./systemtap.base/kmodule.stp"? I would like to see that error messages
>>the compiler is producing when it attempts to produce the module.
>
>
> Hi Will,
> Here goes the output you requested, as mentioned earlier I don't have any problems
> running individual tap scripts manually. It is only when I run runtest I see timeouts.

Hi Anil,

So the kmodule.stp test compiles and runs fine by itself? There was the
following in the testrun.log:

Running ./systemtap.base/kmodule.exp ...
semantic error: FAIL: ./systemtap.base/kmodule.stp compilation

I assumed that there was a problem with the compilation. So the
individual tests work when run, e.g. "runtest --tool=systemtap
./systemtap.base/kmodule.exp"? What do the test results look with the
newer version of elfutils? Still lots of failures?

-Will
Reply | Threaded
Open this post in threaded view
|

Re: testsuits on IA64

Keshavamurthy, Anil S
On Wed, Nov 09, 2005 at 02:26:05PM -0500, William Cohen wrote:

> Keshavamurthy Anil S wrote:
> > On Wed, Nov 09, 2005 at 10:07:52AM -0500, William Cohen wrote:
> >
> >>Keshavamurthy Anil S wrote:
> >>
> >>I don't have a ia64 machine handy to try things out on. Could you send
> >>me the output that comes on the console from "stap -v -k
> >>./systemtap.base/kmodule.stp"? I would like to see that error messages
> >>the compiler is producing when it attempts to produce the module.
> >
> >
> > Hi Will,
> > Here goes the output you requested, as mentioned earlier I don't have any problems
> > running individual tap scripts manually. It is only when I run runtest I see timeouts.
>
> Hi Anil,
>
> So the kmodule.stp test compiles and runs fine by itself? There was the
> following in the testrun.log:
>
> Running ./systemtap.base/kmodule.exp ...
> semantic error: FAIL: ./systemtap.base/kmodule.stp compilation
All the compile issues are gone(fixed) with elfutils-0.116.
Now I don;t have the above compile issues.
>
> I assumed that there was a problem with the compilation. So the
> individual tests work when run, e.g. "runtest --tool=systemtap
> ./systemtap.base/kmodule.exp"? What do the test results look with the
> newer version of elfutils? Still lots of failures?

With newer elfutils though I still see timeout(attaching testrun.log), it is
much better as I don't have any complile issues.

Also when I run the individual failed(timout test) as "runtest --tool=systemtap
./systemtap.base/simple.exp" i.e one test at a time, I don't have any issues.
The systemtap.sum and systemtap.log both will have results as PASS.
It would be better if we can root cause the timeout issue but not a burning
issues at this point in time as compile issue is resolved and I know all the tests works :-)

-Anil







>
> -Will

testrun.log (12K) Download Attachment