Tweaking default java.awt.Robot settings

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Tweaking default java.awt.Robot settings

Steve McKay☄
Hi All,

I've noticed that at least some of the tests using java.awt.Robot are
non-deterministic due to lags is the underlying window system. The
java.awt.Component.keyPressTest, for example, fails some of the time
(on linux, windows, linux+wine, ...). It looks like enabling
autoWaitForIdle (waits for the awt EventQueue to be empty before
adding new events to the queue), and setting autoDelay (pauses for an
arbitrary period of time) to some magic number of millis well above
zero (I use 100) significantly reduces failures. Would anyone object
to configuring the Robot with settings like this by default? If no,
should the config mechanism be updated to allow tweaking these
settings?

--
Steve McKay <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

David Herron @ Sun
Steve McKay☄ wrote:

> Hi All,
>
> I've noticed that at least some of the tests using java.awt.Robot are
> non-deterministic due to lags is the underlying window system. The
> java.awt.Component.keyPressTest, for example, fails some of the time
> (on linux, windows, linux+wine, ...). It looks like enabling
> autoWaitForIdle (waits for the awt EventQueue to be empty before
> adding new events to the queue), and setting autoDelay (pauses for an
> arbitrary period of time) to some magic number of millis well above
> zero (I use 100) significantly reduces failures. Would anyone object
> to configuring the Robot with settings like this by default? If no,
> should the config mechanism be updated to allow tweaking these
> settings?
>
>  

I don't know what the classpath implementation of Robot looks like, but
I do know what Sun's Linux/Unix implementation looks like (having
written the original version).

Generally Robot has to request the OS or X11 to synthesize the event.  
On Windows there's a direct API call, while on Unix/Linux there is a
child process which ends up calling XTEST extension methods.  In both
cases it means there is a nondeterministic delay due to the current
process scheduling characteristics of the given system.  In other words
it depends on an external entity, who Robot cannot coerce into
performing the request within a bounded set of time.

I think that means depending on Robot doing it's thing within a given
period of time is an invalid test.

Robot does not add events to EventQueue but it requests the OS to
synthesize an OS-level event.

- David Herron

Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Steve McKay☄
So would you recommend I ignore the test, delete it, add a comment, ...?

--steve

On 9/24/07, David Herron <[hidden email]> wrote:

> Steve McKay☄ wrote:
> > Hi All,
> >
> > I've noticed that at least some of the tests using java.awt.Robot are
> > non-deterministic due to lags is the underlying window system. The
> > java.awt.Component.keyPressTest, for example, fails some of the time
> > (on linux, windows, linux+wine, ...). It looks like enabling
> > autoWaitForIdle (waits for the awt EventQueue to be empty before
> > adding new events to the queue), and setting autoDelay (pauses for an
> > arbitrary period of time) to some magic number of millis well above
> > zero (I use 100) significantly reduces failures. Would anyone object
> > to configuring the Robot with settings like this by default? If no,
> > should the config mechanism be updated to allow tweaking these
> > settings?
> >
> >
>
> I don't know what the classpath implementation of Robot looks like, but
> I do know what Sun's Linux/Unix implementation looks like (having
> written the original version).
>
> Generally Robot has to request the OS or X11 to synthesize the event.
> On Windows there's a direct API call, while on Unix/Linux there is a
> child process which ends up calling XTEST extension methods.  In both
> cases it means there is a nondeterministic delay due to the current
> process scheduling characteristics of the given system.  In other words
> it depends on an external entity, who Robot cannot coerce into
> performing the request within a bounded set of time.
>
> I think that means depending on Robot doing it's thing within a given
> period of time is an invalid test.
>
> Robot does not add events to EventQueue but it requests the OS to
> synthesize an OS-level event.
>
> - David Herron
>
>


--
Steve McKay <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

David Herron @ Sun
[resending because the mailing list thingy told me I had to use only
plain text messages...]

Hmmm.. here's the meat of the test

  public void runTest(int code, char chr)
  {
    KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr,
KeyEvent.KEY_LOCATION_STANDARD);
    f.dispatchEvent(e);
   
   
    f.setSize(200,200);
    f.show();
    r.mouseMove(60, 60);

    r.keyPress(code);
    r.keyRelease(code);
    h.check(key, (int) chr);
  }

I don't understand this.  If you're going to create a Java event why use
Robot, or vice versa...?

It appears h.check is in gnu.testlet.TestHarness and that it simply does
an immediate check with no waiting.  The dispatchEvent call is going to
cause the listener to fire regardless of what's happening using Robot.

This looks like an incorrect test, and what I'd recommend is:-

a) ditch the two lines saying KeyEvent / dispatchEvent ... they are
completely subverting the intent of the test

b) insert some code so the runTest method waits for the listener to be
triggered.  Such as a wait and notify type of semaphor.

c) I don't know how the test guarantees runTest executes on the event
dispatch thread.  Is the EDT as important to classpath as it is to Sun's
Swing?

- David Herron


Steve McKay☄ wrote:

> So would you recommend I ignore the test, delete it, add a comment, ...?
>
> --steve
>
> On 9/24/07, David Herron <[hidden email]> wrote:
>  
>> Steve McKay☄ wrote:
>>    
>>> Hi All,
>>>
>>> I've noticed that at least some of the tests using java.awt.Robot are
>>> non-deterministic due to lags is the underlying window system. The
>>> java.awt.Component.keyPressTest, for example, fails some of the time
>>> (on linux, windows, linux+wine, ...). It looks like enabling
>>> autoWaitForIdle (waits for the awt EventQueue to be empty before
>>> adding new events to the queue), and setting autoDelay (pauses for an
>>> arbitrary period of time) to some magic number of millis well above
>>> zero (I use 100) significantly reduces failures. Would anyone object
>>> to configuring the Robot with settings like this by default? If no,
>>> should the config mechanism be updated to allow tweaking these
>>> settings?
>>>
>>>
>>>      
>> I don't know what the classpath implementation of Robot looks like, but
>> I do know what Sun's Linux/Unix implementation looks like (having
>> written the original version).
>>
>> Generally Robot has to request the OS or X11 to synthesize the event.
>> On Windows there's a direct API call, while on Unix/Linux there is a
>> child process which ends up calling XTEST extension methods.  In both
>> cases it means there is a nondeterministic delay due to the current
>> process scheduling characteristics of the given system.  In other words
>> it depends on an external entity, who Robot cannot coerce into
>> performing the request within a bounded set of time.
>>
>> I think that means depending on Robot doing it's thing within a given
>> period of time is an invalid test.
>>
>> Robot does not add events to EventQueue but it requests the OS to
>> synthesize an OS-level event.
>>
>> - David Herron
>>
>>
>>    
>
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Thomas Fitzsimmons
In reply to this post by Steve McKay☄
Steve McKay☄ wrote:

> Hi All,
>
> I've noticed that at least some of the tests using java.awt.Robot are
> non-deterministic due to lags is the underlying window system. The
> java.awt.Component.keyPressTest, for example, fails some of the time
> (on linux, windows, linux+wine, ...). It looks like enabling
> autoWaitForIdle (waits for the awt EventQueue to be empty before
> adding new events to the queue), and setting autoDelay (pauses for an
> arbitrary period of time) to some magic number of millis well above
> zero (I use 100) significantly reduces failures. Would anyone object
> to configuring the Robot with settings like this by default?

No objections here, that sounds like a good idea.  When you say "significantly
reduces failures", does that mean some non-deterministic failures persist with
the new auto* settings?

Tom
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Steve McKay☄
Thomas,

I think David's suggestion was better, so I'll work that up instead of
the autoDelay changes I proposed.

--steve

On 9/25/07, Thomas Fitzsimmons <[hidden email]> wrote:

> Steve McKay☄ wrote:
> > Hi All,
> >
> > I've noticed that at least some of the tests using java.awt.Robot are
> > non-deterministic due to lags is the underlying window system. The
> > java.awt.Component.keyPressTest, for example, fails some of the time
> > (on linux, windows, linux+wine, ...). It looks like enabling
> > autoWaitForIdle (waits for the awt EventQueue to be empty before
> > adding new events to the queue), and setting autoDelay (pauses for an
> > arbitrary period of time) to some magic number of millis well above
> > zero (I use 100) significantly reduces failures. Would anyone object
> > to configuring the Robot with settings like this by default?
>
> No objections here, that sounds like a good idea.  When you say "significantly
> reduces failures", does that mean some non-deterministic failures persist with
> the new auto* settings?
>
> Tom
>


--
Steve McKay <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Thomas Fitzsimmons
In reply to this post by David Herron @ Sun
David Herron wrote:

> Steve McKay☄ wrote:
>> Hi All,
>>
>> I've noticed that at least some of the tests using java.awt.Robot are
>> non-deterministic due to lags is the underlying window system. The
>> java.awt.Component.keyPressTest, for example, fails some of the time
>> (on linux, windows, linux+wine, ...). It looks like enabling
>> autoWaitForIdle (waits for the awt EventQueue to be empty before
>> adding new events to the queue), and setting autoDelay (pauses for an
>> arbitrary period of time) to some magic number of millis well above
>> zero (I use 100) significantly reduces failures. Would anyone object
>> to configuring the Robot with settings like this by default? If no,
>> should the config mechanism be updated to allow tweaking these
>> settings?
>>
>>  
>
> I don't know what the classpath implementation of Robot looks like, but
> I do know what Sun's Linux/Unix implementation looks like (having
> written the original version).

It uses the XTEST extension.

>
> Generally Robot has to request the OS or X11 to synthesize the event.  
> On Windows there's a direct API call, while on Unix/Linux there is a
> child process which ends up calling XTEST extension methods.  In both
> cases it means there is a nondeterministic delay due to the current
> process scheduling characteristics of the given system.  In other words
> it depends on an external entity, who Robot cannot coerce into
> performing the request within a bounded set of time.
>
> I think that means depending on Robot doing it's thing within a given
> period of time is an invalid test.

Interesting...

> Robot does not add events to EventQueue but it requests the OS to
> synthesize an OS-level event.

How has Sun implemented GUI testing?  When I was considering how to do GUI
testing in Mauve, I considered the EventQueue-posting approach, but decided on a
Robot-based approach instead.  I thought Robot tests would be more realistic,
testing things like window manager interactions and the native peers' event
processing code.  I knew Robot tests would be more fragile, but I assumed that
we could compensate for the fragility: e.g. fix timing problems by introducing
delays, as Steve has proposed.  Did Sun experiment with Robot tests, then
abandon them?  If Robot can't be counted on to do something within some time
delay, is it also useless in non-test applications?

I've always wondered how the TCK certified AWT and Swing functionality.  Does it
use EventQueue-posting?

Tom

Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Lillian Angel
In reply to this post by David Herron @ Sun
David Herron wrote:

> [resending because the mailing list thingy told me I had to use only
> plain text messages...]
>
> Hmmm.. here's the meat of the test
>
>  public void runTest(int code, char chr)
>  {
>    KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr,
> KeyEvent.KEY_LOCATION_STANDARD);
>    f.dispatchEvent(e);
>        f.setSize(200,200);
>    f.show();
>    r.mouseMove(60, 60);
>
>    r.keyPress(code);
>    r.keyRelease(code);
>    h.check(key, (int) chr);
>  }
>
> I don't understand this.  If you're going to create a Java event why
> use Robot, or vice versa...?

The test was done this way to determine if the right key was pressed.
See the inner class (myFrame) which has 1 function (keyDown) which sets
a variable (key) to the key that was pressed.

While it might be incorrect, I am unsure how else we can determine that
the correct key was pressed without dispatching an event.

Lillian

>
> It appears h.check is in gnu.testlet.TestHarness and that it simply
> does an immediate check with no waiting.  The dispatchEvent call is
> going to cause the listener to fire regardless of what's happening
> using Robot.
>
> This looks like an incorrect test, and what I'd recommend is:-
>
> a) ditch the two lines saying KeyEvent / dispatchEvent ... they are
> completely subverting the intent of the test
>
> b) insert some code so the runTest method waits for the listener to be
> triggered.  Such as a wait and notify type of semaphor.
>
> c) I don't know how the test guarantees runTest executes on the event
> dispatch thread.  Is the EDT as important to classpath as it is to
> Sun's Swing?
>
> - David Herron
>
>
> Steve McKay☄ wrote:
>> So would you recommend I ignore the test, delete it, add a comment, ...?
>>
>> --steve
>>
>> On 9/24/07, David Herron <[hidden email]> wrote:
>>  
>>> Steve McKay☄ wrote:
>>>    
>>>> Hi All,
>>>>
>>>> I've noticed that at least some of the tests using java.awt.Robot are
>>>> non-deterministic due to lags is the underlying window system. The
>>>> java.awt.Component.keyPressTest, for example, fails some of the time
>>>> (on linux, windows, linux+wine, ...). It looks like enabling
>>>> autoWaitForIdle (waits for the awt EventQueue to be empty before
>>>> adding new events to the queue), and setting autoDelay (pauses for an
>>>> arbitrary period of time) to some magic number of millis well above
>>>> zero (I use 100) significantly reduces failures. Would anyone object
>>>> to configuring the Robot with settings like this by default? If no,
>>>> should the config mechanism be updated to allow tweaking these
>>>> settings?
>>>>
>>>>
>>>>      
>>> I don't know what the classpath implementation of Robot looks like, but
>>> I do know what Sun's Linux/Unix implementation looks like (having
>>> written the original version).
>>>
>>> Generally Robot has to request the OS or X11 to synthesize the event.
>>> On Windows there's a direct API call, while on Unix/Linux there is a
>>> child process which ends up calling XTEST extension methods.  In both
>>> cases it means there is a nondeterministic delay due to the current
>>> process scheduling characteristics of the given system.  In other words
>>> it depends on an external entity, who Robot cannot coerce into
>>> performing the request within a bounded set of time.
>>>
>>> I think that means depending on Robot doing it's thing within a given
>>> period of time is an invalid test.
>>>
>>> Robot does not add events to EventQueue but it requests the OS to
>>> synthesize an OS-level event.
>>>
>>> - David Herron
>>>
>>>
>>>    
>>
>>
>>  
>

Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Steve McKay☄
In reply to this post by Thomas Fitzsimmons
Hi Thomas,

> If Robot can't be counted on to do something within some time
> delay, is it also useless in non-test applications?

David's suggestion to use multi-threaded paradigms for dealing with
the issue seems pretty good to me. For example, we can block until a
window is ready to process events with code like this (only half baked
at this time):

// before running any tests

    // show the window
    f.setSize(200, 200);
    f.show();

    waitForWindow(); // blocks until the frame has responded to a
keypress events

  ....

  class myFrame extends Frame {

    public boolean keyDown(Event e, int i) {
      synchronized(lock) { // lock is some explicit lock object
        key = new Integer(e.key);
        lock.notify();
      }
      return super.keyDown(e, i);
    }
  }

  /**
   * Blocks until the frame has responded to a keypress event.
   */
  private void waitForWindow() {
    synchronized(lock) {
      while (key == null) {
        r.keyPress(32); // send a key press event
        r.keyRelease(32); // don't press the key forever
        try {

          // wait for a notify from the frame, or timeout in case the
          // window missed the keypress event entirely
          lock.wait(100);
        }
        catch (InterruptedException ie) {
          // interrupted, if key is still null, we'll try again
        }
      }
    }
  }

The obvious downside is you have to deal with all the pitfalls of
multi-threaded code everytime you write the code.

--steve

> I've always wondered how the TCK certified AWT and Swing functionality.  Does it
> use EventQueue-posting?
>
> Tom
>
>


--
Steve McKay <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Steve McKay☄
In reply to this post by Lillian Angel
> While it might be incorrect, I am unsure how else we can determine that
> the correct key was pressed without dispatching an event.

As I know understand it, when the Robot's keyPress is called, it
should make a keypress happen on your desktop. So if the myFrame
window has focus, a keyPress event would be generated, otherwise, if
your editor is focused, it'll get the key press.

Does anyone know if Selenium uses Robot to do its poking and prodding?

--steve

> Lillian
>
> >
> > It appears h.check is in gnu.testlet.TestHarness and that it simply
> > does an immediate check with no waiting.  The dispatchEvent call is
> > going to cause the listener to fire regardless of what's happening
> > using Robot.
> >
> > This looks like an incorrect test, and what I'd recommend is:-
> >
> > a) ditch the two lines saying KeyEvent / dispatchEvent ... they are
> > completely subverting the intent of the test
> >
> > b) insert some code so the runTest method waits for the listener to be
> > triggered.  Such as a wait and notify type of semaphor.
> >
> > c) I don't know how the test guarantees runTest executes on the event
> > dispatch thread.  Is the EDT as important to classpath as it is to
> > Sun's Swing?
> >
> > - David Herron
> >
> >
> > Steve McKay☄ wrote:
> >> So would you recommend I ignore the test, delete it, add a comment, ...?
> >>
> >> --steve
> >>
> >> On 9/24/07, David Herron <[hidden email]> wrote:
> >>
> >>> Steve McKay☄ wrote:
> >>>
> >>>> Hi All,
> >>>>
> >>>> I've noticed that at least some of the tests using java.awt.Robot are
> >>>> non-deterministic due to lags is the underlying window system. The
> >>>> java.awt.Component.keyPressTest, for example, fails some of the time
> >>>> (on linux, windows, linux+wine, ...). It looks like enabling
> >>>> autoWaitForIdle (waits for the awt EventQueue to be empty before
> >>>> adding new events to the queue), and setting autoDelay (pauses for an
> >>>> arbitrary period of time) to some magic number of millis well above
> >>>> zero (I use 100) significantly reduces failures. Would anyone object
> >>>> to configuring the Robot with settings like this by default? If no,
> >>>> should the config mechanism be updated to allow tweaking these
> >>>> settings?
> >>>>
> >>>>
> >>>>
> >>> I don't know what the classpath implementation of Robot looks like, but
> >>> I do know what Sun's Linux/Unix implementation looks like (having
> >>> written the original version).
> >>>
> >>> Generally Robot has to request the OS or X11 to synthesize the event.
> >>> On Windows there's a direct API call, while on Unix/Linux there is a
> >>> child process which ends up calling XTEST extension methods.  In both
> >>> cases it means there is a nondeterministic delay due to the current
> >>> process scheduling characteristics of the given system.  In other words
> >>> it depends on an external entity, who Robot cannot coerce into
> >>> performing the request within a bounded set of time.
> >>>
> >>> I think that means depending on Robot doing it's thing within a given
> >>> period of time is an invalid test.
> >>>
> >>> Robot does not add events to EventQueue but it requests the OS to
> >>> synthesize an OS-level event.
> >>>
> >>> - David Herron
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >
>
>


--
Steve McKay <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

David Herron @ Sun
In reply to this post by Thomas Fitzsimmons
Thomas Fitzsimmons wrote:

> David Herron wrote:
>> Steve McKay☄ wrote:
>
>> Robot does not add events to EventQueue but it requests the OS to
>> synthesize an OS-level event.
>
> How has Sun implemented GUI testing?  When I was considering how to do
> GUI testing in Mauve, I considered the EventQueue-posting approach,
> but decided on a Robot-based approach instead.  I thought Robot tests
> would be more realistic, testing things like window manager
> interactions and the native peers' event processing code.  I knew
> Robot tests would be more fragile, but I assumed that we could
> compensate for the fragility: e.g. fix timing problems by introducing
> delays, as Steve has proposed.  Did Sun experiment with Robot tests,
> then abandon them?  If Robot can't be counted on to do something
> within some time delay, is it also useless in non-test applications?
>
> I've always wondered how the TCK certified AWT and Swing
> functionality.  Does it use EventQueue-posting?
>
> Tom
>


We make heavy use of Robot.

I haven't been writing tests for a long time so I don't know for sure
the techniques used by the team.  Sometimes I worry about whether they
are using sleep's and the like to synchronize validation that a desired
event has occurred.

If you click on something, move a mouse pointer, type a key.. there's a
listener fired in 99% of the cases you would be testing.  So I think
it'd be more reliable to synchronize the validation based on when the
listener fires.  Unfortunately that can turn into difficult coding to
ensure the test doesn't deadlock or otherwise go off into the weeds
because cross-thread notification of the listener firing didn't happen
in the right order.  I'm sure it's much simpler to code up a simple
sleep and 99% of the time the system will be fast enough responding to
post the listener callback within a couple milliseconds.

I believe that the TCK, where it has automated tests, uses Robot as
well.  I don't know that for sure since I've not been involved with
writing the TCK's, and that's a different team than mine.  The TCK also
has manual tests.

And the reason you give for using Robot is exactly the reasoning we
have.  Since we're testing AWT/Swing a we have to make sure to test AWT
and the peers.  Stuffing events in the EventQueue doesn't test the
native parts of AWT including the peers.  That was the rationale for
putting Robot into the platform in the first place (that and to support
writing freestanding demo apps), and it's true today.

- David Herron



P.S. there's a summary here:
    http://wiki.java.net/bin/view/Javapedia/TestingGUIApplications
of GUI testing methods.. it's been added to over several years and has
some good pointers.
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Lillian Angel
In reply to this post by Steve McKay☄
Steve McKay☄ wrote:
>> While it might be incorrect, I am unsure how else we can determine that
>> the correct key was pressed without dispatching an event.
>>    
>
> As I know understand it, when the Robot's keyPress is called, it
> should make a keypress happen on your desktop. So if the myFrame
> window has focus, a keyPress event would be generated, otherwise, if
> your editor is focused, it'll get the key press.
>  

That's right. If these two lines...
KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr,
KeyEvent.KEY_LOCATION_STANDARD);
f.dispatchEvent(e);

...are removed, myFrame.keyDown is not called and therefore, this.key is
not set. As it seems, the Robot function keyPress is not dispatching the
event to the Component.

Lillian

> Does anyone know if Selenium uses Robot to do its poking and prodding?
>
> --steve
>
>  
>> Lillian
>>
>>    
>>> It appears h.check is in gnu.testlet.TestHarness and that it simply
>>> does an immediate check with no waiting.  The dispatchEvent call is
>>> going to cause the listener to fire regardless of what's happening
>>> using Robot.
>>>
>>> This looks like an incorrect test, and what I'd recommend is:-
>>>
>>> a) ditch the two lines saying KeyEvent / dispatchEvent ... they are
>>> completely subverting the intent of the test
>>>
>>> b) insert some code so the runTest method waits for the listener to be
>>> triggered.  Such as a wait and notify type of semaphor.
>>>
>>> c) I don't know how the test guarantees runTest executes on the event
>>> dispatch thread.  Is the EDT as important to classpath as it is to
>>> Sun's Swing?
>>>
>>> - David Herron
>>>
>>>
>>> Steve McKay☄ wrote:
>>>      
>>>> So would you recommend I ignore the test, delete it, add a comment, ...?
>>>>
>>>> --steve
>>>>
>>>> On 9/24/07, David Herron <[hidden email]> wrote:
>>>>
>>>>        
>>>>> Steve McKay☄ wrote:
>>>>>
>>>>>          
>>>>>> Hi All,
>>>>>>
>>>>>> I've noticed that at least some of the tests using java.awt.Robot are
>>>>>> non-deterministic due to lags is the underlying window system. The
>>>>>> java.awt.Component.keyPressTest, for example, fails some of the time
>>>>>> (on linux, windows, linux+wine, ...). It looks like enabling
>>>>>> autoWaitForIdle (waits for the awt EventQueue to be empty before
>>>>>> adding new events to the queue), and setting autoDelay (pauses for an
>>>>>> arbitrary period of time) to some magic number of millis well above
>>>>>> zero (I use 100) significantly reduces failures. Would anyone object
>>>>>> to configuring the Robot with settings like this by default? If no,
>>>>>> should the config mechanism be updated to allow tweaking these
>>>>>> settings?
>>>>>>
>>>>>>
>>>>>>
>>>>>>            
>>>>> I don't know what the classpath implementation of Robot looks like, but
>>>>> I do know what Sun's Linux/Unix implementation looks like (having
>>>>> written the original version).
>>>>>
>>>>> Generally Robot has to request the OS or X11 to synthesize the event.
>>>>> On Windows there's a direct API call, while on Unix/Linux there is a
>>>>> child process which ends up calling XTEST extension methods.  In both
>>>>> cases it means there is a nondeterministic delay due to the current
>>>>> process scheduling characteristics of the given system.  In other words
>>>>> it depends on an external entity, who Robot cannot coerce into
>>>>> performing the request within a bounded set of time.
>>>>>
>>>>> I think that means depending on Robot doing it's thing within a given
>>>>> period of time is an invalid test.
>>>>>
>>>>> Robot does not add events to EventQueue but it requests the OS to
>>>>> synthesize an OS-level event.
>>>>>
>>>>> - David Herron
>>>>>
>>>>>
>>>>>
>>>>>          
>>>>
>>>>        
>>    
>
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

David Herron @ Sun
In reply to this post by Lillian Angel
Lillian Angel wrote:

> David Herron wrote:
>> [resending because the mailing list thingy told me I had to use only
>> plain text messages...]
>>
>> Hmmm.. here's the meat of the test
>>
>>  public void runTest(int code, char chr)
>>  {
>>    KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code,
>> chr, KeyEvent.KEY_LOCATION_STANDARD);
>>    f.dispatchEvent(e);
>>        f.setSize(200,200);
>>    f.show();
>>    r.mouseMove(60, 60);
>>
>>    r.keyPress(code);
>>    r.keyRelease(code);
>>    h.check(key, (int) chr);
>>  }
>>
>> I don't understand this.  If you're going to create a Java event why
>> use Robot, or vice versa...?
>
> The test was done this way to determine if the right key was pressed.
> See the inner class (myFrame) which has 1 function (keyDown) which
> sets a variable (key) to the key that was pressed.
>
> While it might be incorrect, I am unsure how else we can determine
> that the correct key was pressed without dispatching an event.
>
> Lillian
>

Maybe I misread the code in my brief look.

The side effect of using Robot to keyPress and keyRelease should be the
dispatch of two KeyEvent's.  That's why I said those two lines should
go, because the way I read it they're masking whether Robot successfully
delivered the KeyEvent's.


- David Herron

Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Steve McKay☄
In reply to this post by Lillian Angel
> That's right. If these two lines...
> KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr,
> KeyEvent.KEY_LOCATION_STANDARD);
> f.dispatchEvent(e);
>
> ...are removed, myFrame.keyDown is not called and therefore, this.key is
> not set. As it seems, the Robot function keyPress is not dispatching the
> event to the Component.

That's not my experience. I removed the explicit dispatch lines, then
added code to make sure the window is created and focused before the
test begins. After that, the test basically works. There appear to be
occasional deadlocks, so my thread management is wonky, but the basics
work as expected.

>
> Lillian
>
> > Does anyone know if Selenium uses Robot to do its poking and prodding?
> >
> > --steve
> >
> >
> >> Lillian
> >>
> >>
> >>> It appears h.check is in gnu.testlet.TestHarness and that it simply
> >>> does an immediate check with no waiting.  The dispatchEvent call is
> >>> going to cause the listener to fire regardless of what's happening
> >>> using Robot.
> >>>
> >>> This looks like an incorrect test, and what I'd recommend is:-
> >>>
> >>> a) ditch the two lines saying KeyEvent / dispatchEvent ... they are
> >>> completely subverting the intent of the test
> >>>
> >>> b) insert some code so the runTest method waits for the listener to be
> >>> triggered.  Such as a wait and notify type of semaphor.
> >>>
> >>> c) I don't know how the test guarantees runTest executes on the event
> >>> dispatch thread.  Is the EDT as important to classpath as it is to
> >>> Sun's Swing?
> >>>
> >>> - David Herron
> >>>
> >>>
> >>> Steve McKay☄ wrote:
> >>>
> >>>> So would you recommend I ignore the test, delete it, add a comment, ...?
> >>>>
> >>>> --steve
> >>>>
> >>>> On 9/24/07, David Herron <[hidden email]> wrote:
> >>>>
> >>>>
> >>>>> Steve McKay☄ wrote:
> >>>>>
> >>>>>
> >>>>>> Hi All,
> >>>>>>
> >>>>>> I've noticed that at least some of the tests using java.awt.Robot are
> >>>>>> non-deterministic due to lags is the underlying window system. The
> >>>>>> java.awt.Component.keyPressTest, for example, fails some of the time
> >>>>>> (on linux, windows, linux+wine, ...). It looks like enabling
> >>>>>> autoWaitForIdle (waits for the awt EventQueue to be empty before
> >>>>>> adding new events to the queue), and setting autoDelay (pauses for an
> >>>>>> arbitrary period of time) to some magic number of millis well above
> >>>>>> zero (I use 100) significantly reduces failures. Would anyone object
> >>>>>> to configuring the Robot with settings like this by default? If no,
> >>>>>> should the config mechanism be updated to allow tweaking these
> >>>>>> settings?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> I don't know what the classpath implementation of Robot looks like, but
> >>>>> I do know what Sun's Linux/Unix implementation looks like (having
> >>>>> written the original version).
> >>>>>
> >>>>> Generally Robot has to request the OS or X11 to synthesize the event.
> >>>>> On Windows there's a direct API call, while on Unix/Linux there is a
> >>>>> child process which ends up calling XTEST extension methods.  In both
> >>>>> cases it means there is a nondeterministic delay due to the current
> >>>>> process scheduling characteristics of the given system.  In other words
> >>>>> it depends on an external entity, who Robot cannot coerce into
> >>>>> performing the request within a bounded set of time.
> >>>>>
> >>>>> I think that means depending on Robot doing it's thing within a given
> >>>>> period of time is an invalid test.
> >>>>>
> >>>>> Robot does not add events to EventQueue but it requests the OS to
> >>>>> synthesize an OS-level event.
> >>>>>
> >>>>> - David Herron
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>
> >
> >
> >
>
>


--
Steve McKay <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Lillian Angel
Steve McKay☄ wrote:

>> That's right. If these two lines...
>> KeyEvent e = new KeyEvent(f, KeyEvent.KEY_PRESSED, 0, 0, code, chr,
>> KeyEvent.KEY_LOCATION_STANDARD);
>> f.dispatchEvent(e);
>>
>> ...are removed, myFrame.keyDown is not called and therefore, this.key is
>> not set. As it seems, the Robot function keyPress is not dispatching the
>> event to the Component.
>>    
>
> That's not my experience. I removed the explicit dispatch lines, then
> added code to make sure the window is created and focused before the
> test begins. After that, the test basically works. There appear to be
> occasional deadlocks, so my thread management is wonky, but the basics
> work as expected.
>  


Great! If it works, feel free to change the test. I am obviously doing
something incorrectly on my end.

Lillian


>  
>> Lillian
>>
>>    
>>> Does anyone know if Selenium uses Robot to do its poking and prodding?
>>>
>>> --steve
>>>
>>>
>>>      
>>>> Lillian
>>>>
>>>>
>>>>        
>>>>> It appears h.check is in gnu.testlet.TestHarness and that it simply
>>>>> does an immediate check with no waiting.  The dispatchEvent call is
>>>>> going to cause the listener to fire regardless of what's happening
>>>>> using Robot.
>>>>>
>>>>> This looks like an incorrect test, and what I'd recommend is:-
>>>>>
>>>>> a) ditch the two lines saying KeyEvent / dispatchEvent ... they are
>>>>> completely subverting the intent of the test
>>>>>
>>>>> b) insert some code so the runTest method waits for the listener to be
>>>>> triggered.  Such as a wait and notify type of semaphor.
>>>>>
>>>>> c) I don't know how the test guarantees runTest executes on the event
>>>>> dispatch thread.  Is the EDT as important to classpath as it is to
>>>>> Sun's Swing?
>>>>>
>>>>> - David Herron
>>>>>
>>>>>
>>>>> Steve McKay☄ wrote:
>>>>>
>>>>>          
>>>>>> So would you recommend I ignore the test, delete it, add a comment, ...?
>>>>>>
>>>>>> --steve
>>>>>>
>>>>>> On 9/24/07, David Herron <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>>            
>>>>>>> Steve McKay☄ wrote:
>>>>>>>
>>>>>>>
>>>>>>>              
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I've noticed that at least some of the tests using java.awt.Robot are
>>>>>>>> non-deterministic due to lags is the underlying window system. The
>>>>>>>> java.awt.Component.keyPressTest, for example, fails some of the time
>>>>>>>> (on linux, windows, linux+wine, ...). It looks like enabling
>>>>>>>> autoWaitForIdle (waits for the awt EventQueue to be empty before
>>>>>>>> adding new events to the queue), and setting autoDelay (pauses for an
>>>>>>>> arbitrary period of time) to some magic number of millis well above
>>>>>>>> zero (I use 100) significantly reduces failures. Would anyone object
>>>>>>>> to configuring the Robot with settings like this by default? If no,
>>>>>>>> should the config mechanism be updated to allow tweaking these
>>>>>>>> settings?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                
>>>>>>> I don't know what the classpath implementation of Robot looks like, but
>>>>>>> I do know what Sun's Linux/Unix implementation looks like (having
>>>>>>> written the original version).
>>>>>>>
>>>>>>> Generally Robot has to request the OS or X11 to synthesize the event.
>>>>>>> On Windows there's a direct API call, while on Unix/Linux there is a
>>>>>>> child process which ends up calling XTEST extension methods.  In both
>>>>>>> cases it means there is a nondeterministic delay due to the current
>>>>>>> process scheduling characteristics of the given system.  In other words
>>>>>>> it depends on an external entity, who Robot cannot coerce into
>>>>>>> performing the request within a bounded set of time.
>>>>>>>
>>>>>>> I think that means depending on Robot doing it's thing within a given
>>>>>>> period of time is an invalid test.
>>>>>>>
>>>>>>> Robot does not add events to EventQueue but it requests the OS to
>>>>>>> synthesize an OS-level event.
>>>>>>>
>>>>>>> - David Herron
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>              
>>>>>>            
>>>
>>>      
>>    
>
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Thomas Fitzsimmons
In reply to this post by Steve McKay☄
Steve McKay☄ wrote:
> Hi Thomas,
>
>> If Robot can't be counted on to do something within some time
>> delay, is it also useless in non-test applications?
>
> David's suggestion to use multi-threaded paradigms for dealing with
> the issue seems pretty good to me.

Yes, I shouldn't have said useless.. maybe "hard to use" :-)  In Mauve we traded
timing-independent reproducibility for test case simplicity.  I did try writing
synchronized Robot-using tests but it was a) difficult, and b) invasive, in that
it required overriding and modifying the behaviour of methods that were being
tested.

> For example, we can block until a
> window is ready to process events with code like this (only half baked
> at this time):

Even after you've ensured that the frame has responded, subsequent checks will
still need to be synchronized, since Robot calls are not synchronous.  Or is
this the specific problem that was causing test failures for you before -- that
frames take a long time to be ready to receive events on other runtimes (e.g.
Wine)?  If that's the main problem then maybe a hybrid approach of synchronizing
frame readiness, then assuming small delays for Robot method calls, is a better
approach.  Otherwise the proposal is to rewrite every Robot-using test to be
perfectly synchronous (fine by me, but a big job).

Tom
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Steve McKay☄
Thomas,

> Even after you've ensured that the frame has responded, subsequent checks will
> still need to be synchronized, since Robot calls are not synchronous.

That was just a code snippet to demonstrate how to solve one part of
the problem. The code that does the actual testing is synchronized on
the lock as well. But like I said, there were liveness issues. I'll
try to work those out. Still, I can't honestly say that I'd want to go
through the same amount of effort for every test involving Robot, so
this change is sorta speculative rather than practical.

--
Steve McKay <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Steve McKay☄
Here's a patch for the test. Everything seems to be working well. I
haven't looked through the other tests yet to see how this might be
shared.

--steve

On 9/25/07, Steve McKay☄ <[hidden email]> wrote:

> Thomas,
>
> > Even after you've ensured that the frame has responded, subsequent checks will
> > still need to be synchronized, since Robot calls are not synchronous.
>
> That was just a code snippet to demonstrate how to solve one part of
> the problem. The code that does the actual testing is synchronized on
> the lock as well. But like I said, there were liveness issues. I'll
> try to work those out. Still, I can't honestly say that I'd want to go
> through the same amount of effort for every test involving Robot, so
> this change is sorta speculative rather than practical.
>
> --
> Steve McKay <[hidden email]>
>

--
Steve McKay <[hidden email]>

KeyPressTest.patch (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Tweaking default java.awt.Robot settings

Lillian Angel
Looks good, I tested it out and committed it for you:

2007-10-03  Steve McKay  <[hidden email]>

        * gnu/testlet/java/awt/Component/keyPressTest.java: Fixed test so
        event is not created and dispatched. Completely waits for the
frame to
        receive and respond keypress events.

Lillian



Steve McKay☄ wrote:

> Here's a patch for the test. Everything seems to be working well. I
> haven't looked through the other tests yet to see how this might be
> shared.
>  
> --steve
>
> On 9/25/07, Steve McKay☄ <[hidden email]> wrote:
>  
>> Thomas,
>>
>>    
>>> Even after you've ensured that the frame has responded, subsequent checks will
>>> still need to be synchronized, since Robot calls are not synchronous.
>>>      
>> That was just a code snippet to demonstrate how to solve one part of
>> the problem. The code that does the actual testing is synchronized on
>> the lock as well. But like I said, there were liveness issues. I'll
>> try to work those out. Still, I can't honestly say that I'd want to go
>> through the same amount of effort for every test involving Robot, so
>> this change is sorta speculative rather than practical.
>>
>> --
>> Steve McKay <[hidden email]>
>>
>>    
>
>
>