Selenium-WebDriver makes direct calls to the browser using each browser’s native support for automation. How these direct calls are made, and the features they support depends on the browser you are using. Information on each ‘browser driver’ is provided later in this chapter.

The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, and can follow links and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history.

Most testers, I hope, know the feeling of having to go through tedious, repetitive work, setting up data for a test, going through the login flow of an application to get to the feature you want to test etc. For actions like that, you can very well use automation. In fact, you can quite often use the most simple form of automation, record/playback without having to adjust your scripts for maintainability or re-usability.

I was tasked with testing an ancient Oracle process that had long been in production but had never been officially tested. I created all the necessary test cases, data, SQL, etc. Because this process would need to be tested annually for the remaining life of the system, the director stepped in and asked the usual "can we automate this?" question. I thought it was a bit of wheel-reinvention at that point, but demured and provided the necessary information on performing the test to the automation team. The automator then took 5 weeks to create a massive script, claiming that this would alleviate the pain of testing the process annually at system rollover. I found this claim dubious, so decided to see for myself which was the better (e.g., faster, more accurate) way. I mean, after creating a script with over a thousand lines of code to test a process that can be completed in 5 minutes, I expected this baby to not only perform the task at hand, but to give my car an oil change and walk my dog to boot. So like John Henry with his sledgehammer, I went head-to-head with the machine. I kicked off the script on a lab box and went about manually testing the process on my own PC. Guess what? I completed my manual testing (and found 7 true bugs) before the script was halfway finished.

No programming effort is involved to automate tests and we are proud to take the stance that tests should not be recorded, so your acceptance tests can run continuously and react intelligently.

GUIdancer is different from other GUI test-tools because automated tests can be written before the application under test (AUT) is ready. This means that GUIdancer is not a tool which tests an application by recording user actions and replaying them (capture-replay). Tests can be created from the requirements without access to the AUT, and involve no programming, script or code. GUIdancer tests can be created, run and maintained without support from automation experts.

Glad you enjoyed the talk! Yes, work with GHC is very much in progress right now. For a sneak peak, you might look at the links here:
http://goto.ucsd.edu/~rjhala/liquid/
(hover mouse over identifiers to see inferred types)
http://goto.ucsd.edu/~rjhala/liquid/ghc/lambdaEval.hs.html http://goto.ucsd.edu/~rjhala/liquid/ghc/mapreduce.hs.html http://goto.ucsd.edu/~rjhala/liquid/ghc/meas5.hs.html

With the release of cabal-install 0.14, you can use Cabal to build and run your benchmarks. No more Makefiles that try to replicate things Cabal already does.

When I do this myself, I often set up a multi-module project (although that's not strictly nessecarily) and encapsulate all the integration testing into that one module. I then enable the module with profiles (or not) so that it's not blocking the immediate "yeah, I know I broke it" builds.

"The Failsafe Plugin is designed to run integration tests while the Surefire Plugins is designed to run unit tests." - http://maven.apache.org/plugins/maven-failsafe-plugin/

See below for the Failsafe section.

JSpy is a tool that displays the component properties of any java Swing application in the simplest way, that is, by a simple mouse hover. To automate Swing applications, a user generally has to venture into the application code to retrive the application properties. This sometimes can be very time consuming and also frustrating. Using JSpy the user can overcome this problem, because all a user needs to do to retrive the application properties is to perform a simple mouse hover.

I'm comfortable writing tests in QuickCheck and HUnit and bundling them as
optional executables with cabal, but I understand there's a better
way. Specifically, I'm looking at the test-framework package and cabal's
(newish) test-suite sections. Are these two used together or each to the
exclusion of the other? Is there something else I should be using?

In the case of functional tests, we do not want to test individual GUI classes, we want to access the application as a whole.

Like with any automation project, test automation also involves programming albeit to a lesser extent. You find major success in your test automation project if the project is considered as a software development project and plan it accordingly. Depending on the size of an organization and project structure, different teams might be involved in creating automated tests. We do not assume that a user of Marathon is knowledgeable in programming and thus try to provide as much details as required for using Marathon effectively.

Both (marathon and sikuli) offer python api scripting, so maybe there
is a way, to combine them.

Daden recently demonstrated a 2D web-based scene layout tool as part of their OPAL dynamic scene management system which is the next generation of their PIVOTE solution. Sikuli is rather more prosaic in providing support for staff laying out classes and to students exploring alternative layouts in PowerPoint as a familiar reference point.

In fact, even without OCR, we still can extract text from the screen using Sikuli.

How?

Use the clipboard!

If you are looking at an end to end test solution with OCR functionality for test automation across any OS any device . Try eggPlant from TestPlant.

With traditional screenscrapers, you cannot monitor what the "final" value is for something. For example, if a value is only known after a series of constraints have been solved, and those constrains continuously change, how do you monitor that value and then "finalize" it when the user clicks, say, "Done" or "Next"?

Automated UI tests are the real integration tests. They test the entire system in the way it's actually used when it's live. That makes them the most meaningful tests. However, they also tend to be the most brittle, and the slowest to execute.[...]For more important tests, we found that a single person could spin up several VMs and use scripts to get to the point of a real test. It let them focus on the important bits and not try and follow a 24-step test case.

Tests can be driven in three ways (or any combination). Java provides a
Robot class (as of release 1.3) which can generate native OS events. These get
processed by the OS, passed on to and processed by the VM, and get converted to
Java’s AWT events handled on its event dispatch thread. This level of testing is
closest to real user behavior, since the mouse pointer actually moves and keys are
actually pressed (well, maybe your keyboard doesn’t actually dance). That of
course means you can’t do anything else with the system while it’s busy testing,
and other programs can potentially interfere with input

Best Practice: As a general guideline, the best choice is to use paste() for readable text and type() for action keys like TAB, ENTER, ESC, .... Use one type() for each key or key combination and be aware, that in some cases a short wait() after a type() might be necessary to give the target application some time to react and be prepared for the next Sikuli action.

The early releases of Sikuli were cool, but were missing what I considered a critical feature, text recognition. The 0.10.2 release in August 2010 showed some promise and we decided that we would not renew the Eggplant license at the end of 2010, but instead commit to reworking the automated tests to use Sikuli.

Using Sikuli correctly is means that you are not following a "record and playback" model. Rather, you have to approach developing test automation with Sikuli - as you need to with all tools - as a software development task. [...] From my experience the biggest challenge is image managment. I used the file system and configparser for the first iteration of the test automation. Using configparser worked however it was difficult to implement. In the future I plan on using blobs. Sikuli does not support direct extraction of images from a DB (yet) although I do have a work around.

If your target application has custom components, I would definitely recommend Marathon to automate your tests.

I was given the task of automating an application with several extremely complicated custom components, written in-house from the ground up. I went through a review process that lasted two months, in which I made the decision on which test tool to use, from a list of close to 30 test tools that were available, both commercial and FOSS.

There are many tools for automating GUI testing. All of them are more or less fragile and require pretty much constant maintenance in the face of changing layout. There is benefit to be gained from using them, but it's a long term benefit.

If you're doing GUI apps using Java/Swing, then you can try to use Cucumber and Swinger for writing functional acceptance tests in plain english for Swing GUI applications. Swinger uses Netbeans' Jemmy library under the hood to drive the app.