My current job involves running a suite of automated acceptance and accessibility tests automatically across four browsers (IE8, IE9, Firefox & Chrome) on every check in. These are run automatically using a ThoughtWorks Go pipeline which is run on a freshly deployed integrated QA environment immediately after all unit, integration and JavaScript automated tests pass.
Whilst I set up five agents to run these tests in parallel across the different browsers, the build was as slow as its slowest member (much like a buffalo heard) which happened to be IE8 (followed closely by IE9).
Initially the execution times looked something like this:
- Chrome ~50 secs
- Firefox ~1m 10 secs
- IE9 ~4 m 30 secs
- IE8 ~5 mins
I was wondering why on earth it was taking so long, when Simon Stewart pointed out how screenshots work in the IE Driver. I set up the tests to take a screenshot at the end of each scenario, which meant each browser was taking about 18 screenshots per test run.
I didn’t know but the IE Driver maximizes then restores the IE window every time it takes a screenshot, and it also parses the entire DOM to take a screenshot. This is why it was taking so long to execute the tests.
I removed the screenshots from the IE runs and was able to reduce both IE8 and IE9 to just over 2 minutes execution time. Not the best, after all it’s over twice as slow as Chrome, but better than 5 minutes previously!
In the future, I’ll avoid taking any screenshots using IE Driver wherever possible.
“Parses the entire DOM to take a screenshot”? Jeepers. Yet another good reason to just screenshot the whole desktop rather than relying on browser-specific stuff. I know it doesn’t get the whole page, but it does get all kinds of other goings-on. Once I had a test that was latching onto a modal dialog that had popped up from the antivirus software installed on the VM. If we’d just captured the page within the browser window we wouldn’t have caught it.
How does one take a screenshot of the entire desktop with WebDriver, especially over Grid? Or is there some other VM magic going on there?
Just using WebDriver take screenshot method call
WebDriver has three screenshot methods: get_screenshot_as_file, get_screenshot_as_base64, and save_screenshot, all of which say they get “the screenshot of the current window.”
Unless this is some WebDriver functionality not available in the python driver?
If you’re using Python, you could probably do something like this http://stackoverflow.com/questions/2846947/get-screenshot-on-windows-with-python
Problem is I’m running the python test runners on a separate server, which connects to the Grid, so I’m not actually using python directly on the same machine as the browser. I only have access to things that WebDriver provides.