Death to xpath (and css) selectors in automated tests

Posted on August 9, 2011

I think it’s time for xpath and css selectors to identify elements in automated tests to die. I am sick of seeing them in automated tests, and I don’t know of any good reasons why we should use them.

Let me demonstrate.

Randomly looking through some example automated tests for Etsy.com on github, there’s a couple of xpath selectors included.

One is on this page: http://www.etsy.com/buy?ref=so_buy

@browser.find_element(:xpath, "//ul[@id='user-nav']/li/a[text()='Buy']").click()

The other is on this page: http://www.etsy.com/treasury

def element = findElement(By.xpath("//div[@class='item-treasury-info-box']/h3/a"))

These are by no means complicated xpath expressions, but I just don’t like using them.

In Watir-WebDriver I’d do the first example as:

@browser.link(:text => 'Buy').click
# or
@browser.link(:title => 'Buy on Etsy').click

or Selenium-WebDriver (ruby) as:

driver.find_element(:link_text, 'Buy').click
# Selenium-WebDriver doesn't support identifying elements by #title

In Watir-WebDriver I’d do the second example as:

@browser.div(:class => 'item-treasury-info-box').link.click

or Selenium-WebDriver (ruby) as:

b.find_element(:class, 'item-treasury-info-box').find_element(:partial_link_text, '').click

No xpath. Not needed. Why bother?

I wrote the Etsy.com examples recently in Watir-WebDriver without using a single xpath selector, but the same example in Selenium-WebDriver uses xpath on every page. Why?

Selenium-WebDriver (the official ruby Selenium bindings) has limitations that means it’s easy to revert to using xpath and css selectors:

It’s not possible to use more than one identifier to identify a single element
You can use :index as a method to find elements (but you can find all elements and use the array)
There is no such thing as an element type in Selenium-WebDriver: find_element returns all element types
The identifier tags you can use is limited, for example, you can’t identify elements using :title

In Watir-WebDriver, there are no such limitations. It is a lot easier to nest multiple elements to clearly articulate what you’re looking for.

For example, this

@browser.div(:class => 'item-treasury-info-box').link.click

could be written as:

@browser.div(:class => 'item-treasury-info-box').h3.link.click

which shows the full DOM chain if you like to articulate it this way. I like the fact that each element type is explicit, I would want my test to fail it a span was changed to a div or vice-versa. It would mean I would go and manually check the layout to ensure it still looks okay. An element is an element type for a reason.

The absence of :index methods, and the ability to only use one identifier means that this chaining is not possible in Selenium-WebDriver, without using an xpath or css selector.

Summary

I struggle to understand why anyone would choose to use Selenium-WebDriver over Watir-WebDriver, considering its limitations. But people do, and its limitations mean writing complex xpath or css selectors in your tests.

I avoid writing xpath and css selectors; I would much rather use the Watir-WebDriver API to clearly express what I am expecting.

25 thoughts on “Death to xpath (and css) selectors in automated tests”

Lorenzo Urbini on August 9, 2011 at 6:24 pm said:
Hi ALister, thanks to raise this… I’m just wondering how can you tackle the following case (the Log in button click..) without using xpath or dom navigation which I reckon bad exactly the same:
```
<form name="frmLogin" id="frmLogin" action="/site/include/ProcessLogin.php" method="post">
            <table class="tableSpacing">
            	<tbody>
                <tr>
                	<td><input type="text" name="fldUser" class="login" onfocus="clearFormField(this)" value="Username"></td>
                </tr>
                <tr>
                	<td><input type="password" name="fldPwd" class="login" onfocus="clearFormField(this)" value="Password"></td>
                    <td><input type="image" src="../site/images/btn-chevron-sm.gif" style="border:0px;" alt="Log in button"></td>
                </tr>
            </tbody>
</table>
</form>
```
Reply ↓
Alister Scott on August 9, 2011 at 8:54 pm said:
In Watir-WebDriver it is super-easy:
```
browser.button(:alt => "Log in button").click
```
In Selenium-WebDriver, not so easy without xpath (hence why I won’t use it)
```
d.find_element(:class, "tableSpacing").find_elements(:tag_name, "input")[2].click
```
P.S -> to include html in a comment, wrap it in: elements
Reply ↓
Lorenzo Urbini on August 9, 2011 at 9:36 pm said:

Thanks Alister i neither knew about an alt selector….
Anyway, I still retain that using
d.button(:xpath => “//form[@id='frmLogin']/table/tbody/tr[2]/td[2]/input”).click
or
d.find_element(:class, “tableSpacing”).find_elements(:tag_name, “input”)[2].click
is very similar, both of them can expose the test script to be broken in case of code changes unless there are some performance reasons whether to use one or the other.

Reply ↓
Lorenzo Urbini on August 22, 2011 at 11:34 pm said:

Hi Alister I’ve just found another one interesting…
How would you manage to select the next page (cur+1) from the google search pagination without using any xpath (included the useful ‘following’ keyword) ?

Reply ↓
- Alister Scott on August 23, 2011 at 9:20 am said:
  
  Not sure what example you’re talking about. Can you provide a URL?
  
  Reply ↓
Lorenzo Urbini on August 23, 2011 at 6:26 pm said:

Sure: http://www.google.co.uk/search?q=tottenham+court+road&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-GB:official&client=firefox-a

I’d like to click the next pagination number, the one following that marked with the class ‘cur’, just below “goooooooogle”.

Reply ↓
- Alister Scott on August 23, 2011 at 10:26 pm said:
  
  browser.td(:class => ‘cur’).parent.link(:class => ‘fl’).click
  
  Reply ↓
- Alister Scott on August 23, 2011 at 10:38 pm said:
  
  PS -> Wouldn’t you just click ‘Next’?
  browser.link(:text => ‘Next’).click
  
  Or am I missing something?
  
  Reply ↓
- Alister Scott on August 23, 2011 at 10:44 pm said:
  
  Kinda yucky, but this works:
  b.link(:text => ((b.td(:class => 'cur').text.to_i+1).to_s)).click
  
  Reply ↓
Nat Ritmeyer on August 28, 2011 at 5:33 am said:

I guess it’s a matter of being specific and concise at the same time. The first comparison you make is the following:

@browser.find_element(:xpath, “//ul[@id='user-nav']/li/a[text()='Buy']“).click()
vs.
@browser.link(:text => ‘Buy’).click

I can see what you’re saying – yours would work fine. But, yours would click on any link on the page whose text property is ‘Buy’. The example with the xpath is more specific – it points to a link whose text is ‘Buy’, but it restricts it to a link in a list of bullet points, and not just any bullet point list, but one with a specific ID. I guess in this example, the xpath version is more resilient to change than the alternative that you offer.

That doesn’t mean that xpath is ‘better’ because you could do the same thing in water-webdriver; it would just be more long winded.

In my time in web automation, I’ve found xpath to be the most reliable way of identifying objects that also requires the least maintenance. Anecdotal, I know, but it’s worked for me.

I agree with you on the selenium-webdriver vs watir-webdriver, the fits my way of thinking better than the former!

Reply ↓
- Alister Scott on August 30, 2011 at 9:30 pm said:
  
  It’s a decision we need to make when writing a test on how specific to identify an element, whether it’s off the page itself, or nested under another element.
  
  I personally found xpath the least reliable way of identifying objects, as I’ve seen many examples of crazily stupid ones such as this one posted on stackoverflow a few days ago:
  
  ie.element_by_xpath("/html/body/div[3]/div/div/div/section/div/div/div/form/table/tbody/tr/td[3]/input").click
  
  Reply ↓
  - Nat Ritmeyer on August 31, 2011 at 1:55 am said:
    
    Haha! I’ve seen xpaths like that before… they come from people copying-and-pasting the xpath autogenerated by the ‘View XPath’ firefox plugin. In concept it’s similar to people mindlessly copying tests from SeleniumIDE. Stuff like this is a great way to filter people out at the interview process :)
    
    It would be unfair to curse xpath for the horrible example you give. If someone writes their xpaths like that, chances are they write other stuff badly. From what I’ve seen, there’s a strong correlation between bad xpaths and bad tests (test design, test code, etc). A test that uses an xpath like that won’t stay green for long; so if you see that sort of xpath, there’s a good chance that the test is useless.
    
    Again, not xpath’s fault.
    
    Reply ↓
- Chuck van der Linden on January 28, 2012 at 5:26 am said:
  
  You can do the same thing with watir if you want/need to be more specific.
  
  b.ul(:id => ‘user-nav’).link(:text => ‘Buy”).click
  
  and I’d argue it’s easier to read
  
  Reply ↓
Hamza on September 1, 2011 at 5:58 am said:

Spot on Nat….@alister try this…http://www.w3schools.com/xpath/

Reply ↓
- Alister Scott on September 4, 2011 at 9:21 pm said:
  
  Please don’t be condescending
  
  Reply ↓
Aaron on October 29, 2011 at 11:11 am said:

Sure, it’d be fantastic not to have to use xpaths and CSS selectors. The trouble is that sometimes *you have to*. Sometimes there’s no better or easier way to interact with an element. I try not to use them myself, but will “fall back” to using them if I have to.

Reply ↓
mingzhu (@mingzhu) on February 16, 2012 at 11:44 am said:

Hi Alister,

Thanks for the excellent article. It’s indeed very neat. However, I would question the flexibility.

Imagine we have
…..
and there a lot of inputs and links and text under the div.

If later the code implements a new interface, so we have “user-nav” changed into “menu-nav”. As in automated testing, we then need to go through all of our automation code and find any thing uses menu-nav and change that into user-nav. This would be so troublesome when it’s in transition phase, where some page is using new interface (“menu-nav”) and old interface (“user-nav”).

Cheers,
Ming

Reply ↓
- mingzhu (@mingzhu) on February 16, 2012 at 11:45 am said:
  
  Need some special tag to display the code:
  
  ... </div
  
  Reply ↓
- mingzhu (@mingzhu) on February 16, 2012 at 11:47 am said:
  
  Imagine we have
  a div tag with id=”user-nav”, which contains
  a lot of inputs and links and text under the div.
  
  Reply ↓
  - Alister Scott on February 22, 2012 at 9:45 am said:
    
    Depends whether you care that they’re in the div or not.
    
    Reply ↓
- chuckvdl on February 19, 2012 at 5:39 pm said:
  
  This sort of change would require updating scripts in any case where that div was part of the ‘path’ used to identify objects. It’s more often the case the way I’ve seen xpath most often used, but it would also happen if you are defining a more explicit path using watir-webdriver.
  
  Using a abstraction layer pattern such as Page Objects can simplify this a lot by creating a central place where you identify objects and give them a more friendly name that is then used elsewhere in your scripts. Then you have one place to make such changes when devs rename things, which makes updating scripts a lot easier.
  
  Reply ↓
Ryan C. on February 17, 2012 at 9:42 am said:

Alister,

Thanks for this post – it really adds clarity as to why a person would select Watir-WebDriver over Ruby’s Selenium WebDriver.

I’m running into a problem, which I’ve posted here at StackOverflow:
http://stackoverflow.com/questions/9319833/recursively-loop-through-frames-to-find-an-element-in-watir-webdriver-works-in

It seems like it’s kind of in the same ‘vein’ as some of the issues you highlighted here, albeit loosely. Do you have any insights?

Reply ↓
- Alister Scott on February 17, 2012 at 3:13 pm said:
  
  I posted a response at Stack Overflow: http://stackoverflow.com/questions/9319833/recursively-loop-through-frames-to-find-an-element-in-watir-webdriver-works-in
  
  Reply ↓
Olya on February 18, 2012 at 12:36 am said:

Hello!
Please give me advice, I’m a beginner in Selenium WebDriver and I used Selenium IDE before. And I have a lot of XPath selectors in my tests, but I have no idea of how to change some of them not using XPath, for example I have a table with checkboxes in the first column and IDs in the 3rd column and I need to click a checkbox in a row with ‘required_ID’ in the 3rd cell and this is how I do this with XPath:

//tbody[2]/tr[td[3]/a=’required_ID’]/td/center/input

How should I make this condition without XPath, just give me a hint. Thanks.

Reply ↓
- Jonathan Cole on February 21, 2012 at 9:36 am said:
  
  Stick with XPath when you need to travel up the DOM. If you are concerned with speed, make sure you include this step after starting selenium:
  
  selenium.useXpathLibrary(“javascript-xpath”);
  
  Reply ↓