Zero Soup

 

Questions for the Campers at BarCampMumbai2

Tuesday, October 16, 2007

A couple of days ago, I was at BarCampMumbai2, and I spent some time chatting with other campers on testing PHP-Mysql based web applications.

What I did share with the campers was some of our experiences at BookEazy. More important than that however, I actually approached them with 3 issues we have not been able to solve here.

I now throw these rather well known issues to the world of developers out there in the hope that somebody will share some testing Gyan with us.

Question 1
How do you effectively test a PHP function that looks something like:

function fooBar() {
// some code here
...
...
mysql_query( [some complicated dynamically generated query string here] );
...
// Some minor error checking and result processing here
}

What I'm looking for is a solution that helps reduce the complexity of doing super huge database data setups and tear down, because we know from experience that running the big mama setup route becomes unmaintainable fast.

Question 2
How do you test the mobile version of your application/website in an automated and script-able fashion. I know of no tool that implements a Selenium/Watir/WebDriver like testing framework for Mobile platforms

Question 3
How do you test mobile application delivery and installation across multiple mobile platforms in an automated fashion.

I invite anyone with suggestions, resources to share, ideas or comments to feel free to leave your thoughts in the comments section of this post.

Labels: BarCampMumbai2, PHP, Testing

 

GTAC 2007, Selenium, and the UI as objects in tests.

Monday, October 01, 2007

I was going through all the clips posted on YouTube and Google Video from the recently concluded GTAC 2007. For the uninitiated GTAC is the "ask to be invited" only Google Test Automation Conference.

The keynote by Patrick Copeland was good if a little bland introduction to the rest of the clips posted. The lightning talks were good, informative and concise. Just the way I like my information.

My pick of some the best lightning talks:
1) Code Smells
2) Selenium vs WebDriver
3) Managing thousands of test at Mozilla


Some of the 1 hour sessions posted were not really applicable to the average project team that do much smaller scale stuff than what Google attempts. Anyways, the ones I thought were pretty decent are listed below:

• Patrick Copeland - Keynote
• Simon Stewart - Web Driver for Java
• Adam Porter & Atif Memon - Skoll DCQAS
• Apple Chow & Santiago Etchebehere - Building an Automated Framework Around Selenium
  

I think the best bits of information to be obtained from all the videos could be summed up in a few clear and rather innovative ideas.

Implementing the UI as objects in test code.
Now here's an idea I really like. Google's Apple Chow mentioned this idea in one of the sessions. The basic idea (at least the way I look at it) is that you represent the UI and its components as objects in your test scripts. This way you can keep implementation specifics of test for a particular page/UI component isolated from the main test script. The best pro of this approach is that whenever your UI implementation changes, and lets face it, this happens a lot in web applications, you don't have to go and fix a whole bunch of tests. You just adjust the object that modeled that UI in your test environment.

Interestingly, Simon Stewart mentioned that WebDriver creates a sort of Page object for every HTTP request. That kind of follows this methodology in my opinion. Good stuff. Great ideas, and we are going to use these :-)


"Test Training" and "Happy Path" Tests
Patrick Copeland mentioned the concept of "Happy Path" tests. Nice term for something you really want to avoid. Basic idea is that there is a very real tendency for developers to write tests that behave the way you a want a user to behave. Its a bit like testing for what a user should do instead of what a user could do. Excellent point, and good to keep in mind as a developer.
There's also the issue of "Test Training" which basically occurs when updating and modifying tests over time. Developers tend to subconsciously (or consciously??) modify tests so that they pass rather than fail. This ones hard to detect and weed out. But also a good point to remember.


Using Unit Tests for Negative Testing
Matt Heusser & Sean McMillan talked about the concept of using Unit Test to focus on negative testing since functional tests basically focus on the positive paths anyway. I like this idea a whole lot. If you also write a few unit tests to cover the positive paths, then this approach will also lend itself to the idea of using unit tests to form functional tests, and functional tests to form stress/load tests. (Frank Cohen over at PushToTest.com has been pushing that idea for a long time now.)

I'm going to make special mention of Skoll DCQAS. This talk blew me away. While not the sort thing small teams can use easily, the power of this "soon to be out there tool" is freaky. I'm not going into details, you need to watch this one. Atif mentioned something about the shortest path through an application is often the most used in the real world. I'm not entirely convinced thats true, but if it is, heck, this tools going to be worth even more!

All in all, there's a lot to be learned from these clips. What probably got me most excited was however, the fact that Google is actually actively developing Selenium. Not only are they finding and fixing a whole can of worms in Selenium, but they are trying to release a lot of back to the open source repository. If they also release their Firefox extension to Firebug, for Selenium testing, life is going to get a lot better for us Selenium followers pretty soon.

Labels: GTAC, Selenium, Testing