TDD – What, Why, Problems with TDD and How to overcome its limitation

August 1, 2011 2 comments

TDD (Test Driven Development)

How to implement TDD?

TDD is an agile design technique and is associated with concepts of extreme programming. In this approach tests or precisely the unit tests are written first before actually the team starts implementing the requirement.

In order to implement TDD, the developers begin with writing the tests for the particular functionality. These tests would fail to begin with. The developers then begin implementing the functionality, the code is written to pass the tests created. When the initial tests written passes, the developers think of other tests for the requirement and create more tests for other scenarios as well as boundary case scenarios. This is the time to include re factoring of the code to remove the duplicate code, make the code clear and more robust.

The process of writing the tests first and then writing code + re factoring is repeated until no other tests can be thought of. TDD is a way to ensure that the specifications are met and also validating the requirements.

My experience with TDD through agile?

In the company that I had worked for, we were one of the few projects to start with Agile and later on seeing the way we implemented and the results we got the other teams too followed us.

First, let me begin with the reasons why we started with TDD. We had PHP server side code very tightly coupled, and we were adding functionality very frequently. As a result the code was very fragile and the new functionality would often break some other functionality. So the team had a tough time in testing.

The unit test became all the more important to us. So, the developers made it a point to write the unit tests before actually implementing functionality. For Php we had PHPUnit, as we have Junit for Java. TDD assumes that the xUnit frameworks are available for the technology used. The tests were written and then the code, more tests were written followed by coding until all the tests were passed.

Then came the issue of running these tests each time the build was created. The need to integrate this with the continuous integration environment was evident. We had Cruise control as the continuous integration tool to integrate the builds and were using ant build script to create the build and run the tests. A target for the Phpunit tests was created and made to run with every build and proper notification was sent to the stakeholders in case of failures.

The problems that we faced with TDD and how we overcame it

  • TDD is good with validating the functionality at the smaller levels but not so good at thinking through the bigger issues such as overall design and how people would be using the system. To overcome this issue we backed our unit tests with the GUI automation regression suite in RiaTest(for flex) and Selenium RC(for Java based app) and also the api tests that we tested using Jmeter test plan.
  • The TDD approach might bring you to think in only one direction. Pre-written test cases may prohibit you to see those alternatives lying outside those defined in the test cases. To overcome this we started with another agile extreme programming approach of pair programming. This keeps you in focus and and the discussions results in a broader view.
  • TDD also may end up emphasizing more on composing tests than the actual code. At the initial phase developing a testing framework that test the software would be like writing the code itself. So to setup some time needs to be taken out from the project but its all for good.
  •  A lot of initial conceptions about the software will change or vanish as you develop. Hence the tests may become useless. So proper discussion with the Product owner before designing or implementing was done to ensure no change thereon in the requirement.

Agile is all about the art of learning and adapting.

Advertisements
Categories: Agile

Know your Flex Automation coverage

July 16, 2011 2 comments

Often automation engineers are faced with a question, “What is the coverage of the automation scripts?”. Having an answer to this is very important as it helps the testers in identifying areas which remains not tested or less tested and then model the test scripts according to that. It serves as an important tool to learn and adapt the project automation strategy for better.

In an attempt to calculate the coverage of our automation GUI script we integrated the application with FlexCover.

Flexcover is a code coverage tool for Flex, AIR and AS3. It incorporates a modified version of the AS3 compiler which inserts extra function calls in the code within the SWF or SWC output file. At runtime, these function calls send information on the application’s code coverage to a separate tool. Flexcover computes both line coverage and branch coverage.  (Source: http://code.google.com/p/flexcover/)

Download Link: http://code.google.com/p/flexcover/downloads/list

Flexcover version 0.90 features support for Flex 4.0 & AIR 2.0 . However, there is no direct support of Flexcover with Flex sdk 3.5. Had to try a workaround to make it work. (Workaround: http://code.google.com/p/flexcover/issues/detail?id=37#c5)

Follow the steps below to know your Flex automation coverage:

  1. Modify the custom SDK to use the Flexcover compilers and libraries.Make sure that you are copying sdk-modifications-3_x/lib, sdk-modifications-3_x/frameworks, etc. directly on top of the corresponding subdirectories in the SDK that are named lib, frameworks, and so on.
  2. Generate the instrumented code with sdk modified for flexcover(use the custom SDK created above) and the automation libraries (flex automation libraries and the automation agent swc files). Build the application using the Flash Builder IDE. In the generated build dir, Coverage metadata (.cvm) file would be created.
  3. Install CoverageViewer AIR application and load the cvm file created above in it.
  4. Launch the project application using the Flash Builder IDE.
  5. Run the automation from the automation tool such that it points to the project application created above.
  6. After the automation is complete you can view both the line coverage and branch coverage in the CoverageViewer AIR app. Now you can click on the “Save Report” button to save the file in the Coverage Report (.cvr ) format. The .cvr file incorporates all the information in the drill-down coverage table.

Also, have a look at the documentation present at “/flexcover-0.90/doc/index.html” for detailed information.

PS:  Use the coverage report to continuously learn and adapt your automation scripts till you achieve complete coverage of the application. (An Agile way of doing it)

Categories: Technical

PHPUnit tests for Server-side code interacting with Flex client

October 29, 2010 Leave a comment

Intent: To showcase the ease with which the server-side php code can be tested in a continuous integration environment.

I had an experience working on a flex based application that interacted with php based server. Often it used to happen that the application failed/crashed because of incorrect or no server response. The server side php’s were fragile as new features were constantly being added thereby making the code error prone. The need for continuous integration tests for the server side php code became all the more important for us.

Flex applications follow SOA architecture model, wherein Flex is used to create the client and you connect the client to data using services. In Flex applications you rely on server-side scripts to manage the databases as no direct access to database is supported.

In our application the communication happened over HTTP, Post variables were used to send the request to the server and response was in XML format.

Generally, the way the PHP server side code is organized in the following way:

We have a set of client interacting server api’s (php in this case). On the middle level we have a set handler classes that acts as a wrapper on top of api’s at lower level that interacts with the system such as databases etc.

The client interacting php’s can be tested by a variety of tools such as Jmeter. The problem lies in testing the underneath PHP layer, which often remains untested and is the root cause of failure.

Thanks to ever evolving PHP, there is an efficient way to test it using PHPUnit framework.

The major challenge faced was in understanding the server code and architecture. I also used the webproxy tools like Fiddler and Webscarab to understand the request sent and response from the server.

Pre-requisite

PHPUnit should be installed from http://www.phpunit.de/

Code – Implementation

Lets take a look at the implementation by taking an example. The purpose of this example is to showcase that the PHPUnit test cases can be easily created.

We have a client interacting Login.php file on the server which takes some arguments and call the Login Handler.php “execute()” function. To test the handler class we need to create a PHPUnit test case to test the execute() function of the handler class.

Server-side Code:

Login PHP file


<?php
require_once 'inc/Init.php';
$action = $_GET['action'];
$params = $_POST;

switch ($action) {
case 'login':
require_once 'inc/handlers/user/LoginHandler.php';
ini_set('max_execution_time', '300');
$loginHandler = new LoginHandler( $params['email'],
$params['password'], $params['SSO_TOKEN'],
$result = $loginHandler->execute();
break;
case 'resetPassword':
break;

}
?>

LoginHandler Class


class LoginHandler implements LoginInterface
{      private $email;
        private $password;
        private $sso_token;

function __construct($email, $password, $rememberMe = false, $sso_token = null, $lang = 'en_US')
{
$this->email = $email;
$this->password = $password;
$this->sso_token = $sso_token;
}

public function execute()
{
$successResponse = array();
….....	 \\performs actions for authenticating the user and sending the XML response.
return $result;
}
}
?> 

PHPUnit test case for the above handler class is as follows.

LoginHandlerTest.php


<?php
require_once 'PHPUnit\Framework\TestCase.php';
require_once 'inc/handlers/user/LoginHandler.php';
/** LoginHandler test case. */

class LoginHandlerTest extends PHPUnit_Framework_TestCase {
/**
* @var LoginHandler
*/
private $email;
private $password;
private $LoginHandler;
/** Prepares the environment before running a test. */
protected function setUp() {
parent::setUp ();
\\ Get the email and password for the test.
$this->testHandlerObj = new TestHandlerUtil();
$this->email = $this->testHandlerObj->owner;
$this->password = $this->testHandlerObj->owner_pass;
// Create an object of Login Handler
$this->LoginHandler = new LoginHandler($this->email,$this->password);
}
/** Cleans up the environment after running a test. */
protected function tearDown() {
$this->LoginHandler = null;
parent::tearDown ();
}
/**  Execute the test LoginHandler->execute()  */
public function testExecute() {
$response = $this->LoginHandler->execute();
$respXML = new SimpleXMLElement($response);
$this->assertTrue($respXML['status'] == "success"); \\ The statement checks the success response.
}  }
?>  

The verification points are very important in the test. PHPUnit provides a variety of assert statements to verify the response after execution.

Test Suites are a way to effectively manage the set of multiple test cases.

Test Suite in PHPUnit can be easily created by including all the test cases.

TestSuiteUserHandler.php


<?php
require_once 'PHPUnit\Framework\TestSuite.php';
require_once 'inc/unit_test/handlers/user/ForgotPasswordHandlerTest.php';
require_once 'inc/unit_test/handlers/user/GetCaptchaHandlerTest.php';
require_once 'inc/unit_test/handlers/user/LoginHandlerTest.php';
/** Static test suite. */
class TestSuiteUserHandler extends PHPUnit_Framework_TestSuite {
/** Constructs the test suite handler.	 */
public function __construct() {
$this->setName ( 'TestSuiteUserHandler' );
$this->addTestSuite('LoginHandlerTest');
$this->addTestSuite('ForgotPasswordHandlerTest');
$this->addTestSuite('GetCaptchaHandlerTest');
}

/** Creates the suite.	 */
public static function suite() {
return new self ( );
}   }
?>   

Integrating with Continuous integration environment

Variety of build automation tools can be used with PHPUnit like Ant, Maven and Phing.

Mentioned below is the PHPUnit target for the Ant build script.

<target name="phpunit">
  <exec dir="${basedir}" executable="phpunit" failonerror="true">
   <arg line="--log-xml ${basedir}/build/logs/phpunit.xml TestSuiteUserHandler" />
  </exec>
 </target>

This target can be made to run every time the build is created. Now we have our unit tests integrated with the build machine and bugs if any detected at an early stage of build creation and we get a more robust code.

Categories: Technical