Keyword Test Automation

Keyword based test design and test automation is founded on the premise that the discrete functional business events that make up any application can be described using a short text description (Keyword) and associated parameter value pairs (arguments). For example, most applications require the user to login; the Keyword for this business event could be "Logon User" and the parameters could be "User Id" and "Password". By designing Keywords to describe discrete functional business events the testers begin to build up a common library of Keywords that can be used to create Keyword test cases. This is really a process of creating a language (Keywords) to describe a sequence of events within the application (Test Case).

When properly implemented and maintained Keywords present a superior return on investment because each business event is designed, automated, and maintained as a discrete entity. These Keywords can then be used to design Keyword test cases but the design and automation overhead for the Keyword itself has already been paid. When a change occurs within any given Keyword the impacted test cases can easily be found and updated appropriately and once again any design or automation updates to the Keyword itself are performed only once. Compare this to the Record and Playback approach which captures a particular business event or part of the business event each time a test case traverses it - if there are 100 test cases that start with logging on, then this event will have automated 100 times and there will be 100 instances to maintain.

Keyword Development

Development of Keywords should be approached in the same manner as any formal development effort. Keywords must be designed, coded, implemented and maintained.

Design The Test Designer is responsible for Keyword design - at a minimum the design of a Keyword should include: Keyword Name, Keyword Description, and Keyword parameters.

Keyword Name
A standard Keyword naming convention should be drafted and followed to allow designers to efficiently share Keywords. The Keyword name should begin with the action being performed <action> followed by the functional entity <entity> followed by descriptive text <text> (if required). Here are several common examples:

• Logon User - Logon User
• Enter Customer Name - Enter Customer Name
• Enter Customer Address - Enter Customer Address
• Validate Customer Name - Validate Customer Name
• Select Customer Record - Select Customer Record

The Keyword name should be a shorthand description of what actions the Keyword performs.

Keyword Description
The Keyword description should describe the behavior of the Keyword and contain enough information for the Test Automation Engineer to construct the Keyword. For Designers the description is the Keyword definition, for Automation Engineers, the functional specification. This should be a short but accurate description - here is an example for the Keyword "Logon User":

Logon User Description: On the Logon screen enter specified User Id and Password and then press the OK button.

Keyword Parameters
The Keyword Parameters should capture all the business inputs that could impact the immediate business event being defined by the Keyword. The simplest and most reliable method for getting the appropriate list of parameters is to take a "capture what is displayed" approach. For the Keyword "Logon User" the application displays three elements: "User Id", "Password", and OK button - the two parameters required to support this Keyword are "User Id" and "Password". The OK button does not require a parameter because the Keyword description states that the OK button will always be pressed. If there were multiple buttons (i.e. OK, CANCEL, and EXIT) then a third parameter "Press Button" would be required and the Keyword description would have to be modified.

Code

The Test Automation Engineer takes the Keyword Name, Description, Parameters, Application Under Test, and Keyword development standards and constructs the Keyword. If there are any issues with the design aspects of the Keyword the Automation Engineer liaisons with the Test Designer and the design is modified to clarify the intent of the Keyword. If there are any automation / engineering challenges then the Automation Engineer liaisons with the Development team and the Tool Vendor to find an appropriate automation solution that fits the automation framework.

Implement

Keyword implementation follows the same path of any shareable project resource. At the minimum the completed Keyword should be: reviewed by the Test Designer, Unit Tested by the Automation Engineer, Function Tested, and migrated into the project Testware. This does not need to be a complex or extensive process but it should ensure that any implemented Keyword is published to the testing team and functions as expected.

Maintain

Keyword maintenance occurs when: a Keyword defect is detected, a business event changes, or a when Keyword standards are modified. Keyword maintenance follows the same deployment path as Keyword Development: Design, Code, and Implement.

Keyword Test Case

Keyword Test Cases are a sequence of Keywords designed to test or exercise one or more aspects of the application or applications being tested. Keyword Test Cases must be: Designed, Executed, and Maintained. Keyword Test Cases are the responsibility of the Test Designer / Tester, the Automation Engineer only becomes involved if a defect occurs during Keyword Test Case execution. It should be noted that the Keyword design paradigm is often used in the absence of Keyword automation - it is an effective standalone test design paradigm.

Design
Keyword Test Case design involves planning the intent of the test case, building the test case using Keywords, and testing the design against the application or applications being tested. At first glance this does not appear to be any different than any other method for test case design but there are significant differences between Keyword Test Case design and any freehand / textual approach to test case design. Keyword test case designs are: Consistent - the same Keyword is used to describe the business event every time, Data Driven - the Keyword contains the data required to perform the test step, Self Documenting - the Keyword description contains the details of the designers intent, Maintainable - with consistency comes maintainability, and finally supports automation with no transformation from test design to automated script. Test Designers gain the power of test automation without having to become Test Automation Engineers.

Execution
The tester can perform keyword test case execution manually by performing the Keyword steps in sequence - this should be performed as part of the Keyword verification process. Test Cases, which are constructed using Automated Keywords, can be executed using the Test Automation tool or by an integrated test management tool. Test Case execution should always be a mechanical exercise whether automation is in use or not. The test case should contain all the information necessary to execute the test case and determine its success or failure.

Maintenance
Test Case maintenance must occur when changes occur in the application behavior or in the design of a Keyword that is being used in one or more test cases. A properly implemented Keyword framework will allow the Tester to find all instances of a Keyword via some query mechanism - reducing the often-painful process of finding the impacted test cases to one simple step. Furthermore, a well-implemented Keyword framework should support global changes to Keyword instances.

Keyword Implementations

GUI (graphical user interface)
Keyword solutions for GUI based applications are the easiest to understand and implement. Most shareware, freeware, and commercial applications of Keyword testing deal with this space.

API (application programming interface)
Keyword solutions for API based applications appear more complex on the surface but once these applications are broken down into their discrete functional business events their behavior is much the same as an equivalent GUI application. If the business event were "Logon User" it doesn't really matter what which application mechanism is used to implement the event, the Keyword would look and behave the same if the business drivers were the same. There are several Keyword solution vendors that deal with the API space and the same vendor often has a solution for GUI applications.

Keywords and Test Phases

Unit Test
Keywords can be applied to Unit test but it is not recommended. The development group, using the tools and techniques available in the development suite, should do unit test.

Function (Integration Test)
Keyword test solution focused on designing and implementing Keywords as discrete functional business events offer one of the most cost effective and maintainable test frameworks for Function test. In fact, if test automation of a GUI or API based application is required or desired there are few frameworks that can match its short term or long term ROI (return on investment).

System Test
A Keyword based testing solution that leverages the Keywords from Function Test to the System Test phase will help expedite the testing process. An effective Keyword framework will allow the Test Designer to combine function level Keywords into System level Keywords. System level Keywords deal with complete business events rather than the discrete functional business events that make up a business thread. For example a System level Keyword could be "Complete Customer Application" that could be made up of this chain of Function level Keywords: "Enter Customer Name", "Enter Customer Contact Information", "Enter Customer Personal Information", and "Save Customer Record".

User Acceptance Test
Keywords can be applied to User Acceptance Test but it is not recommended unless this is an extensive phase of testing. The end user community using the tools, techniques, and processes available in production best perform user Acceptance Test