Saturday, August 8, 2015

Leveraging Personas to Support Test Automation


Leveraging Personas to Support Test Automation


It has been far too long since I wrote an article about test automation.  Since my last article, life has placed me on an interesting path!  During my writing hiatus, I had an opportunity to hangout with family and friends as well as expand my educational horizons by taking certification courses in Scrum while visiting Chicago.  Upon my return to Sweden, I embarked upon a cross country drive through Sweden, Denmark and Germany which not only allowed me to see new sites but to visit friends along the way.  Once I had an opportunity to sit still, work kept be unusually busy and left little to no brain power after work to entertain anything but pure relaxation and enjoying the downtime.

Although my work has taken me to new heights in my career, I still love learning new techniques or exploring ideas to support test automation challenges.  One of the latest ideas I have been experimenting with is using UX developed personas to add spontaneous, dynamic interaction during a test automation run.

 For some of us testers, we do not have the luxury of managing our own test environments.  This means we cannot always expect to have a known, stable base state.  So the problem I was trying to solve was automatically testing a site in its entirety when at any given time, the content available is either unknown or can change without expectation.  One idea was using a combination of a well-defined testcase, the information contained in a UX persona and algorithms to select the required data from the currently available content to click on desired elements or exercise the functions defined in the test case.

How to Bring the Idea to Life


First, I collaborated with a UX developer to create personas that included characteristics such as:

-       the functions they would use on the site
-       the type of configurations they would prefer
-       the type of selections they would make
-       the site features are most often used
-       the language and region options they would select
-       The browser / OS configuration used by the fictitous person
-       the devices they would use and when
-       and of course this list can go on and on

Second, I captured the information provided by the personas in a configuration.  You can choose your preferred method but I elected to try the Spring framework to both manage the configuration of the persona, provide information to the methods for function or content selection, and to set or get data when required.

Third, modify the TestFramework to include classes and methods designed to:

-       Setup profiles/accounts or select the persona’s desired language and configurations
-       Select from the currently available content and feature set
-       Use the selection to perform the required action
-       Perform a test verification
-       Report the findings

Fourth, create the test cases to include steps that:

-       Select a persona
-       Call persona getter, setter, action or verification methods
-       Provide the test results which should include, minimally, screenshots for failed results

 Other Uses of UX Personas in Test Automation

In addition to doing smart test automation, I can also leverage personas to do:

-       Performance and load testing
-       Mobile Device Testing
-       Localization testing
-       Model-Based Testing
-       Behavior Driven Development Testing
-       Test setup and configuration

As always, please share your ideas on how UX personas can support both manual and automated testing.  I will continue having fun learning how to develop a fictitious user groups to assist with testing and development.

 
Happy Automation!

______________________________
Margaret Thomas has worked in the testing profession since 1996 and worked within a number of industries including mobile, finance, medical and insurance.  She developed a strong desire for automated testing when she began working within finance and has utilized a number of testing tools both commercial and open source.  She is also the author of the technical blogs found at http://mag4automation.blogspot.se/.  All articles are exclusively based on her experience and are of her own personal opinion and views.

 

 

Thursday, November 21, 2013

Part I – What’s Required for an End to End Data Modeling Test Framework


Part I – What’s Required for an End to End Data Modeling Test Framework

Overview


Data Model testing is increasing in popularity with tools such as TOSCA and GraphWalker and honestly I must say it is an ingenious way to develop tests and becoming ideal for agile development teams.  The purpose of this simple article is to provide you with:

  • Suggested tools and software products required to learn data model test automation (feel free to choose other tools).
  • Provide a recommendation for a holistic approach to test management, testing and test automation.  Many test automation practitioners and companies treat test automation as a separate activity however I sit in the camp of those who include it in their overall approach to testing.

The diagram below is intended to support developing your Test FrameWork using the following techniques:

  • Selecting tools/products for code development, test management, automated test launch and reporting
  • Building the project as a Maven Java project
  • Using Maven to manage your project’s dependencies
  • Download additional resources/libraries by using Maven
  •  Provide a high level idea of how to build your Test Framework


Have fun and good luck and we will continue this discussion in subsequent articles.
_______________________________________________________________________________________________________________
Margaret Thomas has worked in the testing profession since 1996 and worked within a number of industries including mobile, finance, medical and insurance.  She developed a strong desire for automated testing when she began working within finance and has utilized a number of testing tools both commercial and open source.  She is also the author of the technical blogs found at http://mag4automation.blogspot.se/.

Saturday, September 14, 2013

Hi I am UX, Nice to Meet You Automation



UX


In this era of evolving development methodologies from the traditional waterfall type methods to its progeny agile project and development methodologies, we are introduced to a new role known as UX.  Well, the concept is not new but for many of us, we are just now becoming acquainted with it.  This clever acronym can be expanded to describe a team, a concept, a process and a methodology that supports today’s customer-facing type products.  UX is short for user experience and can be defined as “any aspect of a person’s interaction with a given system, including the interface, graphics, industrial design, physical interaction, and the manual.”  The full definition and more information can be found here.

As testers, we have lately been introduced and bombarded with new terms such as:  
  1. Flows and Navigations Maps
  2. Persona
  3. Prototypes
  4. User Experience
  5. User Journey
  6. User Stories
  7. UX
  8.  UX Designs
  9. UX Designers
  10. UX Developer
  11. Wireframes
The above is not an exhaustive list but it identifies people, techniques and processes the agile concept is suggesting to invest in to improve the design-ability and tested feasibility of a desired  innovation to develop a new or improve an existing system.  The output of a UX team can prove to be an invaluable input into the test team’s efforts. 

So what!  What can the UX team provide that I can use for my test effort?


From my short experience working with UX teams, I have identified three major outputs that can help re-invent the testing and test automation approaches.  Those outputs are:

  • Project specifications and user stories
  • Wireframes
  • User journeys
All testers know it is (or should be) a requirement to involve testers in the earlier stages of the development process.  So any and all information shared is of value to our efforts in understanding how to develop the required tests.  Traditionally, in the early stages of receiving the information, it was used primarily as analysis and testers would provide feedback based on their experience and knowledge of the business and solutions already utilized.  We can debate, if there is any value in defining tests at the earlier design stages that are prone to frequent changes.  If one is an automation specialist, then based on many debates I have read, it is way too early for most to begin thinking about test automation.  However, as the above design type documents are finalized and approved, they provide a wealth of input that can both expedite and advance the testing and test automation practices.  For the sake of this article and the subject of my blog, I will limit this discovery to how UX can support test automation specifically.

Project Specifications and User Stories


This output from the UX team provides helpful and invaluable information source to both functional automated and performance testers.  These documents often include important details such as:

  • Design characteristics
  • System flow
  • Error and warning handling
  • Expected performance metrics

If we, automation specialists, have incorporated a test management solution as part of our test automation, then this is a great opportunity to leverage entering the required information to support:  sprint/release/build handling, test user stories/requirements, configurations, testset types, and test case organization.  For more information, review the article Agile Test Planning (using SmartBear’s ALMComplete).  At this early stage, it is not required for us to begin creating code, but we can mature the supplemental test artifacts our framework leverages to support our automated test runs and reporting.
I am not an expert in performance testing however I would imagine the defined performance goals will allow the performance testers to begin creating their approach to the project by identifying how performance can help the project meet the identified goals, if the goals are reasonable and achievable given the expected user load and architecture of the system, and begin identifying the metrics and boundaries in which they should test in and/or surpass to test the system’s performance and scalability.

Wireframes


Wireframe design or modeling provides a visual guide of a customer-facing product.  As both testers and test automation specialist, the wireframes can provide us details we can capture at an early stage of how to expand our exploratory efforts as well as what type of framework support may be required. 
For automated exploratory testing, we can begin to identify characteristics we may wish to routinely verify the look, feel and usability of the system.  We can identify a number of checks we wish to perform such as field min max input, text size and color, select listbox items, clicking buttons, system intrusion features, etc.  For visual characteristics, the test automation specialist can use a number of techniques to retrieve the values of property settings to ensure it adheres to the agreed guidelines instead of relying 100% of a person’s sight and attention span especially after the fifth time someone exercised the same exploratory tests.

In addition, the test automation specialist can begin to identify what framework upgrades or designs that are required as well as collaborate with the developer to identify what techniques and/or tools he will use to bring the designs to life as a way to ensure test automation compatibility.  This information can be used to identify if an upgrade of the existing tools are required or the acquisition of a new tool.

User Journeys


This is perhaps the most exciting output from a UX team.  The current UX team prefers to model a user experience based on a persona type.  These models are often created using diagramming tools such as Visio or SmartDraw.  With the innovation and growing usage of model based test automation tools, this creates a wonderful collaboration between the UX designer, the tester and the test automation specialist!  These user journeys should model how a particular type of person will utilize the system.  This means the model has captured the expected behavior of the system as the user makes the journey through it.
In a nutshell, here are the basic sets of test cases required for testing the new or changed system!  As the tester, why re-invent the wheel!?

As the manual agile tester, we can use the user journey models to add additional models to add more sophisticated journey paths as well as test for negative conditions.  We can enhance the models by injecting keywords, test data or test data location, trace to requirements and defining the edges (actions) and vertexes (verifications) of the models.  If a test management system is incorporated, a reference locater to the model can be included in a defined test case.  The benefit of leveraging the test management system is to have the ability to link the automated tests to other project artifacts.

As the test automation specialist, we can use the information provided to modify and/or add the keyword support required by the models.  Additionally this technique will allow the framework to walk through a model sequentially or randomly which means it will possess a level of artificial intelligence to identify the required permutations for the defined user journey for both smoke and regression testing with a bit of spice by using random generated test runs that can identify unanticipated permutations and lead to discovering unanticipated defects.

Conclusion


The invention of UX provides an invaluable support organization for the testing organization.  If I had my way, the development process would place the test team in a collaborative state with the UX team which empowers testing to be positioned as ready or near ready for testing on deployment from development.   This means we can further support the idea of sophisticated automatic testing on deployment of changes and depending on the results, acceptable pass rates can deploy those changes to production right away which supports the idea of continuous testing and delivery.  This provides greater band-width of time to the test effort for manual exploratory testing, analyzing non-acceptable pass rates, manual bug verification, reacting to production issues, etc.  

For us long time testers, this may feel like a scary trend as we feel our importance is diminished but on the contrary.  I believe it elevates the importance to our role yet sophisticates our efforts so we are part of the desire for continuous delivery and improvements to meet the immediate needs of the customer.

_____________________________________________________________________________________________________
Margaret Thomas has worked in the testing profession since 1996 and worked within a number of industries including mobile, finance, medical and insurance.  She developed a strong desire for automated testing when she began working within finance and has utilized a number of testing tools both commercial and open source.  She is also the author of the technical blogs found at http://mag4automation.blogspot.se/.

Saturday, September 7, 2013

On the Automation Battlefield...Developer vs. Tester, Who Will Be Victorious!



FIRST, LET ME SET THE SCENE

In this corner of the ring we have the testers and in the other corner, we have the developers.  Both groups are from various agile development teams.  Sitting in the center ring is continuous delivery!  Ladies and Gentlemen it is beginning to turn into an orgy of heterogeneous agile and automation practices all trying to ride the homogenous pipeline process of continuous delivery.  LET’S GET READY TO RUUUMMMMBBBBBLEEEE!!

LET’S GET A LITTLE SERIOUS NOW

Ok I know I am just being silly however it is the view I have of our company’s test and automation practices.  The most intriguing discovery being a part of this fascinating team of brilliant people are the various practices towards test automation and not just by different teams but also how developers and testers approach test automation.  It dawned on me that there is such a thing we can call “developer-centric automation” and “tester-centric automation.”  This concept goes far beyond the idea of test automation.  It really describes the desired approach of an individual or team to solve their test automation requirements.

DEVELOPER-CENTRIC AUTOMATION

Developer-centric automation borrows a lot of the same concepts from unit testing.   The test automation suite is often approached with the idea that the test suite is part of the development project and actual tests are part of the java\main and java\test structure.  Test cases are created like functions/methods (depending on the programming language one is using) and sometimes are supplemented by a developer type of testing tool for example like TestNG.  Test data is often married to the test cases or passed in via xml or csv files.  The developer-centric method definitely borrows the concept of using an IDE to develop the code, a code management tool like subversion to manage and share the code, a tool like Jenkins for running the project and capture results by leveraging the reporting facilities of tools like TestNG and/or log4j.  Developer’s love stack trace type reports and can use these reports to not only verify the test results but also analyze the code’s behavior upon failure.  Some developers are beginning to supplement their framework by using BDD tools like Cucumber which allows them to either collaborate with testers or provide a normal written view of the automated test cases.

There are pros and cons to everything and on the pros side, this method of test automation can easily be versioned and managed with the code that supports the application.   If it is a java project then often it leverages Maven which helps to build and run the project as well as integrate into tools like ThoughtWorks Go, Jenkins or Hudson.   It will often execute quite rapidly.  It is less prone to errors and does a better job staying synced with the application under test.  It is more adaptable to supporting both front-end and back-end testing.  Finally, because often a developer at heart is creating these types of automated solutions they encounter less problems related to object recognition primarily because they can improve their code to make the recognition easier.

The downside of frameworks developed using this mindset is they lack true visibility and traceability to other test artifacts.  They are often inflexible by nature which only offers very static regression test type of test automation.   They can be guilty of non-reusability for other applications under test simply in the sheer manner that they are developed.

TESTER-CENTRIC AUTOMATION

The tester-centric automation will take into consideration aspects of test management, traceability, cross-functional and platform support, and test function reusability.  This method demands very detailed reports that outlines the test steps used for every executed test case and visibility of expected and actual results.  It also places a huge value on screen shots during execution as well as the propagation of test status to other test artifacts such as user stories, requirements, releases, test sets etc.  The sheer desire of how the tester prefers to approach automation is clearly visible in the number of test automation tool suites designed for testers which some developers would dismiss as a non-essential tool.  Test automation tool suites provide functionality to allow the tester to provide visibility and insight into the automated test cases and their results.  It allows them the flexibility of reviewing and analyzing results by test run, releases as well as baselines.  It also has the desire to be very configurable allowing test execution on multiple platforms, version, devices, jurisdictions, OS, browsers and the list can go on.   Finally, this method may prefer to maintain a library of mapped object names the test uses to interact with the front end of an application.

The upside is test automation is a natural part of the testing process.  It can provide a wealth of information to the tester allowing him not only to analyze results from a given test run but perform comparative analysis across multiple test runs, configurations, test data, times and even light performance testing which can highlight problems in latency, responsiveness,  and user experience.  Everyone can have insight and visibility into the automated test cases and often understand the results of the test run.

The downside is often automated tests are not managed with the product under tests codebase which can lead to false findings due to the automated test suite being out of sync with the version of the system.  Object recognition can prove to be quite tedious and taxing especially when attempting to do so from a blackbox perspective.  Most tester-centric automation does not leverage tools to help manage nor share the test code.  There is less of a chance to bring together front-end and back-end test automation support depending on which test tool you are using.

HYBRID-CENTRIC AUTOMATION

All of us desire to have our cake and eat too.  We in the development organizations are no different.  I found the best test automation frameworks taps into elements from both develop-centric and tester-centric methods.  This hybrid approach combines all of the positive qualities while mitigating the negatives.  I would argue the only con is finding individuals who can program like a developer yet think like a tester.  My belief is the hybrids are coming via the roles of Agile Testers who work very closely with developers and they either possess the skills required to develop frameworks or they are positioned to collaborate with those who possess the skills as part of an agile development team to deliver the best of both world solutions.

CONCLUSION

AND THE WINNER IS…..EVERYONE!!!!  Every approach has its pros and cons.  Each approach is designed to fit the team’s test automation requirements.  The approach is based on the individual’s or team’s idea of how test automation should support their effort.  So this is my humble hypothesis.  What is your opinion?  What type of innovations and approaches have you had the benefit of participating in or witnessing?