In my previous post, QA Automation and Domain Knowledge, we discussed the importance of acquiring domain knowledge and being intimate with the details of client priorities and needs with empathy. These two things greatly influence the approach that one should have when engaging in an automation testing effort. With regard to automation itself, its general and broad scoped value is reflected in the time and money saved through faster testing and greater assurance of quality. However, the articulation of its specific and more narrow scoped value may vary by project. In other words, the measure to which automation would bring value to your specific project is dependent on your current needs, available resources, flexibility, and expected outcome.

Value

Attempting to truly measure the value of something can be a tricky thing and can become subjective very quickly. Take these Aerodrums for example. As of the writing of this post, it costs $200. But before we talk about whether or not you would pay that much for it, let’s talk about the value that you would get from purchasing it. Let’s start simple.

   
• Numero Uno: Do you have rhythm? If the answer is no, then this item will not be valuable to you at all. Just close that tab and skip to the next paragraph. Just kidding!
   
• Numero Dos: Have you researched what this product is supposed to do or how it is supposed to work? Or have you made assumptions about those two things? Did you notice that you can’t use it outdoors in daylight? Have you also thought about where to mount that high speed motion camera?
   
• Which leads to Numero Tres: How do you plan on using it? If you plan on doing a gig, put simply, this ain’t it. But maybe you live in an apartment and you want to mess around with the drum samples. In that case, this may be a fun way to do some recording.
   
• Based on your answers to these questions, you can more knowledgeably answer this last one: Is it worth the cost? Maybe you do have $200 to spare but in order to really enjoy playing it, you may have to carve time out of your busy schedule. You also may need to make space in a particular room or even get a new laptop for recording. You see, there is a good amount to consider when e-VALU-ating this product and we haven’t even clicked “Add To Cart” yet!

Do You Even Automation, bro?

Here’s a tip: automation testing is usually not for static websites! I mean, you could do it, but manual testing would be best suited for it since the content will not change (much). Dynamic sites would stand to benefit much more from automation since they typically handle data provided by an end-user and have more complex user flows that may or may not change based upon the user provided data. Each additional route in the user flow adds to this complexity incrementally and, in some cases, exponentially making ongoing manual regression testing for every single route or feature overwhelming and impractical over time. Therefore, writing automation scripts for each test case ensures that your application provides its user experience and service as expected in any use case.

Do Your Homework

Let’s be honest. Automation testing comes with a good amount of overhead, especially if it’s your first time doing it. With that said, spend the time to investigate the nature, implementation, and scalability of automation testing. Patience in research leads to a more focused approach and a realistic understanding of your automation needs. If you haven’t yet, set up a QA or Staging environment (which may already be set up by another developer) as well as your local testing environment. Selenium is a staple library for automation testing because of its versatility in handling many programming languages, but there are many others out there that may be more suitable for your needs such as Capybara, Cucumber, and Appium for mobile testing.Determining which programming language in which to write may require more thought than you might realize. If you plan on conducting your automation testing in a silo (apart from the development team and application repository), then writing test scripts in your favorite language may not be a problem. However, if the plan is to have the automation and application code in the same repository, you may need to learn the application’s language and follow the team’s development process if you don’t already. Considering this, the route that you take may also affect how and when your automation test scripts are run. Executing test scripts upon each build of the application code reflects a more continuous integration setup. However, it may be that automation only needs to be run ad hoc or as part of user acceptance testing each month or sprint. Therefore, thinking critically about these things will greatly influence the value that you will gain from automation and the value that you can share with clients.

Identify Need, Acknowledge Expectations and Define Results

How large is the application? Are there multiple user types, including an admin? How many forms are in the app? Is it integrated with any eCommerce tools? What about uploading or downloading files? All of these things, and more, contribute to the overall size and complexity of the application. The larger the size and complexity, the greater value you would experience from automation testing for your particular project.Having a test case management system is a great way to organize your test cases and test runs as well as generate and deliver reports. If you already use one of the many tools out there for manual testing (PractiTest, qTest, Zephyr, TestRail, etc.), check to see if they have an up-to-date API so that your automation test runs can be captured there as well.Does your application need to be tested on various browsers or operating systems? If so, then integrating your automation with a remote/virtual device testing tool such as SauceLabs, BrowserStack, LambdaTest, etc. may be a great resource for you. Depending on the client’s need or the targeted end-user demographic, you may need to ensure that the application runs as expected on older browsers such as Internet Explorer 11 or is responsive in Chrome for mobile devices. The value gained from automation is therefore increased by simply being able and flexible to test according to the client’s specific needs in this way.

Count the Cost

How long do you plan on testing the application? Not all projects are the same and some projects have shorter deadlines than others. If you’re working with a short deadline, you simply may not have the time to setup and configure your automation environment to run your scripts with a reasonable return. So, the longer the duration of the project, the more time you have to set up, configure, write and run tests, refine methodologies, refactor code, and prepare and deliver test results.Remember those integrations that were mentioned a moment ago? Many tools or services require a monthly or yearly subscription plan… and they may even charge per user! So, it is important for you to clearly identify what your needs are and do the math according to your budget.It is often said that many hands make light work. With that said, how many hands do you have for contributing to your automation effort? Again, consider the size and complexity of the application in addition to the project timeline. If you do have many hands, it will also be helpful to determine how to keep the team aligned when writing test scripts and following a particular pattern.If you are new to automation, please believe that none of this is meant to scare you away from engaging in automation testing. However, it is important to take the time to consider and prioritize your automation needs and tooling as much as reasonably possible in order to provide a clearer direction and implementation for your automation testing. It is only then that you can appropriately understand the value that automation will bring to your specific project.

Key Takeaways

   
• The measure to which automation brings value to your specific project is dependent on your current needs, available resources, flexibility, and expected outcome.
   
• Spend the time to investigate the nature, implementation, and scalability of automation testing.
   
• Thinking critically about how and when your automation is executed will greatly influence the value that you will gain from automation and the value that you can share with clients.
   
• The larger the size and complexity, the greater value that you would experience from automation testing for your particular project.
   
• The longer the duration of the project, the more time you have to set up, configure, write and run tests, refine methodologies, refactor code, and prepare and deliver test results.
   
• It is important for you to clearly identify what your needs are and do the math according to your budget.

Up Next! - QA Automation and Manual Testing.Darrin Whitleyis a QA Automation Engineer at Smashing Boxes, providing functional and regression, manual and automation testing support in Java/Kotlin.