[REVIEW] Katalon Studio Analysis

Nowadays, quality assurance activities, especially software testing have become more important than ever due to software products being updated rapidly in order to satisfy customer needs and preferences. One of the most effective and efficient types of software testing is regression testing. Regression tests are performed if any kind of modification, update or migration is done on the software product, in order to verify its integrity and check if there are any new defects that have been introduced into the said software product. In order to increase the effectiveness and decrease the time and money spent on these tests, regression testing is usually done by test automation. Test automation is the type of software testing which automatically performs testing and compares actual results and expected results, using different software tools and frameworks. These frameworks include;  linear scripting framework, modular testing framework, keyword testing framework, data driven framework, hybrid testing framework and behavior driven development framework. Some of these frameworks require a certain level of skill in coding in order to automate the desired test case scenarios. But what if it wasn’t necessary to know coding in order to create these cases? What if manual testers who don’t know coding were asked to create automated test suites while given a small amount of time? That’s where Katalon Studio comes in handy. Katalon Studio is a software tool which provides both keyword driven framework and linear scripting framework for test automation. In this blog post, I’ll be talking about Katalon Studio and how it makes some of the aspects of test automation much easier.

Before we get into the technicalities of it, I would like to say Katalon Studio has one of the most intuitive UIs I’ve ever seen. It feels like every icon, button and window are where they’re supposed to be. With the addition of a very simple and short tutorial, I was able to start creating my first ever Katalon Studio project. The first thing that caught my eye was the “Tests Explorer” tab on the left. By right clicking “Test Cases” and clicking “New > Test Case”, I was able to create my first test case. After double clicking it, I realized that I was looking at the list which I would fill with keywords, based on our case. So I’ve started looking into the keyword library. The amount of functionality the library provided was quite impressive. There were the “regulars” of every automated test case, such as “Click”, “Scroll to” and “Send Keys”. What made me breathe a sigh of relief were all the “Verify” options; I had thought the assertions would have been done from a different place. After checking out all the keywords and observing the first screen that came up on my screen after selecting the test case, which was the “Manual” tab, I started filling the list with the keywords according to the test case given for my first job interview. I specifically wanted to create a test case I had done before, in order to compare the two experiences and the time and effort spent for each one. In order to interact with web elements such as buttons, links, combo boxes and drop down menus, you had to define them as objects by their XPath or CSS from the object repository on the tab on the left and then you had to define them as inputs to whatever function you’re using on the Manual list.

The whole process was very intuitive. Functions such as “Click” and “Mouse Over” only took a single parameter which was one of the objects that we’ve defined earlier and functions such as “Send Keys” took two parameters; the object and the keys I wanted to send. I’ve used the functions that were provided by Katalon only until I had to convert a string character, which was the number of search results, into an integer in order to verify that it was greater than zero, so the search was able to find items. For this, I had to select the “Script” tab from the bottom of the main window and do a bit of manual coding without using the functions. There were several instances like this where I had to do manual scripting, especially when I was managing and comparing web elements in an array. Maybe there was a built-in option to perform these too, but I couldn’t find it and thought it would be easier to just edit the script accordingly. Anyway, I thought it was great that the software didn’t limit you to the provided functions only, so you can both use the functions with ease and edit the script freely. Best of both worlds. Test execution was also very fast and problem-free. The driver needed for the test automation was provided by Katalon Studio itself, so I didn’t have to define a web driver for the test case. This is very convenient; because you don’t have to update the driver or your browser version in the future in case there’s a compatibility issue. The only problem I’ve had throughout test execution was the “Send Keys” function on Firefox. For some reason, on Firefox, there’s an additional ASCII character that’s written after the keys you send into an input field.

Katalon Studio also provides linear scripting framework by its record and playback features for desktop web and mobile web. These features can be found at the top of the window, named; “Record Web” and “Record Mobile”.  The recording doesn’t follow the user’s interactions in an exact manner. It registers the web elements that have been interacted with, into the object repository and maps them to functions just like manual function listing. So the UI for recording is fully provided as linear scripting, but the test case is built by the API from a keyword driven framework. This simplifies the user experience while also making the test case more robust. There are two more options named “Spy Web” and “Spy Mobile” which are used to detect web elements and register them as captured objects. These features can be used in order to simplify the process of defining web elements.

The main disadvantage of using Katalon Studio might be the fact that the only Application Lifecycle Management tool support it provides is JIRA. Companies who are using or willing to migrate to different ALM software such as VersionOne or Swift won’t be able to integrate Katalon Studio to their continuous integration cycle.

If I were to compare my two experiences for the same case; first being the time when I was creating the test suite on Eclipse IDE using Java and the second one being the time I used Katalon Studio; I’d definitely say that the time I spent on the Katalon project was way less and it was definitely easier. I think there’s a huge chance that Katalon Studio will be one of the most popular software tools for test automation in the near future.

Nehir Gökçe

Software Development Engineer in Test