In previous posts I talked about CDD (Contract Driven Development) and what are the advantages and disadvantages of using it. One of the advantages was the possibility to produce REST-Assured tests that will behave in the same way as we configured in the contract, allowing us to increase the code coverage of our software, and therefore the quality.

Well, not everybody uses the CDD approach, but that does not mean that they can not test their software. We can either develop REST-Assured tests by ourselves or choose different tools which make it a little bit easier. Today I would like to introduce you Karate.

What is Karate

Karate is an open source tool that we can use to test our APIs. The main objective of Karate is to reduce the barrier of entry when writing tests, using a syntax that can be understood by non-developers as well.

In this way, it is not necessary to have advanced programming knowledge, since the “scenarios” defined in our tests will be written using the BDD syntax.

These scenarios will define the behavior of our API, and one of the advantages is the ability to make assertions and link one request to another, so that one scenario can make several requests to our API and thus check the functioning of a complete use case instead of a simple http call.

Designing features, the BDD syntax

BDD means Behavior Driven Development and consists of defining the behavior of a feature using a standard language. It encourages tech and non-tech teams to use conversation and concrete examples to reach a common understanding of how the application should behave.  Following the TDD approach, features can be used as well as Acceptance Criteria prior to the development.

One of the main advantages of the BDD syntax is that it can be understood for both dev and non-dev teams. The Features will be then written using the keywords, GivenWhenThen:

  • Given [target]
  • When [action]
  • Then [result]

This is the structure of a scenario using the BDD syntax. We will use it as a template to define our tests, so at the end we will have something like:

  Scenario: get projects should return 200 and list with projects

    Given path 'projects'
    When method GET
    Then status 200
    Then match each $ contains { projectId: '#number' }
    And assert response.length == 2
    * def projectId = response[0].projectId
    * print projectId

As you can see in the example above, it is possible to build multiple scenarios within a feature, so that we could cover all the edge cases related to the login feature in this case. Easy, isn’t it?

Reporting

In my opinion, one of the most important things when working in a development project is the transparency between co-workers or even with the client or consumers.

As developers we have to make sure that our software reaches the desired quality level, and we do not have to be afraid to show it freely. Having a good code coverage is a good start and with Karate it becomes even easier. Furthermore, test reports will be automatically produced and we can share or make them somehow public to the stakeholders.

In the Karate standard reports we can visualize in a browser what our tests are doing and the final result. Those reports are good enough to demonstrate what our tests are doing (http requests, assertions…) and what status they have, either passed, skipped or failed.

However, one of the features I like the most is the integration between Karate and Cucumber-Reports. 

Cucumber is one of the most famous tools used in BDD, and it is in my opinion more standarized and used than Karate. Working on different projects I realized that many teams or companies use Cucumber for their tests, but I will get back to this point later.

One of the things I like the most when working with Cucumber is the test reports. I personally think they are cleaner and display more information that the standard ones generated by Karate, but this is not a problem. As I said above, it is possible to integrate the output of the Karate tests with Cucumber, so running the same tests that on the image above using Cucumber we will obtain:

Using one type of report or another is a matter of taste, but having any of them will allow us to demonstrate if our software is running as it should.

If we run those tests automatically on Jenkins for example, installing the Cucumber reportsplugin will allow us to see the result directly after each build.

Karate is much more than integration tests

Reaching a good quality in a software product is not an easy task. Having a good code coverage with tests is a very good start, but quality means for me many other things like security, performance or even programming practices.

Let’s focus on performance for now. We need to ensure that our software will be available and working even if it is under a huge load of traffic, and for this purpose is advisable to carry out a load test.

I will not deep into what load tests are but I will just say that one of the most used tools for that is Gatling. Gatling tests are written in Scala and basically define scenarios which test our API. Yes, I said scenarios and we have already developed some of them for our integration tests with Karate, so why not reuse them?

This is one of the points that made me choose Karate over Cucumber, because we can develop our features to test our API and afterwards reuse them to perform load tests.

The Karate-Gatling library allows us to write load tests in just a couple of lines of code (Scala) using our already developed features. We can even automate them via pipelines for example to check how our API behaves after making a new deployment. The Gatling plugin for Jenkins will read the generated reports and make them available for us via browser.

This is the global overview, where we can see some statistics about all the http calls that have been made during the test. If we want to know more detailed information about a specific call, we can switch to the details tab:

Summary

Testing is hard but there are several tools or frameworks that make it a little bit easier, or even funny I would say. Using Karate we can:

  • Design our tests even with non-technical people, which allows us to spread the knowledge about how the api should work.
  • We will save a lot of work thanks to its integration with Gatling, which will allow us to use our features for load testing as well.
  • We will be able to generate great reports that we can share with other teams and/or interested members.
  • Posibility to automate everything through pipelines to make sure our software works as it should at all times.

I hope this helped you to understand how Karate works. In the next posts I will show you in a more technical way how to develop some features using Karate and how to use them to run integration and load tests against an API, so stay tuned!

See you!