API Testing Tools

A selection of API testing tools from 2020. Source: Aldaine 2018.
A selection of API testing tools from 2020. Source: Aldaine 2018.

As organizations move towards DevOps, Continuous Integration (CI) and Continuous Delivery/Deployment (CD), there's a need to have quicker feedback from testing. End-to-end or UI testing are slow. API testing is one solution but it must be easy to create, execute and maintain API tests. It needs to be automated as much as possible. To enable this, many tools are available in the market.

There are both open-source and commercial API testing tools. We can't say which of these is the best. We should select a tool that best suits our project. The selection should be based on features and how well it integrates with other tools and our processes.

Since 2018, some tools are beginning to adopt AI/ML. This could become an essential feature in the years to come.

Discussion

  • What should I look for when selecting an API testing tool?
    Cost vs feature comparison of API testing tools. Source: Aldaine 2018.
    Cost vs feature comparison of API testing tools. Source: Aldaine 2018.

    At the minimum, any tool should make API requests and validate responses. Other features to consider are: easy to install, learn and customize; good documentation; available on multiple platforms/devices; supports multiple protocols (REST, SOAP, HTTP, JMS, TCP/IP); integrates with CI/CD pipelines; can sniff exploratory tests to automatically create API tests; tests but also validates API specifications.

    The syntax for writing tests must be simple enough for those with little coding experience.

    Pick a tool that integrates with your automation framework. For example, REST-assured integrates seamlessly with Serenity.

    For teams, the tool should enable collaboration. It should be possible to share test cases and results, or publish tests and allow others to run them easily.

    Some tools such as SoapUI are dedicated to API testing. Others such as JMeter was designed for load testing but supports API testing. Swagger is meant for designing APIs but can also be used for testing. This is worth considering if you're looking for single unified tool.
  • What are the types of API testing tools?

    Broadly, we can identify the following types:

    • Command-Line: cURL is an example that can query an API and print out the response. It uses libcurl that can be used in any client-side program. Thus, developers can create their own CLI-based API testing tool using libcurl. Dredd is another CLI tool. On Windows, PowerShell is a good choice.
    • Browser-Based: DHC and Postman are examples that can run within a browser. Compared to CLI tools, they offer a user-friendly interface for testers. Postman was initially delivered as a Chrome App, then a standalone desktop app, and then a web app since September 2020.
    • Standalone IDE: Katalon Studio is an example that supports test case generation, CI/CD integration, and many types of testing (API, Web, Mobile, Desktop). Karate is an example that can be installed as a plugin/extension and used within IDEs/editors such as IntelliJ or Visual Studio Code. It supports API testing, mocks, performance testing and UI automation.
  • Could you describe some API testing tools?
    Comparing some API testing tools. Source: Software Testing Help 2020.
    Comparing some API testing tools. Source: Software Testing Help 2020.

    Katalon Studio claims to be "all-in-one test automation solution". It offers free and enterprise versions. Free version supports basic record and playback, REST and SOAP, OAuth for authentication, different types of testing (keyword-driven, data-driven, BDD), parallel execution, and headless execution. It can also import tests from other tools/frameworks such as Selenium, Postman, JUnit, TestNG, SoapUI and Swagger.

    In Postman, tests can be organized into collections that can be shared with others. API design and maintenance can be done within Postman, which serves as the single source of truth. Multiple APIs can be version controlled and tagged. Mock servers and environments can be created. It can monitor API uptime and responsiveness. Newman is Postman's CLI tool for easy integration into CI/CD pipelines.

    An alternative to Postman is Hoppscotch (previously called Postwoman). It runs within the browser and supports GraphQL. As is common, requests can be customized with authentication, headers, and pre-request script. Assertions are in JavaScript.

    OpenAPI.Tools has a list of API testing tools, briefly describing each of them.
  • What's the syntax that tools use for specifying API tests?
    Gherkin syntax is supported by Karate. Source: Intuit GitHub 2020.
    Gherkin syntax is supported by Karate. Source: Intuit GitHub 2020.

    Karate uses Gherkin syntax that was popularized by Cucumber for BDD. REST-assured also supports BDD syntax.

    Taurus allows tests to be written in YAML or JSON.

    Parasoft SOAtest uses visual drag-and-drop. Drag-and-drop method of creating tests is also supported by SoapUI. In addition, SoapUI supports Groovy language, which has a concise expressive syntax for the Java platform.

    ACCELQ is a cloud-based application for which no code needs to be written.

  • Which are the API testing tools in various languages?

    Some tools support many languages. Postman has code generators in many languages that developers can use. RedwoodHQ is another tool that has support for Java/Groovy, Python and C#. Fiddler supports .NET, Java and Ruby.

    API testing in Java domain has been traditionally hard. REST-assured simplifies this with support for XML or JSON format in requests and responses.

    In Ruby, Airborne is an option. It relies on RSpec. It works with APIs written in Ruby on Rails.

    WebInject is a tool written in Perl and therefore requires a Perl interpreter at runtime.

    In JavaScript, Chakram is a tool based on Node.js, Chai.js and Mocha. It uses BDD syntax. Another tool in JavaScript is Frisby that uses Node.js and Jasmine.

    In Python, pyresttest is applicable. It's also a micro-benchmarking tool. Configuration is in JSON or YAML. Hence, little coding knowledge is needed.

  • How is artificial intelligence enabling API testing tools?
    AI/ML helps in automatic generation of API tests. Source: Colosimo 2018a.
    AI/ML helps in automatic generation of API tests. Source: Colosimo 2018a.

    Test engineers have always found UI testing to be easier than API testing. They may not know how all the software components work together via their APIs. API testing requires better understanding of the software. AI/ML is attempting to remove some of this complexity. One approach is to simply monitor UI tests, discover patterns and then use this knowledge to automatically create API tests.

    One tool that generates API tests is Parasoft SOAtest Smart API Test Generator. It's installed as an add-on to a web browser. For example, in an e-commerce site, it would learn that a cart ID is important across API calls. A human could indicate that number of items in the shopping cart is important. AI would learn this and validate this in future API calls.

    Other AI/ML testing tools include Applitools, Testim, Sealights, Test.AI, MABL, Retest, and ReportPortal. AI/ML can catch UI changes and identify them as bugs or features; analyze code changes and execute relevant tests; detect code changes and automatically update existing tests; automate result analysis and defect management.

Milestones

2000
Evolution of test automation. Source: Brown 2018.
Evolution of test automation. Source: Brown 2018.

It's in this decade that APIs start getting popular, partly due to Salesforce, eBay and Amazon. Due to Agile practices, there's also a need to automate API testing.

2010

In this decade, focus shifts to scalability. Adoption of DevOps and CI/CD make it necessary to embrace automated API testing. However, even by the end of the decade, many organizations continue with UI testing.

Oct
2018

Parasoft SOAtest 9.10.6 is released. This includes AI/ML capability that's now part of it's Smart API Test Generator. In general, 2018 is the year when tools start including AI/ML. Focus shifts from just test automation to include test generation.

References

  1. Aldaine, Alice. 2018. "Top 10 API Testing Tools in 2020 (Details & Updates Done!)." Medium, May 28. Accessed 2020-10-07.
  2. Brown, Asha Jane. 2018. "Testing trends in 2019 and further." Blog, QATestLab, November 6. Accessed 2020-10-08.
  3. Business Wire. 2020. "Postman Launches New Web Client to Simplify API Development." Business Wire, September 2. Accessed 2020-10-07.
  4. CloudQA. 2019. "AI in Testing: The third wave of automation." Becoming Human, on Medium, April 23. Accessed 2020-10-07.
  5. Colantonio, Joe. 2020. "11 top open-source API testing tools: What your team needs to know." TechBeacon, January 7. Accessed 2020-10-07.
  6. Colosimo, Chris. 2018a. "Revolutionizing API testing with artificial intelligence." SD Times, May 3. Accessed 2020-10-07.
  7. Colosimo, Chris. 2018b. "To make API testing easier, add machine learning to your AI." Blog, Parasoft, October 2. Accessed 2020-10-07.
  8. Groovy. 2020. "Homepage." Groovy. Accessed 2020-10-07.
  9. Hoppscotch. 2020. "hoppscotch/hoppscotch ." Hoppscotch, on GitHub, October 8. Accessed 2020-10-08.
  10. Intuit GitHub. 2020. "intuit/karate." Intuit, on GitHub, September 25. Accessed 2020-10-07.
  11. Katalon. 2020. "Homepage." Katalon. Accessed 2020-10-07.
  12. Lane, Kin. 2019. "Intro to APIs: History of APIs." Blog, Postman, October 10. Accessed 2020-10-08.
  13. Laurent. 2016. "Getting Started with BDD (Part 1)." Blog, Cucumber, April 18. Accessed 2020-10-08.
  14. Parasoft. 2020. "Automated API Testing Tool Powered By Parasoft SOAtest." Parasoft. Accessed 2020-10-08.
  15. Parasoft. 2020b. "Parasoft SOAtest Smart API Test Generator." Parasoft. Accessed 2020-10-08.
  16. Postman. 2020. "Postman: The Collaboration Platform for API Development." Accessed 2020-10-08.
  17. Postman GitHub. 2020. "postmanlabs/postman-code-generators." Postman, on GitHub, October 8. Accessed 2020-10-08.
  18. Postwoman. 2020. "Homepage." Postwoman, August 16. Accessed 2020-10-07.
  19. Skillgigs. 2019. "The 10 Best API Testing Tools on the Market." Skillgigs, May 14. Accessed 2020-10-07.
  20. Software Testing Help. 2020. "10 Best API Testing Tools in 2020 (SOAP and REST API Testing Tools)." Software Testing Help, September 7. Accessed 2020-10-07.
  21. Terefe, Capital. 2017. "Top 10 Most Popular API Testing Tools." Medium, November 25. Accessed 2020-10-07.

Further Reading

  1. Aldaine, Alice. 2018. "Top 10 API Testing Tools in 2020 (Details & Updates Done!)." Medium, May 28. Accessed 2020-10-07.
  2. Colosimo, Chris. 2018a. "Revolutionizing API testing with artificial intelligence." SD Times, May 3. Accessed 2020-10-07.
  3. Bagmar, Anand. 2019. "Test Automation in the World of AI & ML." InfoQ, February 7. Accessed 2020-10-07.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
Avatar of user arvindpdmn arvindpdmn
2
0
931
DevCoins due to articles, chats, their likes and article hits are included.
1217
Words
1
Authors
2
Edits
0
Chats
0
Likes
4548
Hits

Cite As

Devopedia. 2020. "API Testing Tools." Version 2, October 8. Accessed 2023-11-13. https://devopedia.org/api-testing-tools
Contributed by
1 author

Last updated on
2020-10-08 14:11:54
tools testing api

See Also

  • API Testing
  • Integration Testing
  • Levels of Testing
  • Types of Testingz
  • Service Virtualization
  • Representational State Transfer