Shift-Left was the testing meme of 2016. Everyone I spoke with was insisting that testers should be involved earlier in the development process, ideally when features were being defined, something I personally agree with. In practice, that usually looked like testers joining planning meetings and then being uninvolved until there was a fully developed user interface. To shift left in a meaningful way, testers need to get a little more technical. My preferred way to do that is by testing at the API level.
APIs can be scary, a little like high school algebra, especially for people that aren’t used to looking at code. I’ll walk through how I test an API, step by step, drawing on testing techniques you are probably already using. Hopefully, this will make things feel a little more welcoming.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
API Testing Tools Introduction
Deciding what tool to use is probably the first, and most common, problem that people run into when trying to get below the user interface. My stock answer is to choose whatever programming language the developers are using. When implementation problems come up, and they eventually will, the programmers become an in-house support group, instead of shrugging “Sorry, can’t help you.” The question of who should fix a broken test becomes easier when the programmers write code in the same language, and the automation is stored in the same place in version control as the production code.
This exercise will demonstrate API automation using Postman and Airborne. Postman is a standalone app that runs as a Chrome process. It allows you to send calls to and get responses from an API, one call at a time. I like to use Postman to explore and learn about and API. The other tool will use is called Airborne. Airborne is a Ruby gem that provides functionality to perform all the normal API actions – POST, GET, Delete, Update – and also a set of assertions to compare the response with a baseline. Programming libraries like this are good for building automation tools that can be run by the programmer on their environment each time they make a change, and to run on a Continuous Integration (CI) server after each new build.
This might be overwhelming right now, there is a lot of new jargon and terminology that comes along with technical testing, but stay with me, it will be worth it.
Exploring API Testing Details
People tend to associate technical testing with building automation, but for me, testing always starts with exploration. I am going to walk through designing tests and sending calls to the Trello API one call at a time. To follow along you’ll need to create a Trello account, generate an application key and a token. The application key and token ensure security. They are essentially long set of test that look like gibberish. For now copy and save them to two different text files.
With the preliminaries done we start by creating a new Trello board – in code. I don’t know how to do that, so I need to read the Trello API documentation to figure out what endpoint to POST to and what data I need to pass in (the endpoint is basically a URL). We’ll call the URL with some data to get it to execute some code on it’s own and return values to us. In the old days we called this remote procedure call, or RPC. To create a new board I need to post to /1/boards. The examples aren’t that great for POST so I’m going to copy one of the GET examples and update it with the right URL.
Here is our first attempt at creating a new Trello board without using the API. Paste this into Postman, but use your application key and token:
https://api.trello.com/1/boards?fields=name,idList,url&key=[app_key]&token=[optional_token]
Enter name in the key field in the body tab in Postman and whatever you want to call your board in the value field. Don’t worry, there’s a video below. After you hit the Send button Postman will update and display some text called JSON and a HTTP status. This is part of the response from Trello letting you know that you did something right:
Using Postman to send an API request to Trello
That is all there is to it, you just created your first Trello board through the API using Postman. Now you have to make decisions about what you want to test. There is some information in the API documentation that would guide my testing. I see a claim that the name key is required and can range from 1 to 16384 characters in length. Based on that I would perform a few tests:
- Try to create a board with no (null is the technical term) name
- Create a board with a name that is 1 character long
- Create a board with a name that is 16384 characters long to see if the documentation is correct
- Create a board that is 16385 characters long to discover what a failure looks like
Creating a board leaves me wondering what characters are OK to send so I would try to discover what countries are using Trello and try to create boards using character sets from those languages. After that I am wondering about how many boards I can create, can I create boards with duplicate names, can I create boards with white space as a name, and is the response data I get back from all of these tests correct. And what about workflow? Can I create, update, and delete a board? Who has access to my board? These test ideas cover just the name field, looking back at the API documentation I can see that there are many optional fields for this API that we could perform similar tests on.
The hard part of exploring an API is test design, discovering what is important to know about the product and how to find that information, not using tools.
Automating API Testing
If you want a set of tests that can be run repeatedly with each build, or as a developer is writing new code, you’ll want to use an automation framework. I mentioned earlier that I would be using the Ruby gem airborne for this part of the exercise. Once you have Ruby and airborne installed on your computer (“gem install airborne”), type this into a text file and name it trello_test_spec.rb. You will of course need to update line 5 with the key and token you were using in Postman.
require 'airborne'
describe 'sample spec' do
it 'should create a new Trello board' do
post 'https://api.trello.com/1/boards/?lists=open&list_fields=name&' +
'fields=name,desc&key=[key]&token=[token]',
{ :name => 'This is a Trello board' }
expect_json_types(name: :string)
expect_json(name: 'This is a Trello board')
expect_status(200)
end
end
After you have that file created, open a command prompt and type this in:
$ rspec trello_test_spec.rb
When the test completes, you should see something like this:
Finished in 2.38 seconds (files took 0.53251 seconds to load) 1 example, 0 failures
This is a simple test that performs a POST to /1/boards to create a new Trello board. The test then checks the response from Trello to see that there is a key of type string, the name value matches what we sent, and the HTTP response is 200 (success). Make a small change on line 5 to ‘This is a Trello board’ and rerun the script. This is what it looks like when a test fails:
Failures:
1) sample spec should create a new Trello board
Failure/Error: expect_json(name: 'This is a Trello board')
expected: "This is a Trello board"
got: "This is a Trello board23"
(compared using ==)
# ./create_trello_board_spec.rb:7:in `block (2 levels) in '
Finished in 2.4 seconds (files took 0.51082 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./create_trello_board_spec.rb:4 # sample spec should create a new Trello board
Where To Start & Next Steps
The automation example we made is simple, it is probably more simple than something you would normally use on a real software development project. The next challenge for you is to make this realistic. Pick an API endpoint in your product, I suggest using login/authentication, and start exploring with Postman. After that build an automated test that will create (POST), update (UPDATE), then delete (DELETE) a user all within one test. After that, pair with a developer or ops person to get that test running in your Continuous Integration system.
That is step one. Now it’s time to really get started.
This is a guest posting by Justin Rohrman. Justin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop various projects.
