API Testing with Postman Collections in AWS CodePipelinePosted: June 10, 2017
This post explains how to setup an AWS CodePipeline to run Postman collections for testing REST APIs using AWS CodeCommit and AWS CodeBuild.
Step 1 – Creating a Git repository with AWS CodeCommit
AWS CodeCommit is a version control service hosted by AWS. You can create, manage Git repositories on CodeCommit.
For this project, we will create a new repository named postman-sample from AWS Console > Developer Tools > CodeCommit
Clone the newly created repository on your computer
Step 2 – Exporting Postman Collection
Next, we need to export the Postman collection so we can run this using newman cli
Select Collection v2 option from Export Collection dialogue box and click Export
Save the collection in cloned repository folder from Step 1 above
Creating buildspec file
We need to create a buildspec file to tell AWS CodeBuild how we want to run the collection. Before running the collection we need to install newman npm package in pre_build phase and then call newman cli with the collection we want to run in the build phase. We can also specify report options to generate HTML file at the end. We will upload this file to S3 after CodeBuild executes the collection:
Commit & Push the collection and buildspec file to CodeCommit in master branch
You can find the collection and buildspec file used for this example at https://github.com/upgundecha/postman-sample
Step 3 – Creating an S3 Bucket to save report
Let’s create an S3 bucket to save the report file generated by the newman. We can use this file to publish the results.
From AWS Console > Storage > S3 create a new bucket named postmanreport (you will need to use a unique name). You can enable version control on S3 bucket to see historical reports.
Step 4 – Creating an AWS CodeBuild Project
We will use CodeBuild to fetch the changes from CodeCommit and run the collection using newman. We already have buildspec file which has a sequence for execution.
From AWS Console > Developer Tools > CodeBuild create a project named postman-sample.
Set Source Provider as AWS CodeCommit and Repository as postman-sample:
We need an environment to run the build job. Let’s configure the Ubuntu/Node.js environment and Artefacts settings as shown in below screenshot:
Next, we need to configure a Service role. Create a new service role and click Continue
The CodeBuild project is ready. We can test this project by manually starting the build.
Step 5 – Bringing it together with AWS CodePipeline
Finally, we need to create a new AWS CodePipleine to trigger the CodeBuild when a new change is pushed to CodeCommit.
From AWS Console > Developer Tools > CodePipeline create a pipeline named postman-sample:
Configure the Source Location as AWS CodeCommit as shown in below screenshot:
Configure the Build provider as AWS CodeBuild with the CodeBuild project created in Step 4 above as shown in below screenshot:
We just want to run tests and stop there for now. We will not deploy anything in this project so we will select No Deployment option in Deploy section as shown in below screenshot:
Finally, we need to configure AWS Service role as shown in below screenshot:
This will take you to IAM to define the new role:
Back in CodePipeline make sure the role is specified:
Review the CodePipeline configuration and create the pipeline.
You will see the Pipeline created success message. You can try invoking the newly created Pipeline by clicking the Release Change button:
Once the Pipeline is executed you will see a both Source and Build stages in Green (unless there are any errors) as shown in below screenshot:
After a successful run, you can go and check the S3 bucket to view the report.
Newman generates a nicely formatted report as shown in below screenshot:
You can also configure a Lambda function or SNS to send a notification along with the report.