Add CI articles (#131)
* Add CI overview and Circle articles. * Update continuous-integration-circle.md * Update continuous-integration-circle.md * Update continuous-integration-circle.md * Update continuous-integration.md * Update continuous-integration.md * Updates * Update wording in Circle article. * Update continuous-integration.md * Add travis CI article. * Update continuous-integration-travis.md * Update continuous-integration.md * Add jenkins article. * Update continuous-integration-jenkins.md * Updates. * Remove SRN references. Update incorrect prism conduct command. * Update continuous-integration-circle.md * Update continuous-integration-travis.md * Update continuous-integration.md
This commit is contained in:
Ross McDonald
committed by
Robert Wallach
Robert Wallach
parent
27bb1b6d6b
commit
7fbb9211fe
94
articles/testing/continuous-integration-circle.md
Normal file
94
articles/testing/continuous-integration-circle.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Continuous Integration with CircleCI
|
||||
|
||||
Integrating Prism into your Circle CI pipeline is easy. The simplest way to get
|
||||
up and running is by using [Stoplight's Prism Docker
|
||||
image](https://hub.docker.com/r/stoplight/prism/).
|
||||
|
||||
To get started, you will need to reference the Prism Docker image in the
|
||||
`docker` section of either the `build` or `test` sections of your Circle CI
|
||||
configuration file. This file is typically located under the `.circleci`
|
||||
directory in the root of your repository.
|
||||
|
||||
When integrating Prism into a CircleCI pipeline, there are two different
|
||||
approaches:
|
||||
|
||||
* Starting Prism in the background to act as either a mock or
|
||||
contract/validation server
|
||||
* Having Prism conduct a scenario against a running server by running as a
|
||||
dedicated test step
|
||||
|
||||
## Running Prism in the Background
|
||||
|
||||
Below is a sample Circle CI configuration file (version 2) using Prism as a
|
||||
mock or contract/validation server:
|
||||
|
||||
```yaml
|
||||
version: 2
|
||||
jobs:
|
||||
build:
|
||||
docker:
|
||||
# The first image is where your commands will be run,
|
||||
# so be sure that the prism image is not first
|
||||
- image: your-normal-image:your-version
|
||||
# Customize the 'command' to fit your needs.
|
||||
- image: stoplight/prism:latest
|
||||
command: [prism, ...]
|
||||
steps:
|
||||
# Run test suite against http://localhost:4010
|
||||
...
|
||||
```
|
||||
|
||||
Once the Prism container is started, it will automatically start listening on
|
||||
`http://localhost:4010` for any open connections.
|
||||
|
||||
## Running Prism in the Foreground
|
||||
|
||||
Below is a sample Circle CI configuration file (version 2) with
|
||||
Prism conducting a scenario:
|
||||
|
||||
```yaml
|
||||
version: 2
|
||||
jobs:
|
||||
build:
|
||||
docker:
|
||||
- image: your-normal-image:your-version
|
||||
steps:
|
||||
- checkout
|
||||
- run:
|
||||
name: Install prism
|
||||
command: curl https://raw.githubusercontent.com/pytlesk4/stoplight-todos/master/prism.sh | sh
|
||||
- run:
|
||||
name: Start your service
|
||||
command: ...
|
||||
background: yes
|
||||
- run:
|
||||
name: Run scenario
|
||||
command: prism conduct ...
|
||||
...
|
||||
```
|
||||
|
||||
When running `prism conduct` you can:
|
||||
|
||||
* Include the Scenario JSON on your CI server, and pass in its absolute file path
|
||||
* Pass in the absolute URL to the scenario JSON served up via HTTP
|
||||
|
||||
<!-- theme: warning -->
|
||||
|
||||
> Don't forget to pass in any required environment values with the --env command
|
||||
> line flag (or you can provide the filepath to a json file with your environment
|
||||
> variables)!
|
||||
|
||||
<!-- theme: info -->
|
||||
|
||||
> Did you know? You can find the full command to run your scenario collection
|
||||
> or individual scenarios in the Stoplight application. Click on the "Home"
|
||||
> button of a scenario under "Trigger This Collection".
|
||||
|
||||
---
|
||||
|
||||
**Related**
|
||||
|
||||
* [Continuous Integration Overview](./continuous-integration.md)
|
||||
* [Continuous Integration with Travis CI](./continous-integration-travis)
|
||||
* [Continuous Integration with Jenkins](./continous-integration-jenkins)
|
||||
* [Prism Docker Image](https://hub.docker.com/r/stoplight/prism/)
|
||||
92
articles/testing/continuous-integration-jenkins.md
Normal file
92
articles/testing/continuous-integration-jenkins.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# Continuous Integration with Jenkins
|
||||
|
||||
Integrating Prism into your [Jenkins](https://jenkins.io/) pipeline is easy.
|
||||
There are two ways to get started with Jenkins:
|
||||
|
||||
* Running Prism in a pipeline step natively or using Docker
|
||||
* Using the "Stoplight Report" Jenkins Plugin
|
||||
|
||||
## Adding Prism to a Pipeline
|
||||
|
||||
To get started, you will need to either install Prism natively on the Jenkins
|
||||
system, or use the official [Stoplight Prism Docker image](https://hub.docker.com/r/stoplight/prism/).
|
||||
|
||||
When integrating Prism into a Jenkins pipeline, there are two different
|
||||
approaches:
|
||||
|
||||
* Starting Prism in the background to act as either a mock or
|
||||
contract/validation server
|
||||
* Having Prism conduct a scenario against a running server by running as a
|
||||
dedicated test step
|
||||
|
||||
### Running Prism in the Background
|
||||
|
||||
To run Prism in the background, you will need to start Prism prior to kicking
|
||||
off your tests. If running natively, use the command format:
|
||||
|
||||
```sh
|
||||
BUILD_ID=dontKillMe nohup prism mock --spec ... &
|
||||
```
|
||||
|
||||
<!-- theme: warning -->
|
||||
|
||||
> Please note that the trailing ampersand (`&`) is required
|
||||
|
||||
If running in Docker, use the format:
|
||||
|
||||
```sh
|
||||
docker run -d --rm -p 4010:4010 stoplight/prism:latest mock --spec ...
|
||||
```
|
||||
|
||||
For more information on using Prism as a test server, see [here](./overview.md).
|
||||
|
||||
### Running Prism in the Foreground
|
||||
|
||||
To run Prism in the foreground, you will call Prism like you would any other
|
||||
test step. If running natively, use the command format:
|
||||
|
||||
```sh
|
||||
prism conduct ...
|
||||
```
|
||||
|
||||
If running in Docker, use the format:
|
||||
|
||||
```sh
|
||||
docker run --rm stoplight/prism:latest prism conduct ...
|
||||
```
|
||||
|
||||
When running `prism conduct`, you can:
|
||||
|
||||
* Include the Scenario JSON on your CI server, and pass in its absolute file path
|
||||
* Pass in the absolute URL to the scenario JSON served up via HTTP
|
||||
|
||||
<!-- theme: warning -->
|
||||
|
||||
> Don't forget to pass in any required environment values with the --env command
|
||||
> line flag (or you can provide the filepath to a json file with your environment
|
||||
> variables)!
|
||||
|
||||
<!-- theme: info -->
|
||||
|
||||
> Did you know? You can find the full command to run your scenario collection
|
||||
> or individual scenarios in the Stoplight application. Click on the "Home"
|
||||
> button of a scenario under "Trigger This Collection".
|
||||
|
||||
## Using the Plugin
|
||||
|
||||
Members of the Stoplight community were kind enough to create the [Stoplight
|
||||
Report Plugin](https://github.com/jenkinsci/stoplightio-report-plugin), which is
|
||||
a Jenkins plugin that can be used to run tests with Prism. For more information
|
||||
on the plugin, see [here](https://plugins.jenkins.io/stoplightio-report).
|
||||
|
||||
---
|
||||
|
||||
**Related**
|
||||
|
||||
* [Jenkins Website](https://jenkins.io/)
|
||||
* [Continuous Integration Overview](./continuous-integration.md)
|
||||
* [Continuous Integration with Circle CI](./continous-integration-circle)
|
||||
* [Continuous Integration with Travis CI](./continous-integration-travis)
|
||||
* [Prism Docker Image](https://hub.docker.com/r/stoplight/prism/)
|
||||
* [Stoplight Report Plugin Homepage](https://plugins.jenkins.io/stoplightio-report)
|
||||
* [Stoplight Report Plugin Github](https://github.com/jenkinsci/stoplightio-report-plugin)
|
||||
@@ -1 +1,71 @@
|
||||
# Continuous Integration with Travis CI
|
||||
|
||||
Integrating Prism into your Travis CI pipeline is easy. The simplest way to get
|
||||
up and running is by using [Stoplight's Prism Docker
|
||||
image](https://hub.docker.com/r/stoplight/prism/).
|
||||
|
||||
To get started, you will need to make sure the `docker` service is listed under
|
||||
the `services` section of your `.travis.yml` file.
|
||||
|
||||
When integrating Prism into a Travis CI pipeline, there are two different
|
||||
approaches:
|
||||
|
||||
* Starting Prism in the background to act as either a mock or
|
||||
contract/validation server
|
||||
* Having Prism conduct a scenario against a running server by running as a
|
||||
dedicated test step (i.e. added to the `script` section)
|
||||
|
||||
## Running Prism in the Background
|
||||
|
||||
Below is a sample Travis CI configuration file using Prism as a mock or
|
||||
contract/validation server:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
- docker
|
||||
|
||||
before_install:
|
||||
# start the mock server in the background
|
||||
- docker run -d -p 127.0.0.1:4010:4010 stoplight/prism mock ...
|
||||
```
|
||||
|
||||
Once the Prism container is started, it will automatically start listening on
|
||||
`http://localhost:4010` for any open connections.
|
||||
|
||||
## Running Prism in the Foreground
|
||||
|
||||
Below is a sample Travis CI configuration file with Prism conducting a scenario:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
- docker
|
||||
|
||||
script:
|
||||
- docker run --rm -it stoplight/prism conduct ...
|
||||
```
|
||||
|
||||
When running `prism conduct` you can:
|
||||
|
||||
* Include the Scenario JSON on your CI server, and pass in its absolute file path
|
||||
* Pass in the absolute URL to the scenario JSON served up via HTTP
|
||||
|
||||
<!-- theme: warning -->
|
||||
|
||||
> Don't forget to pass in any required environment values with the --env command
|
||||
> line flag (or you can provide the filepath to a json file with your environment
|
||||
> variables)!
|
||||
|
||||
<!-- theme: info -->
|
||||
|
||||
> Did you know? You can find the full command to run your scenario collection
|
||||
> or individual scenarios in the Stoplight application. Click on the "Home"
|
||||
> button of a scenario under "Trigger This Collection".
|
||||
|
||||
---
|
||||
|
||||
**Related**
|
||||
|
||||
* [Continuous Integration Overview](./continuous-integration.md)
|
||||
* [Continuous Integration with Circle CI](./continous-integration-circle)
|
||||
* [Continuous Integration with Jenkins](./continous-integration-jenkins)
|
||||
* [Prism Docker Image](https://hub.docker.com/r/stoplight/prism/)
|
||||
|
||||
@@ -1 +1,92 @@
|
||||
# Continuous Integration
|
||||
|
||||
Continuous Integration (CI) is the practice of continuously merging developer
|
||||
work into a shared repository. By merging work frequently, developers can easily
|
||||
detect conflicts and stay up-to-date with other team members' changes. Since CI
|
||||
allows teams to maintain a high development velocity, testing becomes an even
|
||||
more crucial component to the development process. By constantly testing all
|
||||
changes, developers can verify that they are not breaking existing functionality
|
||||
or introducing new bugs.
|
||||
|
||||
By integrating Prism into your CI process, your development team can be certain
|
||||
that any changes made to a project do not violate an OpenAPI specification. This
|
||||
allows multiple teams with agreed-upon specifications to work independently of
|
||||
one another, while ensuring that any changes meet the specification throughout
|
||||
the lifetime of the project.
|
||||
|
||||
There are a few different approaches available when integrating Prism into a CI
|
||||
testing pipeline:
|
||||
testing with Stoplight [Scenarios](./scenarios-introduction.md),
|
||||
testing with a mock server, and
|
||||
contract testing.
|
||||
|
||||
## Testing with Scenarios
|
||||
|
||||
Stoplight [Scenarios](./scenarios-introduction.md) allows users to define
|
||||
multiple steps for testing an OpenAPI specification. Scenarios provide a similar
|
||||
function to API's as [functional test
|
||||
cases](https://en.wikipedia.org/wiki/Functional_testing) do in software
|
||||
development. Scenarios run (or "conducted") by Prism can be easily integrated
|
||||
into a CI pipeline for validating a server's implementation of a specification.
|
||||
|
||||
To run a scenario with Prism, use the `conduct` command:
|
||||
|
||||
```bash
|
||||
prism conduct my-scenario.json --spec open-api-spec.json -e "myapikey=abc123"
|
||||
```
|
||||
|
||||
For more information on Scenarios, please see [here](./scenarios-introduction.md).
|
||||
|
||||
## Testing with a Mock Server
|
||||
|
||||
Prism's mock server allows Prism to behave like a
|
||||
fully-implemented server for a provided OpenAPI specification. This form of
|
||||
testing is useful for:
|
||||
|
||||
* **Speeding Up Development Time**
|
||||
* Tests can be run locally with nothing more
|
||||
than an API specification. This allows a development team on the client or
|
||||
server side to work independently of one another.
|
||||
* **Validation Testing**
|
||||
* Ensures that a client implementation meets all of the requirements set by an
|
||||
OpenAPI specification.
|
||||
|
||||
To start a mock server with Prism, use the `mock` command:
|
||||
|
||||
```bash
|
||||
prism mock --spec open-api-spec.json
|
||||
```
|
||||
|
||||
For more information on mock testing with Prism, see [Mocking Introduction](FIXME).
|
||||
|
||||
## Contract Testing
|
||||
|
||||
Contract testing is when Prism acts as a proxy between the client and a target
|
||||
upstream server. When Prism receives a request, it forwards it to the upstream
|
||||
server and validates that the server response conforms to the provided OpenAPI
|
||||
specification. This form of testing is useful for:
|
||||
|
||||
* Verifying a _server_ implementation to ensure all _responses_ meet the
|
||||
appropriate requirements set by an OpenAPI specification
|
||||
* Verifying a _client_ implementation to ensure all _requests_ meet the
|
||||
appropriate requirements set by an OpenAPI specification
|
||||
* Augmenting existing test suites with very little extra work and almost no
|
||||
workflow changes required
|
||||
|
||||
To start a contract/validation server with Prism, use the `validate` command:
|
||||
|
||||
```bash
|
||||
prism validate --spec open-api-spec.json --upstream http://localhost:8080
|
||||
```
|
||||
|
||||
For more information on contract testing with Prism, see [Mocking Introduction](FIXME).
|
||||
|
||||
---
|
||||
|
||||
**Related**
|
||||
|
||||
* [Testing Overview](./overview.md)
|
||||
* [Scenarios Introduction](./scenarios-introduction.md)
|
||||
* [Continuous Integration with Circle CI](./continous-integration-circle.md)
|
||||
* [Continuous Integration with Travis CI](./continous-integration-travis.md)
|
||||
* [Continuous Integration with Jenkins](./continous-integration-jenkins.md)
|
||||
|
||||
Reference in New Issue
Block a user