Compare commits
9 Commits
develop
...
articles/t
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5972806727 | ||
|
|
c805cd366b | ||
|
|
390665c014 | ||
|
|
0ab7dbbe1c | ||
|
|
88a7f677f2 | ||
|
|
dc1d63fe5d | ||
|
|
c0b0c1045c | ||
|
|
a57f88662c | ||
|
|
5e9b978b9e |
30
articles/testing/introduction.md
Normal file
30
articles/testing/introduction.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# Introduction
|
||||
|
||||
As APIs continue to spread throughout orginzations it is more important than ever to develop a highly flexible, performant and powerful testing roadmap. We all know why testing is important you can catch bugs faster, iterate without breaking existing features faster, and ultimately deliver a higher quality product to the end user. But lets be honest for a second. Even though we think testing is important we were too busy to actually write them. Also, maintaining a test suite is tedious, right? Lastly, we all write perfect code, no need to test it, plus 10k people thought that Stackoverflow answer was useful. Very compelling arguments if APIs weren't everywhere, otherwise those arguements are the reason why people aren't using your API.
|
||||
|
||||
API testing isn't about you actually and you know this deep down becuase there is nothing more frustrating than using an API that doesn't work as advertised. Your customers will appreciate a well tested service, but customers aren't the only people who will be using your API anymore. Team members are now users and they might be distrbuted in a different time zone, having an awesome test suite means that they can quickly know if a service is working as intended. There is nothing more frustrating than looking for some obscure bug in code you didn't write and in a language you don't know.
|
||||
|
||||
API testing also isn't just about catching bugs and iterating faster. A through test suite is actually really great documentation. Not only are test a great way to understand how an API works under certain Scenarios ;), they can also be a great way to drive your actual implementation if you create them first. Test/Behavoir-driven development(TDD/BDD) forces you to think about design/requirements before actually writting any code.
|
||||
|
||||
Today microservices and serverless make easier than ever to iterate quickly which means it too easy to introduce bugs and the technical debt quickly becomes unmanageble without the proper testing solution. Teams are increasingly becoming smaller and spread out. It is no longer feasible to use slack and email to ask the most basic questions about a service. A comprehensive test suite documents how a service is used at a high level.
|
||||
|
||||
## Stoplight Scenarios
|
||||
|
||||
We engineered Scenarios from the ground up to be:
|
||||
|
||||
* **Powerful** Easily assert, capture, transform, and validate your API Spec (OpenAPI) against your actual API. And if that isn’t enough, Prism has a powerful javascript runtime.
|
||||
|
||||
* **Portable** Scenarios are described in plain JSON with a well thought out, robust specification. Use our visual editor to quickly generate and manage this JSON. Scenarios can be run from our visual tooling, or completely outside of Stoplight, on your own machines, or on your continuous integration server.
|
||||
|
||||
* **Flexible** Your APIs, your tests, your way. Scenarios only test what you want them to. They have no opinion about your architecture (Monolithic vs Microservices), company structure (in house vs distributed), development lifecycle (production vs TDD), and your environment (development vs staging vs production).
|
||||
|
||||
* **Fast** Time can’t be slowed down, and we can’t give it back to you. Creating tests should be quick, and waiting for you tests to run shouldn’t feel like watching water boil. Scenarios are run concurrently for maximum speed - run hundreds of API requests and test assertions in seconds.
|
||||
|
||||
---
|
||||
|
||||
**Related**
|
||||
|
||||
* [Prism Introduction](../prism/overview.md)
|
||||
* [Leveraging Open API](levergage-openapi.md)
|
||||
* [Continuous Integration](continuous-integration.md)
|
||||
* [Run Scenarios In Stoplight](run-test-stoplight.md)
|
||||
@@ -1 +1,75 @@
|
||||
# Pass Data Between Steps
|
||||
|
||||
Most scenarios will involve more than one step, and oftentimes you will need to share data between steps.
|
||||
|
||||
For example, imagine a simple scenario that tests the CRUD operations on a "todos" model. It might involve 5 steps:
|
||||
|
||||
1. CREATE /todos
|
||||
2. GET /todos/{todoId}
|
||||
3. PUT /todos/{todoId}
|
||||
4. DELETE /todos/{todoId}
|
||||
5. GET /todos/{todoId} (to make sure 404 not found)
|
||||
|
||||
In order for this to work, steps 2-5 need to be able to access the `id` of the newly created todo in step 1. To do this, we'll save the new `id` property in step 1 to the $.ctx object.
|
||||
|
||||
### The $.ctx object
|
||||
|
||||
The $.ctx object makes it easy to store temporary data for use during the course of a single scenario run. By default, the $.ctx object will be empty at the start of every run. Use it to share data between steps.
|
||||
|
||||
### Using the $.ctx object
|
||||
|
||||
To use a ctx variable in your scenario, simply use the {$.ctx.variableName} syntax. For example, if your ctx variable is called `todoId`, and you want to use it in your step's URL, you would do something like this:
|
||||
|
||||
`/todos/{$.ctx.todoId}`
|
||||
|
||||
When the step is run, `{$.ctx.todoId}` will be replaced by whatever $.ctx.todoId has been set to in a previous step.
|
||||
|
||||
You can use variables anywhere in your step, including url, auth, headers, request body, assertions, captures, and scripting.
|
||||
|
||||
To use environments in a before script, after script, or step with type script:
|
||||
|
||||
```js
|
||||
// would print the postId
|
||||
console.log($.ctx.get('todoId');
|
||||
```
|
||||
|
||||
In the screenshot below, we are using a `todoId` ctx variable that is set in a previous step. It is helping us create the URL, and is also used in a test assertion to make sure that the API request responds with the exact id we are requesting.
|
||||
|
||||

|
||||
|
||||
### Setting ctx variables
|
||||
|
||||
#### With Captures
|
||||
|
||||
Save values from your step to your ctx object, for use in subsequent steps.
|
||||
|
||||
To accomplish this, open up the Captures tab of one of your steps and select $.ctx on the lefthand side of the assignment. In the image below we are saving the todoId property of the Create Todo response back to a variable on our ctx called `todoId`. We will use this variable in subsequent steps.
|
||||
|
||||

|
||||
|
||||
#### With Scripting
|
||||
|
||||
When you need more complicated conditional logic or processing, scripting is the way to go. Scripts are plain javascript, and give you access to the $.ctx object. You can achieve the same results as the capture screenshot above as seen in the screenshot below.
|
||||
|
||||

|
||||
|
||||
### When to use ctx variables
|
||||
|
||||
You can think of ctx variables as a temporary state in your scenario. In general, we recommend using them for values that are created during the course of a single scenario that need to be shared with subsequent steps. This includes things like ids, usernames, and randomly generated tokens.
|
||||
|
||||
<!-- theme: warning -->
|
||||
> We do **NOT** recommend using ctx variables for more permanent values that need to persist across scenario runs. For this persisted value use case, we recommend using the $$.env object, which is explained in detail [here](variables-environment.md).
|
||||
|
||||
### Debugging ctx variables
|
||||
|
||||
$.ctx variables can be manually set when running a step. This is useful if you are trying to debug a single step in the middle of a scenario that relies on a ctx value set in a previous step. Instead of running the entire scenario to generate the needed ctx value, you can set it in the UI manually. In the screenshot below, we can set the todoId ctx variable before running the step to debug what would happen with a particular todoId value.
|
||||
|
||||

|
||||
|
||||
---
|
||||
|
||||
**Related**
|
||||
|
||||
* [Variables Overview](variables-overview.md)
|
||||
* [Context Variables](variables-context.md)
|
||||
* [Variables Environment](variables-environment.md)
|
||||
|
||||
@@ -1,17 +0,0 @@
|
||||
Stoplight Scenarios is a powerful (but accessible!) tool that takes the pain out of API testing. It is a standalone product, available on [the web](https://scenarios.stoplight.io), and as a [desktop app](https://download-next.stoplight.io).
|
||||
|
||||
We generally recommend the desktop app when possible. It works with local servers, behind firewalls, and exchanges information with tools on your computer like Git or your favorite IDE. You can switch seamlessly between the desktop app and the web app.
|
||||
|
||||
We engineered Scenarios from the ground up to be:
|
||||
|
||||
- **Powerful** Easily assert, capture, transform, and validate your API Spec (Swagger) against your actual API. And if that isn’t enough, Prism has a powerful javascript runtime.
|
||||
|
||||
- **Portable** Scenarios are described in plain JSON with a well thought out, robust specification. Use our visual editor to quickly generate and manage this JSON. They can be run from our visual tooling, or completely outside of Stoplight, on your own machines or on your continuous integration server.
|
||||
|
||||
- **Flexible** Your APIs, your tests, your way. Scenarios only test what you want them to. They have no opinion about your architecture (Monolithic vs Microservices), company structure (in house vs distributed), development lifecycle (production vs TDD), and your environment (development vs staging vs production).
|
||||
|
||||
- **Fast** Time can’t be slowed down, and we can’t give it back to you. Creating tests should be quick, and waiting for you tests to run shouldn’t feel like watching water boil. Scenarios are run concurrently for maximum speed - run hundreds of API requests and test assertions in seconds.
|
||||
|
||||
### Editor UI Overview
|
||||
|
||||

|
||||
BIN
assets/images/$.ctx.todoId-Assertion.png
Normal file
BIN
assets/images/$.ctx.todoId-Assertion.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 219 KiB |
BIN
assets/images/$.ctx.todoId-Capture.png
Normal file
BIN
assets/images/$.ctx.todoId-Capture.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 202 KiB |
BIN
assets/images/$.ctx.todoId-Manual.png
Normal file
BIN
assets/images/$.ctx.todoId-Manual.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 157 KiB |
BIN
assets/images/$.ctx.todoId-Scripting.png
Normal file
BIN
assets/images/$.ctx.todoId-Scripting.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 233 KiB |
BIN
assets/images/scenarios-editor-overview.png
Normal file
BIN
assets/images/scenarios-editor-overview.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 443 KiB |
Reference in New Issue
Block a user