README.md

# PSO-testing

## Tables of contents
- [About PSO-testing](#about-pso-testing)
- [Technologies recommended](#technologies-recommended)
- [Playwright](#playwright)
- [Structure](#structure)
- [Writing tests](#writing-tests)
  - [Keywords](#keywords)
  - [Background](#background)
  - [Steps tense](#steps-tense)
  - [Scenario outline](#scenario-outline)
- [How to Run tests](#how-to-run-tests)
- [How to generate html report](#how-to-generate-html-report)
- [How to add properties file](#How-to-add-properties-file)
- [Conventions](#conventions)
- [CI Pipelines](#ci-pipelines)
  - [Pipeline Stages](#pipeline-stages)
  - [Merge Requests Pipeline](#merge-requests-pipeline)
    - [Recommended Branch Naming Conventions](#recommended-branch-naming-conventions)
  - [Scheduled Pipeline](#scheduled-pipeline)

## About PSO-testing
The PSO-testing is a automated test framework designed to allow automate test cases about Commission European projects.

## Technologies recommended
- Java 11
- Maven 3.9.6
- IntelliJ Community
- maven-atf-reports:1.0.0 (request to testing tower)

## Playwright

Playwright (https://playwright.dev/) is a framework used to automate tests on web pages. It interacts with page elements (named locators) to check their behavior.
Playwright uses core version of the browser (chromium, webkit and stable version of Firefox). It allows testing on stable browser versions which are not polluted by branding content and which are the most used. The version of those browsers will be the same for all people using the test project.
In addition, it can also run tests on the local chrome and edge version.

## Structure
**Configuration file** is split in data and locators (src/main/java/configuration):
- locators: all locators should be defined here. Each ui page should have its own java class. 
- data: all test data should be defined here. Each ui page should have its own java class.

**Helpers file** for all helpers that supports our stepDefinitions (src/main/java).

**Features file** where should be defined all scenario tests. Use scenario outline if needs but avoid define test data here (src/main/java).

**Framework file** where should be defined all code to allow run the tests (src/test/java).

**StepDefinitions** folder where should be defined all cucumber steps (src/test/java).

**TestRunner** folder where should be defined the cucumber test runner (src/test/java).

**resources** should contain config.properties file with sensitive data:

```
//SIMPL-LABS
URL=https://poc-simpl-labs.dev.simpl-europe.eu/data-spaces
USERNAME=
PASSWORD=
SECRET_KEY=

//SIMPL-OPEN
URL_WIZARD=https://sd-creation-wizard.uatpso.com

//COMMON
IS_HEADLESS=false
```

## Writing tests

Writing test cases in a correct way requires you to respect some easy rules. Our test cases are written with **Gherkin** syntax (https://cucumber.io/docs/gherkin/reference/).

All scenario and steps name should start with a lowercase letter.

### Keywords

Gherkin as three main keywords for steps:

- **Given** for prerequisites steps
- **When** for actions steps
- **Then** for expectations steps

Keywords shouldn't be mixed: You should have this order of keywords: Given, When, Then.
Note that a test couldn't have any **Given** steps if there is no prerequisites and in some rare cases it could happen that a test has no **When** steps.

E.g.:

```gherkin
Scenario: a user logs in
  Given a user opened the login form
  When the user fills the login form
  And the user validates the login form
  Then the user name should be displayed
```

### Background

If all tests of your feature share some common steps, you can put them in a background.
All background steps should be of **Given** type.

E.g.:

```gherkin
Background:
  Given the website is displayed
  And the user accepted the cookies

Scenario: a user logs in
  Given a user opened the login form
  When the user fills the login form
  And the user validates the login form
  Then the user name should be displayed
  But the login button shouldn't be displayed
```

### Steps tense

Each type of step has its own tense rule.

- **Given** steps should be written in **past** tense most of the time as they represent prerequisites actions that have to be donne for the test's main action to be executed. But sometimes when the step is written in a passive way, given steps could be written in **present** tense
- **When** steps have to be written in **present** tense as they represent the main actions that will be performed in the test
- **Then** steps have to be written in **conditional** tense as the represent an expected behavior that could be not right when tested

If you have a step that is defined in a certain type and could be used as another type in another test, the step should be duplicated and written with the type format.

E.g.:

```gherkin
Scenario: a user logs in
  Given a user opened the login form
  When the user fills the login form
  And the user validates the login form
  Then the user should be logged in

Scenario: a user logs out
  Given a user is logged in
  When the user clicks the log out button
  Then the user shouldn't be logged in
```

### Scenario outline

In some cases, a scenario can be used for several test cases with only some variations. In such a situation you can use a Scenario outline which will allow you to execute tests with parameters. To do that, you'll have to use create a **Scenario Outline** instead of a **Scenario** and define your parameters into your steps by putting them between **<>** (parameters should be written in pascal case).
Then you'll have the define the parameters combinations in the **Examples** section.

E.g.:

```gherkin
Scenario Outline: a user <userStatus> logs in
  Given a user opened the login form
  When the user fills the login form
  And the user validates the login form
  Then the user <loginResult> be logged in

  Examples:
  | userStatus | loginResult |
  | valid      | should      |
  | invalid    | shouldn't   |
```


### How to run tests
## ATTENTION: BEFORE RUN ANY TEST CONFIGURE PROPERTIES FILE

Run the following command

To simplOpen project

```bash
// to api testing
mvn clean test -Dtest=ApiSimplOpenRunner

// to ui testing
mvn clean test -Dtest=UiSimplOpenRunner
```
To simplLabs project
```bash
// to api testing
mvn clean test -Dtest=UiSimplLabsRunner
```
To run a test by tag it just needed add the following option to the previous command (consider the tag @TCW01 as an example):
```bash
-Dcucumber.filter.tags="@TCW01"
```

### How to generate html report
To generate the report of the first time it´s mandatory install the dependency maven-atf-reports.
It´s not available in the internet. Talk with testing tower to get it

Run the following command
```bash
mvn com.qa.atf.reports:maven-atf-reports:1.0.0:generate@execution
```
After run the command with success, the html report will be available in target/pluma-report/overview-features.html

To add some information to report type the following code to the cucumber step:
```bash
 scenario.attach(<information to add>, "text/plain", <title>);
```

To add a screenshot to report type the following code to the cucumber step:
```bash
byte[] screenshot = page.screenshot(new Page.ScreenshotOptions());
scenario.attach(screenshot, "image/png", <title>);
```

### How to add properties file
- Go to directory src/main/resources
- Add a file with name config.properties
- Add the information that is needed (see the chapter [Structure](#structure))

### Conventions
- cucumber tags: each feature file should be a tag with feature name and each test should be tag with test case number and initial letter of feature name (eg. L01 - first scenario to login.feature).
- write typing:
  - java class - pascal case
  - data and locators - snake case + upper case
  - feature files and methods name - camel case

## CI Pipelines

### Pipeline Stages
The following stages are included in the CI Pipeline for PSO Test Automation Repo:
- **quality** --> for running SonarQube verifications for the repo (TODO)
- **test** --> for running automation tests during Merge Requests and Scheduled Pipeline executions 
- **report** --> for generating reports based on the test results in the previous stage

### Merge Requests Pipeline
- **test** followed by **report** stages of the defined pipeline will be executed during Merge Request events
- It has been agreed that for every merge request, a subset of test cases (marked with tag '**@smoke**') will be executed
- TestRunner and Tags to be used to run the Automation tests will be derived from the branch name
  - `xvfb-run mvn clean test -Dtest=<TESTRUNNER_TO_USE> -Dcucumber.filter.tags="<TAGS_TO_USE>"`

### Recommended Branch Naming Conventions
##### **For Features (test cases):**
The following branch naming convention is recommended when creating or updating test cases:

> feature/`<TestRunner_Identifier>`-`<JIRA_TestCaseId>`
- `<TestRunner_Identifier>` could be any of the following values: 
  - `SOUI` - Implies feature is related to SimplOpen UI &#8594; SimplOpen UI TestRunner is to be used for test execution
  - `SOAPI` - Implies feature is related to SimplOpen API &#8594; SimplOpen API TestRunner is to be used for test execution
  - `SLUI` - Implies feature is related to SimplLabs UI &#8594; SimplLabs UI TestRunner is to be used for test execution
  - `SLAPI` - Implies feature is related to SimplLabs API &#8594; SimplLabs API TestRunner is to be used for test execution
- `<JIRA_TestCaseId>` is the test case identifier in JIRA. `e.g. SIMPL-1234`

> NOTE:
> - Remember to tag the newly created or updated test case with the JIRA id. `e.g. @SIMPL-1234`
> - If the TestRunner_Identifier is not provided in the branch name, SimplOpen UI TestRunner will be used as default
> - If the JIRA_TestCaseId is not provided in the branch name, "@smoke" tag will be used as default

**Illustrations:**
> 1. **feature/SOUI-SIMPL-111** --> For MRs, Tests will be executed with SimplOpen UI TestRunner and Tags "@smoke or @SIMPL-111".
>
>   - Following command will be executed in pipeline: 
>   - `xvfb-run mvn clean test -Dtest=UiSimplOpenRunner -Dcucumber.filter.tags="@smoke or @SIMPL-111"`
> 2. **feature/SOAPI-SIMPL-222** --> For MRs, Tests will be executed with SimplOpen API TestRunner and Tags "@smoke or @SIMPL-222".
>
>   - Following command will be executed in pipeline:
>   - `xvfb-run mvn clean test -Dtest=ApiSimplOpenRunner -Dcucumber.filter.tags="@smoke or @SIMPL-222"`
> 3. **feature/SLUI-SIMPL-333** --> For MRs, Tests will be executed with SimplLabs UI TestRunner and Tags "@smoke or @SIMPL-333".
>
>   - Following command will be executed in pipeline:
>   - `xvfb-run mvn clean test -Dtest=UiSimplLabsRunner -Dcucumber.filter.tags="@smoke or @SIMPL-333"`
> 4. **feature/SLAPI-SIMPL-444** --> For MRs, Tests will be executed with SimplLabs API TestRunner and Tags "@smoke or @SIMPL-444".
>
>   - Following command will be executed in pipeline:
>   - `xvfb-run mvn clean test -Dtest=ApiSimplLabsRunner -Dcucumber.filter.tags="@smoke or @SIMPL-444"`
> 5. **feature/SOAPI-some-feature** --> For MRs, Tests will be executed with SimplOpen API TestRunner and default Tag "@smoke".
>
>   - Following command will be executed in pipeline:
>   - `xvfb-run mvn clean test -Dtest=ApiSimplOpenRunner -Dcucumber.filter.tags="@smoke"`
> 6. **feature/some-feature** --> For MRs, Tests will be executed with default SimplOpen UI TestRunner and default Tag "@smoke".
>
>   - Following command will be executed in pipeline:
>   - `xvfb-run mvn clean test -Dtest=UiSimplOpenRunner -Dcucumber.filter.tags="@smoke"`
> 7. **this-is-some-feature** --> For MRs, Tests will be executed with default SimplOpen UI TestRunner and default Tag "@smoke".
>
>   - Following command will be executed in pipeline:
>   - `xvfb-run mvn clean test -Dtest=UiSimplOpenRunner -Dcucumber.filter.tags="@smoke"`


##### **For Spikes:**
The following branch naming convention is recommended for spikes and changes not related to test cases:

> spike/`<short-description>`
- `<short-description>` is a text description about the spike
- During MRs for spike branches, Tests will be executed with default SimplOpen UI TestRunner and default Tag "@smoke".
- `xvfb-run mvn clean test -Dtest=UiSimplOpenRunner -Dcucumber.filter.tags="@smoke"`

### Scheduled Pipeline
- **test** followed by **report** stages of the defined pipeline will be executed during Scheduled Pipeline events
- It has been agreed that for every Scheduled Pipeline execution, regression test cases (marked with tag '**@regression**') will be executed
- TestRunner and Tags to be used to run the Automation tests have to be provided in the Scheduled Pipeline definition
- Any number of Scheduled Pipelines can be created as needed. e.g. Dedicated Daily Regression tests for different applications (SimplOpenUI, SimplOpenAPI, SimplLabsUI, SimplLabsAPI etc)
- Targets for Scheduled Pipelines can be the main 'develop' branch or any other branch
- Scheduled Pipelines can be created using the UI (necessary permissions needed): `pso-test-automation --> Builds --> Pipeline schedules--> New schedule`

**Example illustration:**
> - Setting up a daily regression for SimplOpen UI:
>   - Running daily at 19:00 UTC+2
>   - With target branch 'develop'
>   - Using TestRunner 'UiSimplOpenRunner' (via variable RUNNER_TO_USE)
>   - Matching tags '@regression or @wizard' (via variable TAGS_TO_USE)

![img.png](src/main/resources/sample_schedule_pipeline.png)