Aqua 2024.2 Help

Pytest

Aqua supports pytest, a fully functional testing framework.

The following features are available:

  • The dedicated test runner.

  • Code completion for test subject and pytest fixtures.

  • Code navigation.

  • Detailed failing assert reports.

  • Support for Python 2.7 and Python 3.5 and later.

  • Multiprocessing test execution.

By default, the suggested default test runner is unittest. So, to utilize pytest, you need to make it the default test runner first.

Install Pytest

  1. Go to the Python Package tool window

  2. Start typing pytest in the Search field of the Python Package tool window. Locate the package in the list of the available packages and click Install.

    Installing Pytest

Once the pytest package is installed, Aqua detects it and makes it the default project test runner. At any time you can change a test runner in the project settings.

Now, that pytest is set as the default testing framework, you can create a small test for the Car.py sample. Let's create a pytest test to check the brake function.

Create a test

  1. Create a Python project.

  2. In the main menu, go to File | New, choose Python file, type car.py, and click OK.

  3. Copy and paste the Car class into the car.py file.

    class Car: def __init__(self, speed=0): self.speed = speed self.odometer = 0 self.time = 0 def say_state(self): print("I'm going {} kph!".format(self.speed)) def accelerate(self): self.speed += 5 def brake(self): self.speed -= 5 def step(self): self.odometer += self.speed self.time += 1 def average_speed(self): if self.time != 0: return self.odometer / self.time else: pass if __name__ == '__main__': my_car = Car() print("I'm a car!") while True: action = input("What should I do? [A]ccelerate, [B]rake, " "show [O]dometer, or show average [S]peed?").upper() if action not in "ABOS" or len(action) != 1: print("I don't know how to do that") continue if action == 'A': my_car.accelerate() elif action == 'B': my_car.brake() elif action == 'O': print("The car has driven {} kilometers".format(my_car.odometer)) elif action == 'S': print("The car's average speed was {} kph".format(my_car.average_speed())) my_car.step() my_car.say_state()
  4. In the editor, place the caret at the brake method declaration.

  5. Do one of the following:

    • Go to Navigate | Test.

    • From the context menu, choose Go To | Test.

      Go to test
    • Press Ctrl+Shift+T.

    Aqua shows the list of available tests.

    New test for pytest
  6. Click Create new test.

    The Create Test dialog opens.

    Create a python test for the brake method

    In the Create Test dialog, specify the following settings:

    • Target directory, where the new test class will be generated.

    • The name of the test file (in our example, test_car.py), and the name of the test class if needed.

    • Select the checkboxes next to the methods you want to include in the test class.

      Note that if you place the caret within a method, only this method name is included in the list. Also, mind the naming convention: the test method has the test prefix. The Run Test icon will appear in the editor gutter for all methods with such a prefix.

  7. Click OK when ready. Aqua generates the test file in the specified location.

  8. Aqua automatically generates a test file with the test method template. Replace the template code with the code that sets the initial speed value of Car to 50 and checks if speed gets properly set to 45 after the brake() function execution.

    from car import Car def test_car_brake(): car = Car(50) car.brake() assert car.speed == 45

Note that Aqua recognizes the test subject and offers completion for the Car class' instance.

Autocompletion for the test subject

Although Go To Test Subject and Go To Test commands of the context menu are not supported for pytest, you can navigate to the tested code in car.py by using the Go To Declaration command.

Run a test

  1. Click Run Test to run the test:

    Run the test
  2. Note that Aqua automatically creates a pytest Run/Debug configuration:

    Suggested run/debug configuration for pytest

    Select Run pytest for test_car_pytest to execute the test.

  3. Inspect test results:

    Pytest run
  4. Alter the assert statement to the following: assert my_car.speed == 4599.

  5. Rerun the test to evaluate the assert failing report:

    Assert failing report

    Note that pytest provides an explicit report on the failure.

With pytest fixtures you can create small test units that can be reused across the testing module. All you need is to mark a reusable unit with @pytest.fixture.

Use fixtures

  1. Modify your pytest test as follows:

    import pytest from car import Car @pytest.fixture def my_car(): return Car(50) def test_car_accelerate(my_car): my_car.accelerate() assert my_car.speed == 55 def test_car_brake(my_car): my_car.brake() assert my_car.speed == 45

    my_car() is a fixture function that creates a Car instance with the speed value equal to 50. It is used in test_car_accelerate and test_car_brake to verify correct execution of the corresponding functions in the Car class.

    Note that the my_car fixture is added to the code completion list along with other standard pytest fixtures, such as tempdir.

  2. Click either of the Run Test icons, or run the entire test module.

You can enable sharing fixture instances across tests using the scope parameter of the fixture function. For more information about pytest fixtures, refer to pytest fixtures documentation.

You might want to run your tests on the predefined set of data. Aqua supports test parametrization implemented in pytest through @pytest.mark.parametrize.

Apply parametrization

  1. let's create a set of speed values to test car.accelerate and car.brake functions: speed_data = {45, 50, 55, 100}

  2. Modify the test code to the following:

    import pytest from car import Car speed_data = {45, 50, 55, 100} @pytest.mark.parametrize("speed_brake", speed_data) def test_car_brake(speed_brake): car = Car(50) car.brake() assert car.speed == speed_brake @pytest.mark.parametrize("speed_accelerate", speed_data) def test_car_accelerate(speed_accelerate): car = Car(50) car.accelerate() assert car.speed == speed_accelerate

    Note that Aqua detects the newly created parameters and adds them to the completion list.

                     Code completion for pytest.mark.parametrize
  3. Run the test for the car.brake() function. You should expect the following test report:

    Three tests failed and one test passed for the car.brake function

You can also run the test for car.accelerate() function to ensure that it fails for all speed values but 55. For more information about parametrized test, refer to pytest documentation.

You can use Behavior-Driven Development (BDD) through pytest_bdd. This is particularly helpful when you need to quickly record your test using the Gherkin language and utilize beneficial features of pytest, such as fixture.

Implement test scenarios

  1. let's modify the Car class to validate the car's speed. Add the following method:

    def speed_validate(self): return self.speed <= 160
  2. Next, enable pytest-bdd in your project. In the Settings dialog (Ctrl+Alt+S) , navigate to Languages & Frameworks | BDD, and from the Preferred BDD framework list select pytest-bdd.

  3. Create a .feature file to record BDD scenarios using the Gherkin language. Right-click the project root and select New | Gherkin feature file. In the opened dialog, specify car.feature as the filename and click OK.

  4. Add the following scenarios to the car.feature file:

    Feature: Speed Scenario: Valid speed Given Speed is less than 160 When Accelerated Then Speed is valid Scenario: Invalid speed Given Speed is more than 160 When Accelerated Then Speed is invalid

    Both scenarios validate the speed of the car. The speed is supposed to be valid when it does not exceed the value of 160. Note the scenario steps are highlighted because they are not defined by this moment.

    bdd scenarios
  5. Aqua enables quick generation of step definitions for the entire feature file. Place the caret at any of the highlighted steps, press Alt+Enter, and select Create all step definitions. In the opened dialog, specify the name of the test file (it should start with test), select Python (pytest-bdd) from the File type list, and, if needed, modify the default file location.

    step definitions generation

    Inspect the test_car_bdd.py file. It contains all required import statements and definitions for each scenario step.

  6. let's modify the test file to use a pytest fixture for a Car object and to add more test logic:

    from pytest_bdd import scenario, given, when, then import pytest from car import Car @pytest.fixture def my_car(): return Car() @scenario('car.feature', 'Valid speed') def test_speed_valid(): pass @scenario('car.feature', 'Invalid speed') def test_speed_invalid(): pass @given("Speed is less than 160") def set_valid_speed(my_car): my_car.speed = 50 @given("Speed is more than 160") def set_invalid_speed(my_car): my_car.speed = 100 @when("Accelerated") def car_accelerate(my_car): my_car.accelerate() @then("Speed is valid") def success(my_car): assert my_car.speed_validate() @then("Speed is invalid") def fail(my_car): assert not my_car.speed_validate()
  7. Run the test by creating the corresponding Run/Debug configuration. You can also run either of the scenarios using the Run icon Run icon in the gutter.

  8. Inspect the test run results.

    pytest-bdd results

    In our example, we have the test_speed_valid test passed and the test_speed_invalid test failed.

Last modified: 08 April 2024