DokuWiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
hierarchy typically looks like:
  * **Unit testing (atomic level)** - individual functions/methods. (Module testing often gets lumped into either "unit testing" (for simple modules) or "component testing" (for complex modules))
  * Component testing - groups of related units working together
  * **Integration testing** - multiple components interactingfPYATS
  * System testing - entire application

----


====PYATS===
  * {{ :automation:pyats_and_network_validation.pdf |}}
  * {{ :automation:pyats_and_network_validation_examples.pdf |}}
  * {{ :automation:pyats_oreilly-danw.pdf | OREILLY-PRES}} / [[https://github.com/dannywade/oreilly-live-training-pyats]]

Topics
  * 'Testbed' - Describes the physical devices and connections in a YAML file. Is an inventory file but augmented. You can have interfaces and topology with links and also credentials. Unicon is a library underneath. Used to control device connectivity (~paramiko)
  * TestScripts: Contains the logic of all the tests.
    * Jobs: executes TestScripts as tasks. Also allows to execute multiple TestScripts in parallel

  * pyATS library (Genie): Data PARSING. Tooling used to extract the network data.
    * Genie Harness: Used to build **test cases using YAML datafiles**

  * Easypy - Runtime
  * Others: Blitz, Clean, 'Health Check'; Robot
  * AEtest (Easy Testing): scaffolding/framework for the Testing


=== TESTSCRIPT ===
Is the python file that holds all your tests.

  * Common setup: Initial configuration and device initialization including loops - decorator : ''@aetest.setup''
  * Testcases: 'Container'(self-contained) for our tests. Setup/Test/Cleanup. testcase.uid (must be unique, is a python 'property') - decorator: ''@aetest.test''
  * Cleanup: Resets the testing environment after the test - decorator: ''@aetest.cleanup''

** see slides 46,47 for examples (classes..)) **
\\
Test Parameters: TestScript > Testcase > TestSection  (they are classes (eg: testcase). see code))

  * Callable 
  * Test Parameters - decorator : ''@aetest.parameter''
  * Test Execution: 
    * Standalone: all logging is sent to stdout. See slide 64-65
    * Easypy: Standardized runtime env within pyATS. Helpful for regression testing

----
----
==== OPTIONS ====
=== Classic1: Unittest / **pytest** libraries ===
== Assertions ==
  * **Definition:** Statements used in tests to verify that a specific condition holds true.
  * **Usage:** Typically done using Python's built-in <code>assert</code> statement.
  * **Example:**
    <code python>
    assert interface.status == "up"
    </code>
  * **Purpose:** To validate correctness during test execution; a failed assertion marks the test as failed.

----
(@ is a decorator. special function that modifies the behavior of another function or class without changing its actual code)
== Markers ==
  * **Definition:** Annotations to add metadata to test functions or classes.
  * **Usage:** Used to categorize, skip, or parametrize tests. 
  * **Examples:**
    * <code>@pytest.mark.sanity</code> (categorization)
    * <code>@pytest.mark.skip(reason="Not implemented")</code> (skipping)
    * pyATS-specific: <code>@aetest.loop</code>, <code>@aetest.setup</code>, <code>@aetest.cleanup</code>
  * **Purpose:** Control test behavior, grouping, and execution.

----

== Fixtures ==
  * **Definition:** Reusable setup/teardown functions for tests.
  * **Usage:** Fixtures are defined using the `@pytest.fixture` decorator and are **injected automatically** into test functions based on parameter names.
  * Provide setup logic for tests; Can include teardown logic after the test (using `yield`); Can be reused across multiple tests; Can depend on other fixtures; Can be parameterised

**Example1:**: The ''greeting'' fixture is provided automatically to both test functions. In this basic example, the greeting fixture returns a simple string. Because both test_uppercase and test_length declare a parameter named greeting, pytest automatically calls the fixture and passes its return value to each test function. This means you don’t have to repeat "hello world" in multiple tests — the fixture provides a single source of truth **for that test data**, ensuring consistency and reducing duplication.

  import pytest
  @pytest.fixture
  def greeting():
      return "hello world"
  def test_uppercase(greeting):
      assert greeting.upper() == "HELLO WORLD"
  def test_length(greeting):
      assert len(greeting) == 11

**Fixture with Setup and Teardown**: The yield keyword pauses the fixture until the test finishes, then resumes for any cleanup. This fixture uses **yield** to provide a temporary **file path to the test**, allowing the test to use the file as needed. Once the test finishes, the code after the yield runs as the teardown phase — perfect for cleanup or logging. The tmp_path argument itself is a built-in pytest fixture that provides a temporary directory, which ensures isolation and avoids side effects between tests.
  import pytest
  @pytest.fixture
  def temp_file(tmp_path):
      file = tmp_path / "sample.txt"
      file.write_text("Hello")
      yield file
      # Cleanup code here (e.g. delete or log)

** Dependent Fixtures **: Fixtures can call other fixtures automatically. Here, the **user_token** fixture depends on the user_data fixture, which pytest resolves automatically. This shows how fixtures can be layered: one fixture can receive another as a parameter, enabling you to build more complex test setups from smaller, testable pieces. The test then uses user_token without needing to worry about where or how the token was created — the fixtures manage it all behind the scenes.
  @pytest.fixture
  def user_data():
      return {"name": "Alice", "id": 1}
  @pytest.fixture
  def user_token(user_data):
      return f"TOKEN-{user_data['id']}"
  def test_token(user_token):
      assert user_token.startswith("TOKEN-")
      
** Parametrised Fixtures ** : The test runs once for each value of the user parameter. This example shows how to create a single fixture that can supply multiple different values across different test runs. Here, the user fixture will run the test twice — once with "alice" and once with "bob". The request.param object is provided by pytest for each parameter. This approach is ideal when you want to test the same logic against a range of inputs without writing separate test functions for each.

  @pytest.fixture(params=["alice", "bob"])
  def user(request):
      return request.param
  def test_starts_lowercase(user):
      assert user[0].islower()
      
** restapi test **
In this example, we are testing the REST API of a network device. The api_base_url fixture provides the root URL of the API, which could represent a router or switch’s management interface. The auth_headers fixture supplies the necessary HTTP headers for authenticating the request, such as a bearer token and the content type. These two fixtures are injected into the test function, which uses them to send a GET request to the /interfaces endpoint. The test asserts that the response status is 200 (OK), and that the response JSON contains an "interfaces" key. This setup is clean, extensible, and easily scalable to test other endpoints, methods, or even multiple devices with different credentials or configurations.
import pytest
import requests

  @pytest.fixture
  def api_base_url():
      return "http://192.168.1.1/api/v1"
  @pytest.fixture
  def auth_headers():
      return {
          "Authorization": "Bearer test-token",
          "Content-Type": "application/json"
      }
  def test_get_interfaces(api_base_url, auth_headers):
      url = f"{api_base_url}/interfaces"
      response = requests.get(url, headers=auth_headers)
      assert response.status_code == 200
      assert "interfaces" in response.json()




===== Further Reading =====

  * [[https://docs.pytest.org/en/stable/how-to/fixtures.html|Pytest Fixture Documentation]]
  * [[https://realpython.com/pytest-python-testing/#using-pytest-fixtures|Real Python: Pytest Fixtures]]
  * [[https://docs.pytest.org/en/stable/how-to/fixtures.html#parametrizing-fixtures|Parametrising Fixtures]]
  * [[https://gist.github.com/kwaldrip/0ed22c6e3c8b476b8a84cf3c137b3e15|Pytest Fixture Cheat Sheet (Gist)]] 
  * [[https://packetpushers.net/blog/open-source-networking-projects/#automation]] << more examples in this packetpushers site