Dcnm image upgrade v2 #315
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rts. 1. ImageUpgradeTask: - Convert type hints to non-deprecated dict/list and remove typing imports. - _build_params_spec(): Add a return None at the end to appease pylint. - get_want(): self.task_result does not have a result member. Remove self.task_result.result["changed"] = False since this is now set by ImageUpgradeTaskResult().did_anything_change().
1. Separate the setup and cleanup scripts from the test scripts. This shortens each test script. Complete - setup scripts - deleted.yaml - merged_global_config.yaml TODO - cleanup scripts - merged_override_global_config.yaml - query.yaml
1. Finish the changes from last commit. 2. Modify test case titles and descriptions for consistency. 3. Remove verification from PRE_TEST cases. 4. Update test run times.
In preparation for dcnm_image_upgrade sharing module_utils/image_policy/image_policies.py with dcnm_image_policy, copy module_utils/image_policy/image_policies.py to module_utils/common/image_policies.py
Initial set of commits which modify dcnm_image_upgrade support classes to use RestSend(), Results(), and other v2 classes and reduce dependency on AnsibleModule().
1. dcnm_image_upgrade.py
- import v2 classes.
- raise exceptions where fail_json() was previously called.
- Organize into classes associated with Ansible states e.g. Merged(), Deleted(), Query()
- remove ansible_module arguement.
- Update docstrings
2. module_utils/image_upgrade/switch_issu_details.py
- Leverage EpIssu() endpoint
- Leverage ConversionUtils()
- Leverage Properties() for rest_send, results, and params properties.
- Update docstrings
- raise standard exceptions rather than callling fail_json()
3. module_utils/image_upgrade/image_policy_attach.py: ImagePolicyAttach()
- Refactor ImagePolicyActions() into separate classes for attach, detach, query.
- This is the first class to arise from this effort, for attach.
Work in progress. Changes to migrate dcnm_image_upgrade support classes to use RestSend() v2, Results(), and Api() classes.
Also, update a few debug log messages.
Since we are sharing the RestSend() instance between various classes, and each class modifies RestSend().path and RestSend().verb, we need to be careful to update path/verb appropriately. Updating these in closest proximity to RestSend().commit() is the best practice to avoid unexpected results.
During integration testing, which we updated to do back-to-back upgrades across two different images, we hit a situation where ImagePolicyAttach() started receiving 500 responses from the controller because it was trying to attach an image policy to a switch while one of stage, validate, or upgrade was still in progress. Added a call to _wait_for_current_actions_to_complete() prior to calling attach_policy(). Hopefully this detects whatever was in progress and avoids the 500 response.
1. WaitForControllerDone() replaces method _wait_for_current_actions_to_complete() in the following classes:
- ImagePolicyAttach()
- ImageStage()
- ImageUpgrade()
- ImageValidate()
And is used in the following new class:
- ImagePolicyDetach()
2. ImagePolicyDetach() new class.
3. dcnm_image_upgrade.py
- Deleted(): initial modifications for v2 classes.
We want these endpoints to be self-sufficient, so require a list of serial_number and build the path to include the requisite query string.
1. image_policy_detach.py:
- Remove unneeded policy_name and params properties.
- Leverage WaitForControllerDone()
- Update docstrings
2. dcnm_image_upgrade.py
- Finish Deleted() class.
3. wait_for_controller_done.py
- Update docstrings
- Run through linters.
4. image_policy_attach.py
- Refactor.
- call results.register_task_result() to update results.
5. switch_issu_details.py
- Fix misleading docstrings.
6. policymgmt.py
- EpPolicyDetach().serial_numbers: was not seting self._serial_numbers.
1. Merged.commit(): the following were not populated correctly:
- stage_devices
- validate_devices
2. Deleted()
- Add class docstring
- Add validate_commit_parameters()
3. Merged()
- Add class docstring
- Add validate_commit_parameters()
4. Query()
- Add class docstring
- Add validate_commit_parameters()
1. image_stage.py
- Rename self.endpoint to self.ep_image_stage
- Add method return type hints
- update docstrings
- refactor commit() diff construction into build_diff() method.
- Fix results handling
2. image_validate.py
- Rename self.endpoint to self.ep_image_validate
- Add method return type hints
- update docstrings
- refactor commit() diff construction into build_diff() method.
- Fix results handling
1. items.setter: should raise TypeError. 2. commit(): improve error message.
All: wait_for_controller()
- Add this method if not already added.
- Update docstrings.
All: add pylint: disable: no-member where needed.
All: update docstrings for many methods.
image_upgrade.py: refactor commit() and organize similarly to image_stage and image_validate.
Log debug message needed to be in the outer while loop. Fixed.
1. Add debug logs to trace method execution order. 2. ImageStage().__init__(): initialize rest_send and results properties. 3. ImageStage().register_unchanged_result(): rename input parameter to avoid varname collision. 4. Run through linters.
…ses.
1. Rename all unit tests from "test_image_upgrade_validate_*" to "test_image_validate_*"
2. Rename response files to include the endpoint (since the responses directly corresponsd to endpoints).
3. image_validate.py
- ImageValidate().validate_serial_numbers: Align Exception behavior with image_stage.py
- Convert serial_numbers_todo to class-level var.
- Align serial_numbers property with image_stage.py
4. utils.py
- Remove MockAnsibleModule argument from issu_details_by_serial_number fixture.
- Rename responses_image_validate() to responses_ep_validate_image()
1. test_image_validate.py
- complete unit test alignment.
- Run through linters.
2. image_validate.py
- commit(): In raise ControllerResponseError block, update all results before calling register_task_result()
- Add debug log on entering all methods to trace code flow.
- Run through linters.
1. Update unit tests to align with changes made to leverage v2 support libraries.
2. switch_issu_details.py:
- SwitchIssuDetails().__init__(): rename self.endpoint to self.ep_issu
This completes the alignment of SwitchIssuDetailsBy*() unit tests with the v2 support libraries.
Also:
switch_issu_details.py:
- SwitchIssuDetailsByIpAddress()._get(): remove self.failed_result from ValueError() params.
1. Rename response and payload files to include the endpoint.
2. test_image_upgrade.py
- Rename all test cases from:
- test_image_upgrade_upgrade_*
to:
- test_image_upgrade_*
- Rewrite test cases to use v2 support libraries.
- Rewrite _init_properties testcase to align with 3. below.
3. image_upgrade.py
- Remove properties dict and replace with self._<property_name> for all properties.
…plete.
1. test_image_upgrade.py
- All unit tests have been updateed to align with v2 support library behavior.
- Renumber unit tests.
2. image_upgrade.py:
- ImageUpgrade().commit()
- Add a try/except block around rest_send.
- Update docstring.
- ImageUpgrade()._init_properties()
- Remove unused self._response_data
3. responses_ep_image_upgrade.json
- Remove multiple unused responses.
1. utils.py
- Add devices_image_upgrade() to return devices configurations from devices_image_upgrade.json
2. test_image_upgrade.py
- Use ResponseGenerator() to return devices configurations.
- Move all devices configurations to devices_image_upgrade.json
3. image_upgrade.py
- Remove debug log message.
- Run through linters.
1. install_options.py
- refresh(): move epldModules handling to epld_modules.getter
- Update docstrings.
- Remove unused properties.
- Add pylint: disable=no-member where needed.
2. test_image_install_options.py
- Align test cases with v2 support libraries
- Renumber test cases
- Update docstrings
Also: - Update several log messages.
The following test cases started failing. I think due to change with RestSend(). The specific failure is due to Results().failed set being populated with False (where before it was not). This should not affect functionality so I've just removed the asserts from these test cases that tested for False not being in Results().failed set(). test_fabric_delete_00042 test_fabric_update_bulk_00033
0. Update common_utils.py
- Changes for v2 version of ControllerVersion()
- Rename responses_controller_version() to responses_ep_version()
- Rename responses_image_policies() to responses_ep_policies()
1. ControllerVersion(): Complete aligning unit tests with v2 support classes.
test_controller_version.py:
- Rewrite / renumber all test cases.
- Rename responses_ControllerVersion.json to responses_ep_version.json
- Update docstrings
controller_version.py
- Rename self.endpoint to self.ep_version
- Remove unused properties result and response.
2.ImagePolicies(): Initial work on aligning unit tests with v2 support classes.
test_image_policies.py
- Initial preparation, import requisite classes.
- Change import from responses_image_policies to responses_ep_policies
- Update docstrings
image_policies.py
- Minor ordering change to __init__()
1. image_policies.py
- ImagePolicies().refresh(): raise ControllerResponseError() if RestSend().result_current indicates the request failed.
2. test_image_policies.py
- Align tests with v2 support classes.
- Update docstrings
- Renumber tests.
EpPolicyDetach() was earlier modified to require serial_numbers to be set. Updating two unit tests to align with this requirement.
Removing files that exactly duplicate unit tests. We renamed these files to include the full API path and forgot to remove the original files
test_v1_api_policy_mgmt.py ->
test_api_v1_imagemanagement_test_policymgmt.py
test_v1_api_staging_management.py ->
test_api_v1_imagemanagement_rest_stagingmanagement.py
test_v1_api_image_mgnt.py ->
test_api_v1_imagemanagement_rest_imagemgnt.py
test_v1_api_switches.py ->
test_api_v1_lan_fabric_rest_control_switches.py
test_v1_api_image_upgrade_ep.py ->
test_api_v1_imagemanagement_rest_imageupgrade.py
test_v1_api_templates.py ->
test_api_v1_configtemplate_rest_config_templates.py
Removing files that exactly duplicate unit tests. We renamed these files to include the full API path and forgot to remove the original files
test_v1_api_fabrics.py ->
test_api_v1_lan_fabric_rest_control_fabrics.py
1. controller_features.py
- Use add_rest_send class decorator and remove local rest_send property.
- Remove unused result and response properties.
- Collapse _init_properties() into __init__()
- Remove self.properties and use self._<property_name instead.
- Remove need to pass params to the class.
- Rename self.api_features to self.ep_features
- Update docstrings
2. test_controller_features.py
- Update all test cases to align with above changes.
- Remove params["check_mode"] testcase.
- Update docstrings
3. test_controller_version.py
- Remov unused Results import.
1. test_switch_details.py
- Add testcase test_switch_details_00550a to verify behavior when 500 response is received.
- Remove asserts for instance.path and instance.verb since these are now removed.
- Remove assert isinstance() for conversion and ep_all_switches. Test these using instance.class_name instead.
- Remove ConversioniUtils() and EpAllSwitches() imports since we are not testing with isinstance anymore.
-Remove ControllerResponseError() and "copy" imports which were never used.
2. switch_details.py
- Remove self.path and self.verb and pass these directly to RestSend via ep_all_switches.path., and ep_all_switches.verb
This commit removes deprecated type hints Any and Dict from typing. 1. from typing import * statements are removed. 2. function signatures are modified to use the new recommended type hints. While a lot of files are changed, there are no other changes to any of the files.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR aligns
dcnm_image_upgradewith the v2 support libraries.AnsibleModuletomain()in dcnm_image_upgrade.py.Sender()insender_dcnm.pyandsender_file.pybranch
dcnm_image_upgrade_v2created off of branchdcnm_image_policy_v2.Changes