It appears that some rules in jsonschema and flake8
are now more strict. This patch propose fixes, so
that pep8 job passes.
Change-Id: I0a7c533dd0c576e564749e5d83242fb1b050f74f
The tools/consistency.sh needs to respect upper constraints
when installing plugins, otherwise we may install newer version
of dependecies which aren't recommended by the constraints
and that may lead to other issues.
Change-Id: I29c3a3020f3d8df911b7f0a0fe91d391d3a52bba
We forgot to document the latest guidelines and include them in our
project's documentation. This patch fixes that plus it modifies
the tools/jsonToRst.py so that it takes into account the 2 new programs,
added in the 2022.06 guidelines as next ones.
In order to avoid this mistake in the future, the Internal Process
Documentation is extended about the steps required to document a guideline.
Change-Id: I0447adb2685a0766f7935e47816cee57a50e6d77
Based on the analysis in
https://etherpad.opendev.org/p/refstack-test-analysis
and gathered feedback for neutron tests provided by neutron
team, this patch adds new tests to the guideline as well as to
the exclude file.
Change-Id: Id16480651760ae0e6dbca8dc88d854ee2fd24766
The commit adds octavia-tempest-plugin to the cross_check tool.
This is a preparation for adding octavia support in our interop
tooling.
Change-Id: Iaae4538a078048b2e4e446fb42d32fbee2e66544
The commit adds barbican-tempest-plugin to the cross_check tool.
This is a preparation for adding barbican support in our interop
tooling.
Change-Id: Ie8fd66a944678332683ff898b37abfc7989eb448
Added support to 1.x schema to make sure that
older guidelines can still be converted.
* created new script for 1.x schema
* argparse used for argument handling
* script can be run from top level dir
* added --outdir option
Change-Id: I051008040b4dc05e6a7a2d2ff045ddbe6629deb1
The patch converts the 2021.11 guidelines to a .rst file so that
it can be included in the documentation.
The patch also fixes a few links pointing to the old interop
namespace (osf/) - seems like we accidentally reverted some of the
changes.
Apart from that the patch unifies target_approval format for
shared_file_system add-on so that it matches the format of other
add-ons.
The patch then copies all 2021.11 files to their "next" versions so
that we can build a new guideline on the top of the previous one.
Change-Id: I1dea3743ca6bf657313299177aabe69ddf234784
Fixing a few links as the original ones were returning 404.
The commit fixes .rst syntax issues to make the pages
render "nicer". The commit changes only format of the .rst
code, the content of the pages is not changed at all.
Change-Id: Icf0da2d7d1b42b065639c9105fbc16e1999bf630
The commit updates tempest SHA used for test analysis so that
it reflects tempest version used in refstack-client.
Change-Id: I0060ba298faf08e7f43f4deff0627f88d426473c
* works with python3
* argparse used for argument handling
* script can be run from tools dir and from repo's top dir as well
* support for schema 2.0 added
* support for add-ons added
* creates single guideline for core + add-ons on same date
* added --file and --all options
* keep old script in file jsonToRst_schema_1.py
Task: 42762
Story: 2009028
Change-Id: I6deb70fa39e8a51fcff0372217bbfc46282d7237
Chectest.py wasn't able to catch bug [1] as it didn't take into
consideration inheritance during the test discovery.
The cause of the issue is the fact that chectest.py parsed python files
by itself to discover the tests. This patch fixes this by using unittest
library for test discovery instead.
[1] https://review.opendev.org/c/osf/interop/+/806178
Story: 2009146
Task: 43097
Change-Id: I6e1b11eeb3ca1915ca41b6af88f9f568e6d674eb
Based on our analysis [1], there were several network tests
available which weren't included in any interop guideline.
This commit adds those tests under a few new capabilities which
are added as advisory for now. They can be moved to required
after one cycle being under advisory. Interop WG will review
these tests for inclusion for the next guideline cycle.
[1] https://etherpad.opendev.org/p/refstack-test-analysis
Change-Id: I5a044ebf06d62988a6e69fe458ef7f3193cc84d8
When Tempest loads tests class inheritance plays a big role,
the test name (or path if you will) is affected by the inherited
classes.
ShareNetworksTest class inherits from ShareNetworkListMixin,
see [1], therefore the test in questions appears to be under
ShareNetworksTest and not ShareNetworkListMixin which is its
litteral position.
This issue wasn't uncovered by the consistency checks
because tools/checktest.py parses test files manually and
therefore the check passed as it found the litteral position
of the test not the based on the inheritance.
[1] 8ccbbebe6d/manila_tempest_tests/tests/api/test_share_networks.py
Change-Id: I685d94d045f4c20cfb9430d200b52e1c5cf4d268
This patch introduces script that finds candidate tests from
tempest (and tempest plugins) that can be used for refstack testing.
The script lists all test from tempest (and tempest plugins) except those
that:
- are/were part of interop repository or
- are mentioned in exclude file or
- are scenario tests or
- require admin access.
Change-Id: I7c087bca458943b3ff8b6fc098626af466210474
Hacking provides valuable standard checks such as imports in
alphabetical order. Having this included in our pep8 check/job
the quality and readability of the code will be improved.
Change-Id: Ieedfcd340e9b700bdead0073bac39a505c8628b5
Based on our analysis [1], there were several identity tests
available which weren't included in any interop guideline.
This commit adds those tests under a few new capabilities which
are added as advisory for now. They will be moved to required
after one cycle being under advisory.
[1] https://etherpad.opendev.org/p/refstack-test-analysis
Change-Id: I38e7860245470689aad6f9d79bbcb7b8528b48f7
Reorganize guidelines into guidelines directory and create
current_guideline that is softlink to the latest approved guideline.
The same with add-ons guidelines - they are moved to guidelines
directory within add-ons one and soft links are created pointing
to the latest guidelines for each add-on.
Also cleaned up some tooling that hardwired where guidelines lived.
Change-Id: I5ad4b91b1afb44a0a6987b339f7efba14f395302
Adds a job which runs all next tests on a devstack master and
experimental jobs which run the same on supported stable releases.
Fix names of a few manila_tempest_tests and designate_tempest_plugin
tests so that they match the names from the plugins.
Change-Id: Iedbaa09e425a29156304eea63b66a1ce69dd77bd
Reorganize guidelines into previous_guideline directory
and current_guideline that is softlink to the latest approved guideline.
Cleaned up some tooling that hardwired where guidelines lived.
Change-Id: Ia6be9ca6326718488ee5668df3806da5f76dc456
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.
This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.
This update should result in no functional change.
For more information see the thread at
http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html
Change-Id: Iddb49d37012489e0d04a8b5886389dce26920e0b
This mostly fixes print statements and some pep8 issues.
Change-Id: Ie4fb0ac8955759954d882988d698fb4aace61de1
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
This patch updates some of the consistency checking tools
to make them more generally usable by allowing both
automatic checkout of tempest and manual checkout. It builds
on this to add a Zuul job that gates against the consistency
check.
Change-Id: Id0d9148af39c2d2ccf8f11502cc9aff6699f0ead
This tool checks a guideline against a Tempest test suite
to make sure that:
* all idempotent IDs in the guideline exist in the tests
* test names in the test suite match names in the guideline
By gating against this tool in both the Interop project and in
Tempest, we can help prevent typos and catch renaming of tests.
Change-Id: I8b6778c2063eeac8b0d7c0d64127eefe420c617f
To support community goal python35. Make sure script works correctly.
Also fixed a couple of references to defcore cgit repo
Change-Id: Ia75e5e0848813f895952663680630f9e9ca6a5c8
push-csv.py takes the resulting modified csv file and loads it back
into scoring.txt, then retabulates the scores based on the new lines.
Change-Id: Ic82d7c54a9ca6969400dd4b7a9d9ef1a9dec2981
At the Board of Directors meeting on April 24, 2016 the Board of
Directors indicated that as DefCore has evolved it's focus on
interoperability and it's working structure, it's name may be a source
of some confusion to those outside of the community or who are new to
the community. The Board made an informal request that the DefCore
Committee consider changing it's name to more clearly reflect it's focus
and structure.
This patch is the first step in that process. It updates references
in various documents to change the name "DefCore Committee" to
"Interop Working Group" as agreed at the summer 2016 DefCore Committee
Sprint [1].
It should be noted that this patch should be considered a work in
progress to generate discussion until the new name is approved by
the DefCore Committee and the Board of Directors. Should we elect
to go forward with the new name, some other actions will also need
to be taken, including but not limited to:
1. We will need to consider updating references on the OpenStack wiki.
2. We will need to consider updating the name of our IRC channel,
mailing list, Launchpad project, and git repository. Most of these
changes will need to be carefully coordinated with the OpenStack
Infrastructure team.
3. We will need to take into account external resources that point to
DefCore artifacts, such as Foundation-maintained websites (such as:
http://www.openstack.org/interop ).
4. We will need to coordinate with RefStack to minimize impact.
5. We will need to clearly communicate the name change to the rest of the
community.
Note also that I've intentionally left many historical documents that
have been superceded (such as the 2015A process docs, Guidelines that
are no longer used, etc) in tact. There seemed little value in
spending time on them and cluttering the patch with them since they're
now obsolete.
[1] https://etherpad.openstack.org/p/DefCoreSummer2016Sprint
Change-Id: I79d337c193e75c54d49f1d847468f6347e2ef2b3
We're starting to accumulate a number of python scripts in
various parts of the tree, and probably will continue to do so.
To make maintenance easier over time, we should have basic
pep8/flake8 checks in place for python code, as is best practice
elsewhere in the OpenStack project. This patch adds basic checks
and corrects existing errors in our python scripts so that going
forward we can opt to gate on clean python code.
Change-Id: Ie32ee7a14ee608e12f42288e137a0849555b5ed8
We recently moved the RST guidelines into the doc tree so they
can be eventually be published. Due to the timing of the change,
we missed the 2015.07 Guideline, which was just approved. We also
need to update the jsonToRst.py script to reflect the new location.
This patch catches us up on both fronts.
Change-Id: Ia041309b9738ffc8007a054ba611dd6d66943961
The .rst Guideline files currently say that they're generated from
"the master JSON verison" but don't actually contain a link to their
JSON counterparts. This patch makes that reference a hyperlink so
folks can more easily locate the JSON files that are the authoritative
documents. This is especially useful when viewing rendered versions
of the RST docs.
The patch also adjusts the script we use to generate the RST docs
from their JSON sources so future Guidelines include similar
linkage.
While we're at it, we've also removed a chunk of code in the
RST generator script so that it doesn't rely on the "name" field
since we never really added those fields universally, which could
cause the script to fail. Instead we just use the "project" field.
Further, a few capabilities that are marked as removed are lacking
the aforementioned "project" field, so those have been added where
they were missing.
Following the changes to the RST generating script, it has been
re-run so that the existing 2015 Guideline RST documents have
the same formatting. While the formatting has changed (added links
and modified capitalization), the content has not (no capabilities
or tests added/removed).
Co-Authored-By: Mark T. Voelker <mvoelker@vmware.com>
Co-Authored-By: Rob Hirschfeld <rob@zehicle.com>
Change-Id: Idbdd7d6825aa8af2404b9e2791fa67240cdef239
As per the process, we want to use generated RST
instead of hand created files.
In generating the files, I found a comma error in 2015.03
and removed spaces from 2015.04 so there are minor
changes to those files.
Because 2015.next is still in process, it does not make
sense to generate the RST file at this time. We do
NOT want to have to maintain multiple files for each
change. I've removed the 2015.next.rst file.
Also added check to make sure we're using the
correct JSON source using the hardcoded DefCore
repo path as an extra check.
Change-Id: I443f4c1cb84eb2c42ad9e6914cee0ce6d498a9bb
Added checks for json keys and readable error messages.
Added checking for json files that are actually lists
Change-Id: If307a6e0c939bfac722081cce2e4934fee432e65
Roman Popelka