Commit Graph

785 Commits

Author SHA1 Message Date
Martin Kopec fa704ce067 Include openstack-tox-py311 job
Getting rid of the antelope template and replacing it with the list
of the jobs instead - it's gonna be easier to add newer python
jobs as we won't have to check which ones are a part of a template
and which need to be added directly.
mysql-server and mysql-client seem to be changed for mariadb-server
and mariadb-client:
https://github.com/geerlingguy/ansible-role-mysql/issues/336

Change-Id: Id37a96f32e58c5a91f25d731c89c7a565614d99f
2023-09-20 10:28:04 +02:00
Vida Haririan 69fd83bc43 Remove deprecated folder docker and it's contents
This was replaced by Ansible docker

Change-Id: I162cba0e44c83ad644d0a8ad3030b6c7e39ac8bf
2023-06-22 06:24:46 +00:00
Vida Haririan 5eac91ce6b Update Refstack Docs
Removed Option 2 install docs and associated files.
Renamed Option1AnsiblePlaybook.rst to refstack_setup_via_ansible.rst
, move to /refstack/doc/source

Change-Id: I1cc309c9a1bec622aadfcf0840c546d3d243e7c5
2023-06-05 11:50:39 -04:00
Zuul 620d54c00f Merge "Adding 2 missing programs to menu on home page" 2023-05-18 09:26:37 +00:00
Zuul 1b80853ea7 Merge "Update jobs for Bobcat development cycle" 2023-05-18 09:24:25 +00:00
jskunda 75eb3c0bf1 Update jobs for Bobcat development cycle
- Adds an Antelope job as upstream has just released it and moved to
  the Bobcat development cycle
- Update nodejs version to Node.js 18

Change-Id: Ia83c1a0e7ae011657bd195c23b0aae36381ff3a7
2023-05-18 08:53:36 +00:00
Martin Kopec f33b8624ed Adding 2 missing programs to menu on home page
Couple of months ago we added 2 new add-on programs to the
refstack, however, we forgot to edit the home page.
This patch fixes that.

Change-Id: Idccaca8b0af7928ee4f323719d801ff037c1446a
2023-05-10 21:15:19 +02:00
jskunda 9e79b3b4ae Fix missing about content in refstack-ui
Refstack webpage's about section is not rendered correctly. The bug is
tracked in here:
https://tree.taiga.io/project/openstack-interop-working-group/issue/81

This patch will update documentation build process to generate html
files by changing paths to html files in about.html.

Replace deprecated options in doc/source/conf.py by the new ones:
* openstackdocs_bug_project
* openstackdocs_bug_tag

Change-Id: Id296b9cecf75bb289b0da41bec880a13694298e1
2023-04-18 19:07:50 +02:00
Zuul aa239130d6 Merge "Fix security vulnerabilities using Bandit" 2023-02-08 13:25:41 +00:00
Lukáš Piwowarski d18f8ad221 Make refstack jobs work again
There were three main issues in the code from refstack repository that
were preventing refstack jobs from working:

1) Improper handling of sessions in the database API.
2) PEP8 job was failing because of outdated version of flake8.
3) The new version of cryptography library doesn't support signer() and
   verifier() functions.

Issue #1 was solved by using the get_session() function as a context
manager instead of using session.begin() as a context manager. Using
session.begin() as a context manager does not ensure that the session
will be closed at the end of the context (see "Opening and Closing
a Session" and "Framing out a begin / commit / rollback block"
here [1]).

Issue #2 was solved by updating the libraries in
test-requirements.txt file. This change also forces flake8 to ignore
some pep8 errors (similar to the ones ignored in tempest project).

Issue #3 was solved by using the sign() and verify() functions instead
of verifier() and signer() functions [2].

Related Taiga issues:
 - https://tree.taiga.io/project/openstack-interop-working-group/issue/77
 - https://tree.taiga.io/project/openstack-interop-working-group/issue/79

[1] https://docs.sqlalchemy.org/en/14/orm/session_basics.html
[2] e71c0df301

Change-Id: If98670475b371d1ece7c877a0eea3158f6c1b3f5
2023-02-01 15:29:36 +01:00
jskunda 5ba7c56a7f Fix security vulnerabilities using Bandit
Refstack was using:
-1024 bits long key[1]
-hardcoded sql expressions[2]

Both of these was vulnerable to security attacks.

Instead of 1024 bits keys refstack is now using 2048 bits keys.
Sql expressions were rewriten in the recomended way[3].

[1] https://bandit.readthedocs.io/en/1.7.4/plugins/b505_weak_cryptographic_key.html
[2] https://bandit.readthedocs.io/en/1.7.4/plugins/b608_hardcoded_sql_expressions.html
[3] https://security.openstack.org/guidelines/dg_parameterize-database-queries.html

Change-Id: I91a18ef0fd4c300094553f7c3b21d8a745c057c7
2022-10-17 13:31:51 +00:00
Martin Kopec f222f3fb95 Fix dash-separated options in setup.cfg warning
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Let's get ahead of the issue by replacing the dashes with underscores.
Without this, we see 'UserWarning' messages like the following on new
enough versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cba

Change-Id: I33fcd3a39b58b07585f21950f58ea35cd8419282
2022-08-11 16:11:08 +02:00
Martin Kopec fbe48dbad7 Add support for barbican
The commit edits refstack page so that it supports key_manager
add-on program.

Change-Id: I8727298f7f988f40832318f2b810c184ac76f38c
2022-08-11 16:06:20 +02:00
Martin Kopec 9ebea4b13b Add support for octavia
The commit edits refstack page so that it support load_balancer
add-on program.

Change-Id: Iab3e4e82506652405a0884e10e0faf9d7e8fc42a
2022-08-11 16:06:13 +02:00
Martin Kopec 334dd378c9 Fix compatibility issues
A lot of things have changed at once (or have been ignored
in refstack for some time) and in order to keep refstack
functioning this commit addressed the following:
* Update nodejs jobs so that we keep testing refstack
with the newest nodejs
* Update jobs for Zed development cycle
* Add support for py3.10
* Fix 2009327 story - refstack is now compatible with
jsonschema>=3.2.0 (schema definition got fixed)
* Drop py3.6 and 3.7 (upstream tooling as e.g. devstack
isn't supporting those anymore)
* Fix mysql setup script used to run functional unit tests
* Add WebTest dependency needed by unit tests
* Update alembic and jsonschema version to at least
currently recomended versions (by constraints)

Task: 43761
Story: 2009327
Change-Id: I37bb7dc520119bdab280d62e23922641ef7658ce
2022-08-11 15:52:14 +02:00
Martin Kopec 8daea3e908 Add contributing page and some updates
The commit adds contributing page for easier orientation
in our projects.

Change-Id: Ida438dd72ffd004e858f21065f8c65fde4d7a23a
2021-11-15 14:36:25 +00:00
Martin Kopec bb0e985734 Cap jsonschema to 3.2.0
refstack fails with jsonschema>3.2.0 on result schema validation,
until we will have resolved that the commit caps the jsonschema to
the last working version - 3.2.0

Story: 2009327
Task: 43762
Change-Id: Ie70c85febbe723562f16633b6175ebc7a50925d8
2021-10-26 09:53:15 +02:00
Zuul 4d2c7db14d Merge "Reflect osf/ to openinfra/ namespace rename" 2021-10-25 06:57:53 +00:00
Martin Kopec 5eb0e719fc Reflect osf/ to openinfra/ namespace rename
Lately osf/ namespace got renamed to openinfra/ [1], this commit
reflects this change within refstack's repo.

[1] https://review.opendev.org/c/openstack/project-config/+/808479

Change-Id: Id8ff246faf19ac67ed6c9dc5ab2514c4655a0610
2021-10-21 22:35:42 +00:00
Martin Kopec a87eb6c818 Revert "Fix uploading of documentation to opendev.org #2"
This reverts commit 1e40c8784c.

Reason for revert: It has turned out the previous job was correct

Change-Id: I78c43c3bfbd14797debcd1dfa74e448e2e1257bb
2021-10-19 19:29:20 +00:00
Martin Kopec 1e40c8784c Fix uploading of documentation to opendev.org #2
Previously used opendev-promote-docs didn't publish the docs
to docs.opendev.org as expected, this patch is trying the
following job instead: promote-tox-docs-infra

Change-Id: Id9005daf2848f07e353ecdf1387395a669c5fddf
2021-10-19 18:46:19 +00:00
lpiwowar ae592877b6 Fix uploading of documentation to opendev.org
This patch makes sure that the promote pipeline uploads the
generated documentation to opendev.org. Previously the promote
pipeline called opendev-tox-docs job (this had no effect on
uploading of the documentation).

Change-Id: Ib1f6742a6614a9f64d128a7fd611044372d66f26
2021-10-19 16:26:09 +02:00
lpiwowar 42df359990 Fix documentation structure
This patch fixes the structure of the Refstack documentation and
tries to make it more readable.

Change-Id: I4ae3a0274b1f3201a44bbac516c00f16dec5ce55
2021-10-18 11:27:11 +02:00
lpiwowar 7bbd4efa37 Publish docs to docs.opendev.org
docs.opendev.org will provide easier access to the doc pages
in comparison to the Gitea (opening the doc files in the repo).

Change-Id: I88f587b14f911ee34210f6a249f789929eb3a65c
2021-10-18 11:14:11 +02:00
Goutham Pacha Ravi 2f508df44d [project-rename] Update .gitreview
This project was moved from the osf/ namespace
to openinfra/.

Change-Id: Ifebeb374c17223b4b69ff06956dcb849bfe11267
2021-10-16 18:07:56 +00:00
Martin Kopec 1e6acbec85 Switch testing to Yoga testing runtime
Updating the testing template to Yoga testing runtime:
https://governance.openstack.org/tc/reference/runtimes/yoga.html

The commmit also removes refstack-tox-pip-check-reqs job due to a
bug in the pip-check-reqs tool:
https://github.com/r1chardj0n3s/pip-check-reqs/issues/70
and the fact the tool is not used in the gates by openstack
projects - still can be executed manualy though.

Change-Id: I6d64cb67529585b458b28ed4fe011d50d1f23d6d
2021-10-15 11:38:07 +00:00
Martin Kopec 5790634ff4 Update documentation, add playbook for developers
The patch updates the user-faced documentation and adds a playbook
together with a doc meant mainly for RefStack developers which
helps with running the server instance locally.

Change-Id: I268fdf717c8f0e2cd0c130c560306bf54e48f348
2021-06-25 10:11:36 +02:00
Martin Kopec 0395f676ab Clean requirements, add pip check job
Removes redundant requirements and adds a new job which runs
'tox -epip-check-reqs' in order to avoid having redundant
dependencies specified or missing some.

Change-Id: I5ca35922213cfbe4c2b4b49388bffcc0b8fc6a96
2021-06-23 12:50:15 +00:00
Martin Kopec 95680a792d Update refstack.conf.sample
The sample config file hasn't been regenerated for years. A few
config options got changed over the years, mainly those from oslo.
The file has just been regenerated by oslo-config-generator -
'tox -egenconfig'.

Change-Id: I298baf65460fa4f1bb4c43dbd5d4758df35c9f18
2021-06-23 12:31:19 +00:00
Martin Kopec 2a0a147dfa Update the guidelines location, try #2
[1] wasn't fully complete change/preparation for guidelines
move which lead to some issues and we had to revert [2].
We revisited [2] and proposed the change again with a few
improvements [3].
This patch should change everything we need in refstack in
order to merge [3].

[1] https://review.opendev.org/c/osf/refstack/+/790940
[2] https://review.opendev.org/c/osf/interop/+/786116
[3] https://review.opendev.org/c/osf/interop/+/796413

Change-Id: Ib9f083d1f1a713b3fa26c5daca781fedfa4923fc
2021-06-22 22:43:10 +00:00
Martin Kopec 6ecd3a925c Replace freenode by OFTC
As a result of recent event the OpenStack community is moving
to new IRC network - OFTC
http://lists.openstack.org/pipermail/openstack-discuss/2021-May/022718.html

Change-Id: I20d653dabb4f9460b62305a46b3c3a76495bd1f8
2021-06-14 08:23:23 +00:00
Zuul 481eb0b9aa Merge "Update meeting info and remove old links" 2021-05-15 03:10:18 +00:00
Martin Kopec 81928c2e10 Update the guidelines location
This patch updates the guidelines location. The guidelines will be
moved there by [1]. In order to shorten the transition period we
are intentionally doing this *before* [1] is merged (before the
guidelines are actually moved) because it takes some time to get
this change to production. Due to that the patch also implement
a fail-safe mechanism - if the get request for guidelines returns
404, the code will try again with the old (at this moment the
current) location.
When this gets to production, we will merge [1]. Then the
fail-safe mechanism will no longer be needed, therefore it will be
removed in a follow up patch.

[1] https://review.opendev.org/c/osf/interop/+/786116

Change-Id: I5d81d0168601f2b6c3b5cae996b531ebdb33d9d0
2021-05-12 10:12:58 +00:00
Martin Kopec 1ab07795dd Update meeting info and remove old links
README showed old meeting time, the patch updates that and redirects
a user to a Wiki page where the current meeting info can be found.
The patch also removes outdated links from the README.

Change-Id: I48352caa1408960df5bad49cf8b2ecf9b4aef3de
2021-05-12 09:04:03 +00:00
Martin Kopec 65ee83693a Use Xena zuul job template
The template contains also py39 job.

Newer releases of requests-cache are not compatible with
requests package (or the other way around) resulting in:
AttributeError: 'Response' object has no attribute 'from_cache'
Therefore the patch caps requests-cache for now.

Change-Id: If64a52d975bacc43b23ee055a118dec6e3e6b42d
2021-05-03 09:18:04 +00:00
Martin Kopec 6566d87dc7 Add shared-file-system guideline
Refstack backend was missing shared-file-system in the mapping
which resulted in backend not being able to parse the Shared
File System guideline download from the interop repo.
The refstack-ui needs to be edited as well so that it can properly
render the Shared File System guideline on the website.

Change-Id: I017a331ab010f94b223add22a2b55a02b6996db5
2021-03-08 12:21:44 +00:00
Martin Kopec a33296ed0a Target name in the downloaded list's name
When downloading a test list from [1] (click on Test List link)
include the target name in the filename.

Example for the orchestration target:
Current name of the file would be: 2020.11-test-list.txt
The new name will be: orchestration.2020.11-test-list.txt
Or the compute target:
2020.11-test-list.txt -> compute.2020.11-test-list.txt

[1] https://refstack.openstack.org/#/guidelines

Change-Id: I3337120e16e59c9e48a6a0b1d37f877e4b839a13
2021-03-08 12:21:03 +00:00
Martin Kopec 0074bf3b70 [Trivial] Assign value after the encode
In the previous fix [1] the value of pubkey.encode() was
forgotten to be assigned which basically makes it a noop
operation. This patch fixes that.

[1] https://review.opendev.org/c/osf/refstack/+/776168

Change-Id: I93a4e5d8afde7e3abfdcf66220c64cda111d4a48
2021-02-26 01:15:29 +00:00
Martin Kopec 476f4ce243 Encode pubkey in validators
Follow up change for py2->py3 migration.
The pubkey must be encoded in order to avoid 'data must be
bytes-like' error. The key is a type of str which in py2 meant
bytes, however, in py3 it's just string. Therefore the pubkey
must be encoded before it's passed to load_ssh_public_key which
requires bytes-like data.

Change-Id: I943438a59923bd9c4e9dd38058020e49fd2f3d24
2021-02-17 14:10:23 +00:00
Martin Kopec bbbe1f525b Use opendev interop repo for querying guidelines
Refstack server was still downloading guidelines and
capabilities from the old interop repositoty hosted
on github.com. This patch changes default repo URLs
so that the guidelines are download from the current
interop repo hosted on opendev.org.

Change-Id: Ic05a2a784c8e513e09bc430139011aa73bea062e
2021-02-17 11:42:29 +00:00
Martin Kopec 23fcee1239 Remove decode call from db migrations script
This is a follow up on 'py3 only' changes. Previously in py2
current_name variable had unicode type, however in py3 it is
a str type which leads to an Attribute error when the code
is executed by py3 ('str' object has no attribute 'decode').

Now when refstack is py3 only, we don't need to be decoding
current_name variable, the patch removes this conversion.

More about str and unicode and their meaning in py2 and py3:
 * https://portingguide.readthedocs.io/en/latest/strings.html

Change-Id: Ibc76a8797b467d45b05c6e6e61d4bc703a5d701f
2021-01-28 19:11:46 +00:00
Zuul 94b1fa594f Merge "Use unittest.mock instead of mock" 2021-01-25 20:44:23 +00:00
Hervé Beraud da9cb79e93 Use unittest.mock instead of mock
The mock third party library was needed for mock support in py2
runtimes. Since we now only support py36 and later, we can use the
standard lib unittest.mock module instead.

Change-Id: I70a406c060ba681ddc000788b3fa5ab2bba4d41f
2021-01-25 19:38:53 +00:00
Hervé Beraud 4047e51216 Stop to use the __future__ module.
The __future__ module [1] was used in this context to ensure compatibility
between python 2 and python 3.

We previously dropped the support of python 2.7 [2] and now we only support
python 3 so we don't need to continue to use this module and the imports
listed below.

Imports commonly used and their related PEPs:
- `division` is related to PEP 238 [3]
- `print_function` is related to PEP 3105 [4]
- `unicode_literals` is related to PEP 3112 [5]
- `with_statement` is related to PEP 343 [6]
- `absolute_import` is related to PEP 328 [7]

[1] https://docs.python.org/3/library/__future__.html
[2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
[3] https://www.python.org/dev/peps/pep-0238
[4] https://www.python.org/dev/peps/pep-3105
[5] https://www.python.org/dev/peps/pep-3112
[6] https://www.python.org/dev/peps/pep-0343
[7] https://www.python.org/dev/peps/pep-0328

Change-Id: I6bc5768e9c06f266c71188b867d15c3dde417d44
2021-01-22 08:57:16 +00:00
Martin Kopec 052de8653b Replace six by python3 code style
As we have already dropped python2 support,
usage of six module is redundant.
The patch also edits the classifiers in setup.cfg
which comply with the wallaby cycle goals.

Change-Id: I2b0e4050e489ddfc7cbabe4165d5499ba6788422
2021-01-11 20:33:56 +00:00
Martin Kopec a1f2375875 Inclusive jargon
tox deprecated its whitelist_externals option in
3.18 release. The patch bumps the min version of tox
to 3.18 and renames the option from whitelist_externals
to allowlist_externals.

Change-Id: I12d4bb75839adf78c17e8e562535fbf0d7e2e816
2021-01-09 15:26:19 +00:00
Goutham Pacha Ravi 78c4ce8ac8 Fix broken links and redirects
This repository is hosted on opendev.org's
gitea server, and doesn't get mirrored to
github currently (we may add mirroring in
the future). We can swap a bunch of URLs
that assume the content is on github and
older infra cgit servers with correct URLs.

Meetings have moved from IRC to MeetPad,
fix the corresponding details in the README.

Also use this commit to strip trailing
whitespaces across the repository.

Change-Id: Ica0a7ba08d9c437f94fbc9ab282bd929f01df8ff
2021-01-08 21:17:29 +00:00
Martin Kopec 69c4d81773 Comply with the newest release of PyJWT
A new version of PyJWT==2.0.0 got released a few days back which
changes a few significant things such as return types and option
names, see:
03610f0103/CHANGELOG.md (v200)
This commit edits the code of refstack in order to comply with the newest
changes of PyJWT.

Change-Id: I153384cd39ebadb8ee9eb3d99cd21aa02b8c0673
2021-01-08 20:15:27 +00:00
Zuul aa78240eec Merge "fix a typo" 2021-01-06 14:38:16 +00:00
Martin Kopec 35b3a1da4f Fix CI, Drop python2 and 3.5, update jobs to python3
* Starting in mysql8, we can't expect a db user to be
  created with the GRANT command, we'll need to create
  one explicitly.

* OpenStack has dropped the py2.7 and py3.5 support
  in ussuri cycle. From now on, refstack will test only
  with py3 jobs. Complete discussion & schedule can be
  found in
  - http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
  - https://etherpad.openstack.org/p/drop-python2-support

  Ussuri Communtiy-wide goal:
  https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

* Replace the "refstack-tox-py36-func-mysql" with
  "refstack-tox-functional" to simplify naming and use the system default
  python3 version for this functional job.

* replace testr unit test runner by stestr. testr runner hasn't been updated
  for ~ 6 years - stestr is its successor

Story: 2003257
Task: 24062
Change-Id: I90ab2eca6366f0b6256f097712efac2ffe08b2c0
2020-12-18 20:59:21 +00:00