Commit Graph

1025 Commits

Author SHA1 Message Date
Zuul 0b5cb8b368 Merge "Remove unused host_restart_script_dir" 2024-03-17 16:10:32 +00:00
Takashi Kajinami f64f957a34 Remove unused host_restart_script_dir
This has been unused since the SUID binary to restart docker containers
was removed[1].

[1] d60176cc2f

Change-Id: If2050e6a8e44217536cd80504934cd4949583b0c
2024-03-17 15:28:25 +00:00
OpenStack Release Bot d3cdac34e2 Update master for stable/2024.1
Add file to the reno documentation build to show release notes for
stable/2024.1.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/2024.1.

Sem-Ver: feature
Change-Id: I4d87c52977d1569cf3f84a25241fd3ebea2825ca
2024-03-14 02:33:13 +09:00
OpenStack Release Bot d58585737c reno: Update master for unmaintained/xena
Update the xena release notes configuration to build from
unmaintained/xena.

Change-Id: Iecf654eadad27064cc31995d9d18936a5b427c7d
2024-03-12 14:03:01 +00:00
OpenStack Release Bot 96e3e2a668 reno: Update master for unmaintained/wallaby
Update the wallaby release notes configuration to build from
unmaintained/wallaby.

Change-Id: I069d656078890b2a22798101df1ae2bc9b6320c6
2024-03-12 14:02:44 +00:00
Takashi Kajinami 5f254dd3fa Add release note for 13.0.0 release
Change-Id: I3a9acffbb4dc1ae659e2e0db6f186ff8f731a77c
2024-03-12 00:41:44 +09:00
Takashi Kajinami 3c7d0b7f2d Document PUT with X-Storlet-Extra-Resources
Change-Id: I4572ee561e933b598285e122c97f160d3d401c61
2024-02-14 10:12:00 +09:00
Takashi Kajinami 8401b6c581 Allow limiting maximum extra resources
Using many extra resources may cause high load at proxy server, because
the process needs to get all of the requested objects concurrently.
This change introduces a new option to limit maximum number of extra
resources per request to avoid DoS attack by too many extra resources.
If a request contains extra resources over the limit, then the request
is rejected at an early stage. The default value is -1 which means
unlimited. In case this option is set to 0 then users are not allowed
to use extra resources at all.

This also refactors handling of extra-resources header. One side
benefit of the refactoring is that now users can use additional inputs
when executing storlet over PUT requests.

Change-Id: I0ea7d78614f2b1ef5bf4961d2d5fe773264ef448
2024-02-14 01:03:52 +09:00
Takashi Kajinami b75465ab51 Refactor FD handling
* Fix duplicate closure of _input_read_fd
* Delegate closure of _input_write_fd to the dedicated write thread

Change-Id: I74128fe1f55abafea322111cf289fb0a978fc2b4
2024-02-12 17:36:36 +00:00
Takashi Kajinami 639fee6cd2 Decouple StorletData from StorletRequest/Response
Each StorletRequest or StorletResponse can contain multiple StorletData
when using multi input/output feature.

Change-Id: I755583116d55588083ec9703de28344f3e20f7ee
2024-02-13 02:36:02 +09:00
Zuul f1ee4f16a3 Merge "Split out general implementation for containers" 2024-02-12 14:57:22 +00:00
Takashi Kajinami 43c038adf4 gateway: Ensure required interfaces are implemented
... using the built-in abc module.

Change-Id: I70a5ddde1f235bdd208997c3012aa0e8588b32e2
2024-02-11 16:41:27 +09:00
Takashi Kajinami caae8c45a7 Split out general implementation for containers
This is a prep work to support different container runtimes such as
podman.

Change-Id: Ia7787bdb1d86683d316a32b34c0a080b7b4ebb87
2024-02-11 01:37:25 +09:00
Zuul f1e59923ee Merge "Remove unused constant" 2024-02-10 16:23:10 +00:00
Zuul 50c6680a55 Merge "Use built-in subprocess.DEVNULL" 2024-02-10 15:24:46 +00:00
Zuul 037bcf34db Merge "Add CentOS Stream 9 job" 2024-02-10 15:18:49 +00:00
Takashi Kajinami 207f80b73c Remove unused constant
Change-Id: I13d013af686382716a943b4bc1c23157f4ff48b3
2024-02-11 00:08:30 +09:00
Zuul 25c8337393 Merge "Remove dev(el) package from container image" 2024-02-10 14:43:22 +00:00
Takashi Kajinami 16c292305f Add CentOS Stream 9 job
This introduces a functional tests which runs on CentOS Stream 9.

Currently we use docker-ce in this job, but the job may be later
updated to use podman, which is part of the distribution packages.

Change-Id: Ic93c1c7faf46f641aa5988c688ebbef4a6e32ab4
2024-02-10 23:32:09 +09:00
Takashi Kajinami 876b695219 Use built-in subprocess.DEVNULL
... instead of creating a file object.

Change-Id: I6f61369f1770f8223544ec5040deeded7640b81b
2024-02-10 23:30:28 +09:00
Takashi Kajinami 464310c62b Remove dev(el) package from container image
... because it's required only during builds.

Change-Id: Icb8ce4dc11e3c47bb57282e004ce7a37ceb5b80b
2024-02-10 22:21:09 +09:00
Zuul 224421d789 Merge "Pick up jdk version from environment" 2024-02-10 10:47:43 +00:00
Takashi Kajinami 13a505d69d Bump java dependencies
... to the latest versions currently available.

Change-Id: I08933c950ed1d3853cac0ff42ce4fa717f3eddc1
2024-02-10 14:33:11 +09:00
Takashi Kajinami 1315cf9c36 Pick up jdk version from environment
... instead of hard-coding the version in ant build file.

Change-Id: I382803ee7c557e02fc02bda53f140b46aa0f2146
2024-02-10 14:22:55 +09:00
OpenStack Release Bot 40031d0420 reno: Update master for unmaintained/yoga
Update the yoga release notes configuration to build from
unmaintained/yoga.

Change-Id: I70df86fafd317d924dd635f4f1634ef2b0880d3e
2024-02-05 16:55:19 +00:00
Zuul 3e1f097e11 Merge "Fix redundant default posargs for unit tests" 2024-02-02 01:28:26 +00:00
Zuul aa192c23f5 Merge "Update repository structure" 2024-02-01 19:21:43 +00:00
Zuul 773b2817aa Merge "Remove unused .unittests" 2024-02-01 19:21:42 +00:00
Zuul b9499dfb06 Merge "Skip functional tests for change in .gitignore" 2024-02-01 19:03:02 +00:00
Takashi Kajinami cafb36b1af Fix redundant default posargs for unit tests
The same is already defined by .stestr.conf .

Change-Id: I9afd30ef02570cc8eee43dd48f2b9b6a52f05d2d
2024-02-01 19:01:55 +00:00
Zuul bc48403c79 Merge "Skip functional tests when README is updated" 2024-02-01 18:40:03 +00:00
Zuul a1d4e8a20a Merge "Update links in README" 2024-02-01 18:28:50 +00:00
Takashi Kajinami 0ccfe557e5 Remove unused .unittests
We generally use tox to run unit tests.

Change-Id: I94409abebb7670a0174557db95bb385cf7f1eec7
2024-02-02 03:14:18 +09:00
Takashi Kajinami c9e35be319 Skip functional tests for change in .gitignore
Change-Id: I7756b3c6f7d343f7246d1e368799c32c47b9c16c
2024-02-02 03:12:09 +09:00
Takashi Kajinami 77509add21 Skip functional tests when README is updated
Change-Id: I058531f9aade756720085555939ca659f85590b2
2024-02-02 03:07:20 +09:00
Zuul b84c9a634e Merge "Remove unused STORLETS_DOCKER_SWIFT_*_ID" 2024-02-01 17:55:10 +00:00
Takashi Kajinami 7c8a63a274 Update repository structure
The structure currently described is very old and does not match
the current one after refactoring.

Change-Id: I2b39a6f14175f0478c7fc14698f4e477ce05351b
2024-02-02 02:47:49 +09:00
Takashi Kajinami 1b5dd530d9 Update links in README
... to use docs.openstack.org . Also the deployment guide is no longer
maintained so remove the link to it.

Change-Id: I51378ae474e1edc515858ae629aa88a8b4d2c6a4
2024-02-02 02:36:22 +09:00
Takashi Kajinami 9d80bd25a1 Remove unused STORLETS_DOCKER_SWIFT_*_ID
Change-Id: I80b47c126957cea7908dbb8fdbb059a2648c17d4
2024-02-01 21:41:21 +09:00
Takashi Kajinami 7e3b21de8d Ensure '-' in storlet object metadata
... to avoid detecting malformed headers like;
  X-Object-Meta-StorletFoo

Change-Id: I16b6488c48f75cf2519d8dd387449f4fde69f3b1
2024-01-31 12:17:36 +09:00
Takashi Kajinami 88369fee13 Run codespell to fix typos
This fixes the spell fix by;

$ tox -e codespell -- -q 2 -S '*.csv,*.po,*.yaml' -w

Change-Id: I11a3142d6eef40bec2324b98bfe4b0d871779f64
2024-01-30 19:09:27 +09:00
Zuul f565b62fa5 Merge "Skip storlet execution for symlink at object-server" 2024-01-29 09:54:56 +00:00
Takashi Kajinami c227b51739 Refactor functional tests with broken python storlet
... and also add a test case to cover proxy execution scenario.

Change-Id: Ic2c508ca57d37878b44f97425f326d7d7e46f506
2024-01-29 18:16:56 +09:00
Takashi Kajinami b43dca80a2 Skip storlet execution for symlink at object-server
... because the actual object content is not available in symlink.

Change-Id: I5e4ba77eb5c689b83a7503682ccabddea8aa1608
2024-01-29 18:10:03 +09:00
Zuul 3eb72a4bda Merge "Refactor SLO tests" 2024-01-29 06:16:20 +00:00
Takashi Kajinami 50a4d4f73a Refactor SLO tests
* Use context manager to open files
* Response codes are already checked within swiftclient methods
* etag is now returned by put_object method

Change-Id: I446118a314929c7d646c66bc2d8a0f1fa41e4fed
2024-01-29 14:29:22 +09:00
Zuul dc90c16808 Merge "Generalize container related options" 2024-01-29 02:23:38 +00:00
Zuul 6b80385cda Merge "Add option to limit pids within a container" 2024-01-29 01:51:36 +00:00
Zuul c17b6ef0e6 Merge "Remove unused bin/ files" 2024-01-29 01:25:28 +00:00
Takashi Kajinami 5818bc046d Add option to limit pids within a container
Because we run container side applications using the swift user, we
need a mechanism to limit number of processes launched inside storlet
containers so that user workloads do not consume all allowed pids in
host.

This introduces a new option to set pids limit in each container.

Change-Id: Idc07807ab7dba12c795d19d6405fc998e7b09893
2024-01-29 10:01:01 +09:00