If we have import methods enabled, we must have a staging directory
in order to complete those operations. This adds a startup check
for the staging directory with a warning log message if it is not
found.
Change-Id: Iac31d7f15ec6783a59b5e314883fb58f88fe2fd1
This teaches glance-api how to do async threading things when it is
running in pure-WSGI mode. In order to do that, a refactoring of things
that currently depend on eventlet is required.
It adds a [wsgi]/task_pool_threads configuration knob, which is used
in the case of pure-WSGI and native threads to constrain the number
of threads in that pool (and thus the task parallelism). This will
allow tuning by the operator, but also lets us default that to just
a single thread in the backport of these fixes so that we can avoid
introducing a new larger footprint in the backport unexpectedly.
Partial-Bug: #1888713
Depends-On: https://review.opendev.org/#/c/742047/
Change-Id: Ie15028b75fb8518ec2b0c0c0386d21782166f759
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I44e7b6f76e2d12f620ec602afc77ce11ba6b9d9a
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Added new periodic job which will run as per interval set using
'cache_prefetcher_interval' configuration option and fetch images
which are queued for caching in cache directory.
DocImpact
Change-Id: If2875f7a215aca10a6ed2febc89b02219f90a10d
We would like to fully remove mox from the test tree. Even for tests
that don't use mox's validation, many of them are using the symbol
patching with self.stubs.Set. We can do the same thing with the
monkeypatch fixture instead.
This introduces self.stub_out to nova/test.py and an example of what a
stubs => stub_out change would look like.
The teardown function in the converted test was removed at the same
time, as those should no longer be used.
Part of the mox community goal for Rocky.
Change-Id: I8f471ff8fee600ebb4e8907bf240007b7b4fe59f
Signed-off-by: Chuck Short <chucks@redhat.com>
Replace mox testing library by mock one in the file
glance/tests/unit/api/test_cmd.py
Change-Id: I2dde0ca59c4a80bd35c6c6b74c4d3231a97eaf74
partially-implements: blueprint replace-mox-with-mock
Replace the known exception tuple with a map of exception classes
to error codes to preserve backward compatibility of exit codes.
Also change the code to handle unknown members of the Exception
hierarchy without breaking.
Co-authored-by: jiangpch <jiangpengcheng@navercorp.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Closes-Bug: #1726213
Change-Id: Iabfc2ded45a576a18bdb30a6c3ada8b9799a3196
Config option 'default_store' in group 'glance_store' only allows value
in [file, filesystem, http, https, swift, swift+http, swift+https,
swift+config, rbd, sheepdog, cinder, vsphere] due to the parameter
choices, oslo.config will ensure that, other value will lead ValueError.
So don't need test this case.
Related-Bug: #1517839
Change-Id: I3459a8e9f07ed9464073133931fadc77f8186602
Add the ability to dynamically change configuration settings of a running
glance server with no impact to service.
A running glance server consists of a parent/master process and one or
more child/worker processes.
On receipt of a SIGHUP signal the master process will:
* reload the configuration
* send a SIGHUP to the original workers
* start (a potentially different number of) new workers with the new
configuration
* its listening socket will *not* be closed
On receipt of a SIGHUP signal each original worker process will:
* close the listening socket so as not to accept new requests
* complete any in-flight requests
* complete async requests (V1 create with copy-from option and V2 task api)
* exit
This approach is based on nginx's behaviour and avoids some of the
disadvantages of the current oslo Launcher reload:
* Race conditions: Launcher does not shutdown eventlet cleanly, existing
requests can fail.
* If all workers are busy (likely if #workers is small) there can be a
lengthy delay when new requests are not processed.
* Long lived pre-SIGHUP idle client connections can stall request
processing indefinitely.
* Not all parameters can be changed, eg number of workers.
* The wsgi pipeline cannot be changed, eg to enable caching.
Change-Id: I73875c654410f207e3efdb2932147cf26d75b3f1
Partially implements: bp sighup-conf-reload
oslo_config was moved out of the oslo namespace in oslo.config>=1.6.0.
Related-to: blueprint drop-namespace-packages
Change-Id: I30ecbf8f9de77496fcb643c7ad9738d79ad359f0
This commits removes the old `store` package from glance and adopts the
usage of the new glance.store library. The library was designed to
preserve backwards compatibility as much as possible. In fact, most of
the changes in this patch are related to function args ordering and not
function renames or workflow changes.
Some changes that are worth mentioning:
1. Glance store doesn't rely on a global config object. All config
options must be explicitly registered.
2. All store operations now accepted an optional context. This is a
fallout from the context not being required in the `Store` constructor
anymore.
3. Store drivers are behind a private package called `_drivers` and
they're not suppose to be accessed directly. Instead, functions like
`get_store_from_scheme` should be used.
4. Stores are disabled by default
5. All the store specific options are under the `glance_store` group.
DocImpact:
The old store related configuration options have been moved under the
`glance_store` section. However, the old options will go through a
deprecation path. That is, they'll still be read from the `DEFAULT`
section to give deployers enough time to update their config files.
In k-2, the deprecated options will be completely obsolete.
Closes-bug: #1291848
Implements-blueprint: create-store-package
Change-Id: Iaacc70993ad5da292b93de42bbecda73d53b19fd
to keep Python 3.x compatibility, use six.StringIO/BytesIO to
replace StringIO.StringIO
StringIO works for unicode
BytesIO works for bytes
Change-Id: I93e043c633e1de9e4dedcb0a313032403b6a70fb
Closes-Bug: #1280100
Related to blueprint glance-cmd+unittests
Integration test test_cache_pruner_main added to cover
chain of calls between glance/cmd/cache_pruner.py and glance/image_cache
module.
Unit test test_cache_pruner_main_runtime_exception_handling added to
cover exception handling for main() method of cache_pruner.py
Change-Id: I7d7b182260a407aed1288a5d136e03cbe029523f
Integration test test_cache_cleaner_main added to cover
chain of calls between glance/cmd/cache_cleaner.py and glance/image_cache
module.
Unit test test_cache_cleaner_main_runtime_exception_handling added to
cover exception handling for main() method of cache_cleaner.py
Change-Id: Ic05f35dd3f0a93d4331ccb5e470cdb71d9eab605
run run_tests.sh or tox to test the glance code, it will get
following failure:
test_get_bind_addr_default_value
test_get_path_non_exist
Analysis
--------
The root cause is resided in "glance/tests/unit/api/test_cmd.py".
In TestGlanceApiCmd.setUp, the sys.argv is overrided to ['glance-api'].
Since sys.argv is "global" variable, and this test entry it run
firstly by nosetests while running tests in "glance/tests/unit"(since
nosetests search test items alphabetically), the other test entries
will inherit the sys.argv value ['glance-api']. Finally the system
config file "/etc/glance/glance-api.conf" is read in "setUp" via
oslo.config.cfg:ConfigOpts.__call__ -> _pre_setup -> find_config_files.
CONF.bind_port and CONF.bind_host is set to the value reading from
system config file: /etc/glance/glance-api.conf.
A simplied call backtrace:
TestGlanceApiCmd.setUp --> sys.argv = ['glance-api']
->JSONRequestDeserializerTest.setUp
->glance/common/config.py:parse_args
->oslo.config.cfg:ConfigOpts.__call__
->CONF.bind_port CONF.bind_host is set
wsgi.get_bind_addr() will read the CONF.bind_port and CONF.bind_host
to return value. if system config file is not read, CONF.bind_host
is set to "0.0.0.0", CONF.bind_port is not set. wsgi.get_bind_addr()
will return the expected value. This is the reason that individual
test of test_get_bind_addr_default_value got passed, but failed while
testing all.
The failure of test_get_path_non_exist is induced by same reason.
Test case should clear up the setting in tearDown(), especially the
"global" setting, e.g. sys.argv.
Solution
--------
The solution is simple:
In class TestGlanceApiCmd, add a private variable to backup the
sys.argv in setUp(), and restore it in tearDown().
Test method
-----------
$ ./run_tests.sh
$ tox -evenv -- nosetests \
--tests=glance.tests.unit.common.test_config:TestPasteApp
tox -evenv -- nosetests
--tests=glance.tests.unit.common.test_wsgi:JSONRequestDeserializerTest
Fixes Bug #1204186
Change-Id: I2dac8e506fa794d1faa9509258936d78e2f981d8
This change improves coverage while saving ~10 seconds per test run.
Part of bp:faster-better-stronger-functional-tests
Change-Id: Id4d68ccb223205f27bed0c1e9d3072b51310b979
Dan Smith