Commit Graph

6768 Commits

Author SHA1 Message Date
Tony Breeds 74ce40a1f7 Retire Packaging Deb project repos
This commit is part of a series to retire the Packaging Deb
project. Step 2 is to remove all content from the project
repos, replacing it with a README notification where to find
ongoing work, and how to recover the repo if needed at some
future point (as in
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project).

Change-Id: Ice2229c7d55b29f6896ded5bcc96c0657a80fe40
2017-09-12 16:20:13 -06:00
Jenkins 7c3d218b2b Merge "Stop using deprecated 'message' attribute in Exception" 2017-07-24 18:32:20 +00:00
Jenkins d40c9ed3a2 Merge "Close stdout *after* binding ports" 2017-07-23 01:12:33 +00:00
Tim Burke 31f4794c12 Close stdout *after* binding ports
Otherwise probe tests think services are up when they aren't actually
available yet.

Related-Change: I045f4c9aa2c07cdc5ed95afd028540f1b3874637
Change-Id: Iab923c4f48ac7a5dd41237761ed91d01a59dc77c
2017-07-22 00:46:16 +00:00
Jenkins 4e4028e59f Merge "moved install guide and removed tox env definition" 2017-07-21 23:21:52 +00:00
Jenkins c72de31759 Merge "Replace dict.iteritems with six.iteritems" 2017-07-21 21:37:12 +00:00
Alistair Coles 5b10cf530b Add more structure to the deployment guide
Previously it was hard to navigate to a particular config section in
the deployment guide, and not possible to provide a link directly to
one section.

This patch makes each config section a heading so that it appears in
navigation tables and can be easily linked to. A list of config
sections is also added at the start of each server section.

Change-Id: Iecb0637fde521600a9163fa66b3dbdc176a71dff
Related-Bug: #1626290
2017-07-20 17:01:36 +01:00
bhavani.cr 35cf1891f3 Stop using deprecated 'message' attribute in Exception
The 'message' attribute has been deprecated and removed
from Python3.
For more details, please check:
https://www.python.org/dev/peps/pep-0352/

Change-Id: I152e79352705a3e43ce464a8f087b0e93f072035
2017-07-20 11:33:31 +00:00
Alistair Coles 9c5628b4f1 Add reconstructor section to deployment guide
Change-Id: I062998e813718828b7adf4e7c3f877b6a31633c0
Closes-Bug: #1626290
2017-07-20 11:40:17 +01:00
Jenkins 3c11f6b8a8 Merge "Make dict deletion idempotent in dump_recon_cache" 2017-07-18 03:28:16 +00:00
Jenkins 269d314dd8 Merge "Fix StopIteration noise in obj/test_replicator.py" 2017-07-18 03:28:08 +00:00
Jenkins c66fff645f Merge "Add tests for Recon's object replication_time time unit" 2017-07-18 03:28:00 +00:00
Jenkins 83b62b4f39 Merge "Add Timestamp.now() helper" 2017-07-18 03:27:50 +00:00
Alistair Coles c2e59b9b8b Make dict deletion idempotent in dump_recon_cache
Calling dump_recon_cache with a key mapped to an empty dict value
causes the key to be removed from the cache entry. Doing the same
again causes the key to be added back and mapped an empty dict, and
the key continues to toggle as calls are repeated. This behavior is
seen on the Related-Bug report.

This patch fixes dump_recon_cache to make deletion of a key
idempotent. This fix is needed for the Related-Change which makes use
of empty dicts with dump_recon_cache to clear unwanted keys from the
cache.

The only caller that currently set empty dict values is
obj/auditor.py where the current intended behavior would appear to be
as per this patch.

Related-Change: I28925a37f3985c9082b5a06e76af4dc3ec813abe
Related-Bug: #1704858
Change-Id: If9638b4e7dba0ec2c7bd95809cec6c5e18e9301e
2017-07-17 17:14:50 -07:00
junboli e1a2c28ec3 Fix redundant UTF-8 check
In the file proxy/server.py, handle_request method does UTF-8
check twice. This patch is to fix it.

Change-Id: I6db742bce6fa4fa9e935b15460037e0ec90e9373
Closes-Bug: #1704539
2017-07-15 17:05:15 +08:00
Jenkins ad3f1667ce Merge "Add X-Copy-From-Account to api-ref" 2017-07-14 15:32:18 +00:00
Tim Burke 95ff34d4ed Add X-Copy-From-Account to api-ref
While Donagh was kind enough to add Destination-Account in the related
change, we still hadn't documented the PUT-with-X-Copy-From equivalent.

Change-Id: I156ae2d8664873d3f6cc1f742bf950913fd462b0
Related-Change: I315b4e550b7d10880fbc00fce9311127ba609c2d
Closes-Bug: 1367975
2017-07-12 22:30:43 +00:00
Tim Burke 13a07aa77a Misc doc cleanup
* Change some absolute URLs to internal links
* Fix some bulletted list indentation
* Choose a better lexer for some syntax highlighting
* Use ``inline code`` instead of `italics` for some example command
  lines
* Change some quoted paragraphs that only included inlined code to be
  proper code blocks

Change-Id: Iaaa7eefb690122f5af9dcb1c871358c22335c743
2017-07-12 12:14:45 -07:00
Tim Burke 63c4215994 Fix rst for link
Change-Id: I2f63ce16ae67277504a47e68c7dca3c3412fe92c
2017-07-11 18:38:43 -07:00
Jenkins 5cb1303d5b Merge "Add small note for handoffs_only acceleration" 2017-07-11 19:46:00 +00:00
Alistair Coles 74700eb890 Fix StopIteration noise in obj/test_replicator.py
Insufficient arguments are passed to create MockProcess instances
resulting in StopIteration errors being raised during the repeated
replicator run_once cycles added in [1].  The test passes because
the replicator just logs these exceptions, but the logger noise is
distracting when running the test [2].

[1] Related-Change: Ib5c9dd17e40150450ec57a728ae8652fbc730af6
[2] nosetests ./test/unit/obj/test_replicator.py:\
TestObjectReplicator.test_run_once -s

Change-Id: I36208e93c81744068a3454577a30d0c5a8d9cb9b
2017-07-11 15:41:48 +01:00
Jenkins 263bab9818 Merge "Clean up some assertions in test_reconstructor" 2017-07-11 06:19:33 +00:00
Jenkins 995e009c5e Merge "Untangle ServersPerPortStrategy.bind_ports" 2017-07-11 04:16:01 +00:00
Kota Tsuyuzaki 65b6b12165 Add small note for handoffs_only acceleration
The related change benefitts us greatly with decreasing hash
calculations for ignored partitions when handoffs_only mode is on
however, at the same time, it has been not able to care misplaced
fragments in the primary partition so that add the notation into the
code to let us know as "this is by design".

Change-Id: I16e3c1170af710e2a6ecc0b7267ab369450095bf
Related-Change: 44c63c6990
2017-07-10 18:58:36 -07:00
Tim Burke 20a0d67340 Clean up some assertions in test_reconstructor
While reviewing https://review.openstack.org/#/c/464982/ I noticed that
assertions like

    self.assertEqual(len(captured_ssync), 4)

aren't terribly helpful when they fail.

Change-Id: I5c5df7ed60e58c1d1bca5a5bfef9352a39a41f2f
2017-07-10 15:32:07 -07:00
John Dickinson 239dfb559c moved install guide and removed tox env definition
Change-Id: I966f849804cd5eeeae566bc7e732c7bf56bc9eaf
2017-07-10 13:49:28 -07:00
Jenkins 8472cb6538 Merge "Fix sporadic failures in test_reconstructor.py" 2017-07-10 17:18:53 +00:00
Jenkins ded0de7aa5 Merge "Don't rehash primaries in reconstructor handoffs_only mode" 2017-07-10 16:16:00 +00:00
Clay Gerrard 44c63c6990 Don't rehash primaries in reconstructor handoffs_only mode
The reconstructor handoffs_only needs to aggressively avoid erroneous
I/O related to rehash of primary suffixes.

While in handoffs_only mode the reconstructor won't even look at primary
partitions.

This has a *huge* impact on cycle time once the node has completed
processing handoffs; which results in a much faster and stronger signal
that that it's either time to rebalance again or turn off handoffs_only.

Related-Change-Id: Idde4b6cf92fab6c45f2c0c2733277701eb436898

Change-Id: If4bbb778d511efe13713590639c8b91615556f22
2017-07-07 15:16:00 -07:00
Tim Burke d112b7d29d First pass at admin-guide cleanup
Change-Id: I005232d95a3e1d181271eef488a828ad330e6006
2017-07-07 17:33:54 +00:00
John Dickinson 4cb76a41ce docs migration from openstack-manuals
Context for this is at
https://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html

Change-Id: I9a4da27ce1d56b6406e2db979698038488f3cf6f
2017-07-06 16:44:25 -07:00
Jenkins 37a1935198 Merge "Switch from oslosphinx to openstackdocstheme" 2017-07-06 18:16:43 +00:00
Jenkins c3f6e82ae1 Merge "Write-affinity aware object deletion" 2017-07-06 14:00:05 +00:00
Jenkins e94b383655 Merge "Add support to increase object ring partition power" 2017-07-05 14:40:42 +00:00
liuyamin 006a378193 Add license in swift code file
Source code should be licensed under the Apache 2.0 license.
Add Apache License in swift/probe/__init__.py file.

Change-Id: I3b6bc2ec5fe5caac87ee23f637dbcc7a5d8fc331
2017-07-05 10:07:11 +08:00
Jenkins f1e1dbb80a Merge "Make eventlet.tpool's thread count configurable in object server" 2017-07-04 11:49:24 +00:00
Jenkins c22bab4b34 Merge "Version DLOs, just like every other type of object" 2017-07-03 14:06:07 +00:00
Van Hung Pham bfb5759a53 Switch from oslosphinx to openstackdocstheme
As part of the docs migration work[0] for Pike we need to switch to use
the openstackdocstheme.

Fix one display problem with wrong section markup in the index file.

[0]https://review.openstack.org/#/c/472275/

Change-Id: Ide31218a7f37ba5d959de99cab48fc6513bf426f
2017-07-02 09:27:05 +02:00
Jenkins 638fcc152b Merge "Bind SAIO services on different loopback addresses" 2017-07-01 05:17:34 +00:00
Clay Gerrard 58d7812596 fix flakey time for test_default_sorted_output
Change-Id: Ib7f0c22336e8354d4f46e2343149495bef382f9c
2017-06-30 16:45:43 -07:00
Jenkins fe3dcf5007 Merge "Close all versioned_writes subrequests' app_iters" 2017-06-30 18:44:44 +00:00
Jenkins 3361cd083e Merge "Order devices in the output of swift-ring-builder" 2017-06-29 22:06:37 +00:00
Clay Gerrard 14b07ca726 Untangle ServersPerPortStrategy.bind_ports
The "method" `bind_ports` only gets called once on startup - and *during
it's execution* it gets mutated to a list inside `_reload_bind_ports` -
afterwards all references are only to the attribute `bind_ports`.

It 100% works, but at first glance seems dubious.  Renaming the method
do_bind_ports seems more obvious.

Change-Id: Ic01621e478f1140644e26ba41715d93963a00945
2017-06-29 13:34:43 -07:00
Jenkins 537f9a3f64 Merge "Delay port binding to reduce wait at process start" 2017-06-29 16:39:57 +00:00
Jenkins 44f6059394 Merge "Test: Use assertIsNone() in unittest" 2017-06-29 14:50:02 +00:00
Jenkins fc06f6b7af Merge "Fix the reST field raises in docstrings" 2017-06-29 14:49:38 +00:00
Clay Gerrard 806b18c6f5 Extend device tier pretty print normalization
* use pretty device tier normalization in additional debug message
* refactor pretty device tier normalization for consistency
* unittest asserting consistency

Change-Id: Ide32e35ff9387f1cc1e4997eb133314d06b794f3
2017-06-28 15:40:34 -07:00
Samuel Merritt 62509cc8f4 Improve debug logging from ring builder.
The gather/place debug logs used to just contain device IDs; now they
include region, zone, and IP. This makes it easier to see what's going
on when debugging rebalance operations.

Change-Id: I6314e327973c57a34b88ebbb4d3b1594dbacd357
2017-06-28 15:09:07 -07:00
liuyamin 1eeb354c27 Fix the reST field raises in docstrings
Probably the most common format for documenting arguments is reST field
lists [1]. This change updates some docstrings to comply with the field
lists syntax.

[1] http://sphinx-doc.org/domains.html#info-field-lists

Change-Id: I0c35c6b4df840018534737bca2ca32dc977b0e05
2017-06-28 09:10:24 +08:00
Matthew Oliver e11a38c63a Bind SAIO services on different loopback addresses
Currently all devices in the ring and all services in a SAIO
all bind to the same loopback address 127.0.0.1. But this
breaks servers_per_port if you want to do any testing on that.

This change binds each service to a different loopback address
and updates the rings (remakerings) accordingly.

To make sure rysncd binds correctly the bind address needed
to be changed to listen on all addresses (0.0.0.0).

Change-Id: I7e77434f275df1e2699de495d8b622b90157a9d7
2017-06-27 16:08:08 -04:00