Commit Graph

438 Commits

Author SHA1 Message Date
OpenDev Sysadmins 1395515c82 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:51:28 +00:00
Phil Bridges 9608a30508 Update documentation for HPSS client API integration
A few things have changed, so I'm touching up the docs before
I leave.

Change-Id: I0b860fa9a8819c4c61922faa8601e3c72d237ee3
2018-07-03 13:09:04 -05:00
Phil Bridges 70984cdd9f Use HPSS client API bindings instead of FUSE wrapper
We're doing this to more easily support future development and advanced features,
like parallel I/O and HPSS identity mappings.

Change-Id: I0192ceb0d65f0d51d9aa77f1f21800cb29e5a262
2018-07-02 14:40:16 -05:00
Phil Bridges e6dc857543 Fixup build for Zuul
Change-Id: Ief549956b954ac1c6d36f40c91d5ba67ba5648f7
2018-06-29 16:34:47 -05:00
Luong Anh Tuan e9878374b4 Replace assertEquals with assertEqual
The method assertEquals has been deprecated since python 2.7.
http://docs.python.org/2/library/unittest.html#deprecated-aliases

Also in Python 3, a deprecated warning is raised when using assertEquals
therefore we should use assertEqual instead.

This is part of blueprint assertequal

Change-Id: Ie8887ba468bee10ddeb5b7a1d095821f898b9302
2016-11-21 15:57:21 +07:00
Christopher Bartz cdb589c32d Config option to specify default COS
Currently, in certain HPSS configurations, it is not possible to make
a PUT without adding the Header "X-HPSS-Class-Of-Service-Id".
With this commit, a default COS can be specified in the config, in order
to let the user allow to omit the Header on each PUT request.

Change-Id: I4ad2fadc95cf557a1d226e336013324c339d51b6
2016-08-02 12:40:35 +02:00
Christopher Bartz dc62c40a01 Config option to (dis)allow purgelock header
Before this commit, users could submit a file with a purge
lock by adding a specific HTTP header.
This is not desired for every HPSS deployment.
This commit adds a boolean option, which indicates if the
header should be ignored or processed.

Change-Id: I39a9da6537fd4984e9af01c0235ec10f938b8238
2016-07-13 17:26:38 +02:00
Phil Bridges baf6a24490 Fix fd leak on HEAD requests
The previous cleanup patch ended up taking too much code, and caused
an fd leak on any HEAD request because I had intended to clean up a
bad use of DiskFile's _close method in the ObjectServer. Fixed that.

Change-Id: I19da95ebdfa146fe90d5f126f470a2d42c35ee33
2016-06-23 18:17:17 -05:00
Phil Bridges d0c884c143 Update copied Swift tests for 2.5.0.0 release
There were a few tests that were out of date compared to upstream
Swift, so I pulled just enough to make everything be green.
I have no idea why we have our own copy of these tests to maintain,
but we do. Maybe we should look into getting rid of them and just
linking straight to whatever Swift tests we have?

Change-Id: I1eb7d891fa54c8627b8aac93bfefbd413df8529d
2016-06-23 17:38:03 -05:00
Phil Bridges ad9cd13c2c Fix 'dummy' HPSS ioctl code
In order for the unit test suite to run happily without HPSS installed,
some dummy implementation of the ioctl() call normally given by the
'hpssfs' module needed to be added. This was done when bringing the code
into a known good state, but the implementation would not actually work
and that seemed like it would be an issue down the road. So here's an
actually working implementation of that.

Change-Id: I52c8ede8ac2d38bb59c2d7d2fb399fd65e75577a
2016-06-21 18:29:34 -05:00
Phil Bridges de609374f7 Clean up debugging cruft
In the process of working on SwiftOnFile, there was a bunch of
unwanted debugging messages and cruft just to trace dataflow.
I don't think this should be in the release build, so cleaning
it up now.

Change-Id: I7fdb1de48cb97ec7e1629102d032f722210675fc
2016-06-20 23:07:19 +00:00
Phil Bridges 3c7bcc0c7a Fix Gerrit gates and other build issues
There are a few problems with the configuration in our source tree that
causes the Jenkins gates to always fail, and things that might cause
a problem later. This patch set will fix those and other problems, to bring
us into a known valid state for future commits.

Change-Id: Idf7a0ce5902c40985caa78390b01f9fc2dfbfcf4
2016-06-20 17:17:20 -05:00
Phil Bridges 8127749f17 Add description to setup.py 2016-06-06 17:51:27 -05:00
Phil Bridges 2e36b46504 Change metadata URL, and prevent system test from being installed into site packages 2016-06-06 17:30:28 -05:00
Phil Bridges 363a319651 Merge branch 'master' into hpss-integration 2016-06-06 17:17:50 -05:00
Phil Bridges 6017c03c51 fix versioning scheme for openstack import 2016-06-06 17:09:57 -05:00
Phil Bridges e7210dfb6e Pile of patches for pushing out to the wider world; adopted Prashanth Pai's dup fd fix patch (https://review.openstack.org/#/c/289773/), changed metadata serialization to dodge an HPSS UDA bug, and fixed up the package metadata some among other things 2016-06-02 18:19:58 -05:00
Phil Bridges 3b7796f5b1 add HPSS-specific functional tests 2016-05-16 18:04:37 -05:00
Phil Bridges d5f47f45e4 update tests for SwiftOnHPSS 2016-05-16 17:57:33 -05:00
Phil Bridges 33b930271e attempt to fix issues with HPSS-specific features; this won't work 100% until we get the HPSS API instead of FUSE 2016-05-16 17:38:02 -05:00
Phil Bridges 37ce2e360d modified documentation for SwiftOnHPSS 2016-05-16 17:29:45 -05:00
Phil Bridges 408503ca8f Revise deployment guide a bit 2016-03-18 18:50:31 -05:00
Phil Bridges cb2c1022dc Add performance tests to SwiftOnHPSS 2016-03-18 14:10:48 -05:00
Phil Bridges ef2d07ea48 Moved some HPSS-specific things from the ObjectServer into the DiskFile, where it belongs.
Also, fixed an issue with object uploads failing due to ETag calculation not keeping up.
2016-03-18 12:02:41 -05:00
Phil Bridges 2b7b787d56 Add manpage installation step to installer 2016-03-18 12:00:37 -05:00
Phil Bridges c7a25662de Fixed HPSS metadata headers not appearing in a GET or HEAD request 2016-03-09 14:04:57 -06:00
Phil Bridges 56238e890c Add more documentation 2016-02-11 18:38:46 -06:00
Phil Bridges 3f19b07d3a Don't calculate ETag ourselves when our FUSE process does it for us; add namespace sync tool 2016-02-10 18:59:19 -06:00
Phil Bridges 58fde922f8 Update README.md 2016-02-01 10:38:18 -06:00
Phil Bridges 2d4cc93deb Adopting change 151897 (https://review.openstack.org/#/c/151897/), previously abandoned.
This is in order to support a tool that will synchronize the names on the HPSS filesystem to the names in Swift metadata.
2016-01-29 15:32:59 -06:00
Phil Bridges 14599e460b Merge upstream updates to support the new Liberty release
Conflicts:
	swiftonhpss.spec
	swiftonhpss/swift/__init__.py
	test/functional/tests.py
2016-01-05 10:27:31 -06:00
Prashanth Pai 328c5549bf Update spec file for 2.5.0 release
Change-Id: Ie72ce4cbdf8d0c0dfe3fa7fd3e4a98b7252ab684
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-12-23 11:40:16 +05:30
Prashanth Pai c649130a1f Set version to 2.5.0
Change-Id: I2f9ed5aec822d01ad77f36a70e293953bfe19e4d
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-12-23 10:42:16 +05:30
Jenkins b562c56729 Merge "Rebase to Swift 2.5.0 (liberty)" 2015-12-22 18:42:01 +00:00
Prashanth Pai a8f474d844 Rebase to Swift 2.5.0 (liberty)
Change-Id: I2743c6fd2b2579751b5d03dacf320401f08563b1
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-12-14 12:43:56 +05:30
Phil Bridges 4a578f8c93 Add HPSS-specific logic to SwiftOnFile, and kick this fork off! 2015-12-11 15:51:00 -06:00
Ondřej Nový 7a6d68e7a3 Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching

Change-Id: Ic576b1e453b1884beb5b30d920baf78f2023f69b
2015-12-11 21:28:52 +01:00
Jenkins 55857a2e21 Merge "doc: Last write wins behaviour" 2015-12-08 14:00:40 +00:00
Prashanth Pai 0475666746 doc: Last write wins behaviour
Change-Id: I1ef2dbc167921a9b08c8b0cd7f240082adc16a0e
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-12-08 19:01:50 +05:30
Jenkins ea0ecdf985 Merge "Introduce fallocate support" 2015-12-08 12:55:50 +00:00
Jenkins cc1051f031 Merge "Fix func test incorrect test result" 2015-12-08 12:51:46 +00:00
Prashanth Pai 9e65dd9ceb Introduce fallocate support
fallocate() allows for reserving disk space for a particular inode and
fd. Hence, a client can be sure that he won't see a ENOSPC (eventually
a 507 HTTP response) later during writes. Swift's object server has
had fallocate() support from a long time.

P.S: Older versions of glusterfs (<3.6) did not support fallocate
because FUSE did not support it earlier.
http://review.gluster.org/4969
http://fuse.996288.n3.nabble.com/fallocate-support-in-FUSE-td10668.html

Change-Id: Ida4b16357901707d624f92bf1b2dc8f07da4f1ad
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-12-08 11:06:55 +05:30
Prashanth Pai b8eb887b81 Fix func test incorrect test result
Running `tox -e functest` when swift services are not running
is incorrectly reported as success! The fix for this borrowed
from here: https://review.openstack.org/235933

Also, sphinx package is not needed in swiftonfile as we do not
have docs to build.

Change-Id: I675a438367497bc9a3c4aca21a0e48458673ab04
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-11-20 11:25:03 +05:30
Prashanth Pai 60eaebbb01 Remove redundant syscalls
This will optimize the first GET on files added from file interface.
More info: https://gist.github.com/prashanthpai/62e0bec770421561ea79

Change-Id: I3f0fd897eedf1413c3e7d5dca0f6196c62549fcb
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-11-20 11:12:30 +05:30
Prashanth Pai 8bce87a2bb Minor cleanups
* Make bin/* scripts pep8 compliant
* Cleanup tox.ini
* Update references of old repo
* Add swiftonfile-migrate-metadata to spec file

Change-Id: Icfa29cffcedfc357ab860a9023b3adfdbf3eeee8
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-11-17 17:55:23 +05:30
Jenkins 3115575d5d Merge "Detect file change when file size remains same" 2015-11-10 19:42:01 +00:00
Prashanth Pai 435a0dcfc7 Do not use pickle: Use json
Change-Id: Ic4217e82fc72f9a32c13136528e1bc3cc05d0c73
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-11-04 18:53:41 +05:30
Prashanth Pai fbc07d3ad9 Refactor read_metadata() method
This change:
* Simplifies read_metadata() method.
* Validates pickle header before attempting to unpickle.

Change-Id: I08d4187f7f4cc963d095b2cd2bcee3039a7dc858
Signed-off-by: Prashanth Pai <ppai@redhat.com>
2015-10-21 15:46:27 +05:30
Jenkins eb9abdc934 Merge "Change repositories from stackforge to openstack" 2015-10-19 18:19:26 +00:00
Jenkins fc1ee9298f Merge "Remove keyword arguments from os.open call, instead add check for mode bits" 2015-10-19 18:19:24 +00:00