This is a port from master branch fix
http://review.gluster.org/5670 to grizzly branch.
For whatever reason, it appears that GlusterFS, or perhaps FUSE can
continuously return ENOENT on a rename system call even when we have
double checked that there is no reason to do so. That is a bug for
that sub system. However, our response to that bug can result in an
infinite loop, which is bad.
This code reduces that to 10 attempts.
In addition, we restructed the open retry loop to match, providing
module constants for the upper bounds of both retry loops.
BUG: 1005379 (https://bugzilla.redhat.com/show_bug.cgi?id=1005379)
Change-Id: Ia2d6dd427daba3ea0461863c5ffe3aef27c88f9b
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5670
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5848
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Change-Id: Ib9b04c2733cac954b6acb87ddaaeb175f25319d1
Original-author: Peter Portante <peter.portante@redhat.com>
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/5698
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Our initial implementation only required Jenkins
to export binary RPMs, but as we move foward, we really
need to also export SRPMs. To support SRPMs, the spec
file in the RPM has to have the correct NAME, VERSION,
and RELEASE information.
Change-Id: Icd7132b4aafdbe7a1f02a35d0be7ad63b2e7c056
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5669
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
gluster-swift-gen-builders now changes directory to /etc/swift prior to
creating ring files. It does display a message indicating the location of
the ring files.
BUG: 986458
https://bugzilla.redhat.com/show_bug.cgi?id=986458
Change-Id: Iea33c925d98c47c914e8a95d11faa3220967b4d6
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5658
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5667
A gluster volume could yield an ENOSPC condition seeing
that a volume is full. This needed to handled correctly.
Added error handling.
BUG: 985253
https://bugzilla.redhat.com/show_bug.cgi?id=985253
Change-Id: I85472c0a81a354a2796327fead606da3a938d4bf
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5362
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5570
Copyrights are updated across the code to reflect 2013.
Copyrights message was added to couple of scripts.
Change-Id: I2cfc7bdc6adaa5c9ab0075aab49021a5f0dea7a1
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5371
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5456
gluster-swift-gen-builders copies the older ring files to
/etc/swift/backup_ring. And then creates the new ring files to /etc/swift
BUG: 986458
Change-Id: I06e93179e46da56d5046419e66c8864aa08f184d
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5384
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5457
* Container and accounts performance increase by removing
the need to update either the object count or the
container count, respectively. New hidden
configuratoins added to re-enable the functionality.
* object_only configuratoin removed from fs.conf and
replaced with a hidden configuration. The new hidden
configuration reports gratuituosly crated directories
as objects, to support a compatibility behavior with
previous version 1.4.8 (even though it was incorrect).
BUG 988969: https://bugzilla.redhat.com/show_bug.cgi?id=988969
Change-Id: Idca20b8629ec38606ff5692fe62bc2cadabffc86
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5403
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Simple tool to print out an object's metadata
Change-Id: Ic2d79f28241fcf21b96c1245ef1d802e78710500
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5386
The overridden configuration files of gluster now disable the usage of
fallocate. Also modified the functional test configuration files
accordingly.
BUG: 951661
Change-Id: I3c7d518e0415b7eea5b31c88dd647069f9d730ac
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5303
Remove a number of stat() system calls that were redundant.
BUG 903396 (https://bugzilla.redhat.com/show_bug.cgi?id=903396)
Change-Id: I2e89ce8a023650cd745f136f5c697902cf2462ea
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5293
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Changed all "we" wording to "you," and altered and rearranged some
wording to help the document flow better when being read.
Change-Id: Idf39eb93666a71e3fda25375080eaae90a23ed52
Signed-off-by: Kyle Schneider <kinnder@gmail.com>
Reviewed-on: http://review.gluster.org/5306
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
The RPM can now be installed on a system to upgrade previous
installation of gluster-swift and gluster-ufo (which replaced
gluster-swift-plugin)
It also replaces all the glusterfs-swift-* servers
Change-Id: I1c1eeaa1b0a93fdb1eaf2ed018a365c4415c06b0
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5297
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Change-Id: Ibcef5d817390a11d85cf125bb9addfbfded1b019
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5301
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Added instructions on how to amend changes to a previous commit.
Included commands and brief instructions.
Change-Id: I078c9392d71273f559357a014768aada85ad3405
Signed-off-by: Kyle Schneider <kinnder@gmail.com>
Reviewed-on: http://review.gluster.org/5295
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Changed "usage" to "user" to match title. Also, prefixed
"pip install" with proper "sudo" command, due to command requiring
super user privelages.
Change-Id: I972bcb96fd476a4022385a418c86916038ec1aab
Signed-off-by: Kyle Schneider <kinnder@gmail.com>
Reviewed-on: http://review.gluster.org/5294
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
By storing the functional tests configuration files in the
repo, we can now run the functional_tests.sh to setup,
run the functional tests, and teardown.
Most likely this will be able to be run as a user from
the same directory as the repo, but at the moment, the
configuration files are copied to /etc/swift.
The only requirements are:
1. /etc/swift does not exist. That way the tests will
not interfere with an existing deployment.
2. /mnt/gluster-object/test and /mnt/gluster-object/test2
must have been created and setup correctly on an XFS
or GlusterFS volume
3. sudo rights without password prompt
4. glusterfs-openstack-swift-* rpm must not be installed
on the system
Once the requirements are met, you can execute the tests
as follows:
$ bash tools/functional_tests.sh
Change-Id: Icdbcd420355b02e64f294df7298a3e473b343655
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5281
Reviewed-by: Peter Portante <pportant@redhat.com>
Apparently, on RHEL 6, the valid archive type is tar, not tar.gz.
It causes git-archive and thus makerpm.sh to abort with:
fatal: Unknown archive format 'tar.gz'
To fix, use a regular pipeline.
Change-Id: I9174ad1d1c0e087f46ecd31eade61eeea75cb9cb
Signed-off-by: Pete Zaitcev <zaitcev@kotori.zaitcev.us>
Reviewed-on: http://review.gluster.org/5271
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This commit has the following changes:
* G4S no longer accepts URLs that end in /. A HTTP code
of 400 is returned when a / at the end of the object
is detected.
* Directories can be created as objects setting the
content-type to application/directory and content-length
to 0.
* Functional tests have been adjusted to work with
G4S constraints
Change-Id: I31038a59699a8e3eeaba902db322218c6400093e
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5246
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Community documentation is being written in Markdown
format because we can leverage GitHub.com's ability
to render Markdown into HTML directly from files
in the repo. Also, the GlusterFS Community project
has decided to use Markdown as an input into the
tool called pandoc which can convert the documents
into multiple formats.
Change-Id: Iec530f05a9a1ab3a95a1e97b791e8390068b99b4
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5256
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
It was painfully slow to gather a list of all the files in a directory tree
when there were as many directories as files, since we also stat'd all of the
directories as well. We only did that to cache using memcache the result, to
know when it changed. However, there was no way for a memcache object to be
passed down to this code. So we have removed the memcache support entirely,
and removed the stat of the directories as a result.
See BZ 911448 (https://bugzilla.redhat.com/show_bug.cgi?id=911448).
Change-Id: I2625f82eca36c31eeffa84dc76ef7f3b48aafec5
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/5252
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
This is a step towards making fd based system calls where ever possible to avoid
path lookups.
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Change-Id: I482ea29ebe0859d0a5307ff25ecb5945d54bc7ca
Reviewed-on: http://review.gluster.org/5251
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Copy the functional tests to our tree so that
we can edit and skip any tests we know we are
not going to support for this release
Change-Id: I93a76550aaaa58de49ec9a7178a34e081b7b7cf0
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5211
The code was raising an exception when the container (which
happens to be a directory) did not exist. To be compatible
with OpenStack Swift, we need to handle an object which its
container/directory does not exist.
BUG: 960944 (https://bugzilla.redhat.com/show_bug.cgi?id=960944)
Change-Id: Ibb2db354a655e040fb70ebbe6a7d8f815d33dc0f
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5201
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
We have refactored DiskDir and DiskAccount so that truly common,
shared functions live in the base class DiskCommon, and each of these
then sub-class that. This allows us to isolate unique behaviors.
We have also enabled all the skipped unit tests in test_diskdir.py
that we could, removing a few that we don't need right now.
Change-Id: I48d9c915108df8cc92c3e9a764563e1d10c50050
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5148
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
By using a generator pattern, we avoid creating whole new lists each
time, instead we iterate through the original list once (after it is
sorted), constructing the final list only once.
We also address the behavioral differences between the swift filtering
results and our code so that ported unit tests work the same
(non-slash objects, that is).
Change-Id: If32c1987f24781ff81ab4c28c9ddfff17c2e7787
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5145
Tested-by: Luis Pabon <lpabon@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-by: Luis Pabon <lpabon@redhat.com>
The various filter_* methods of the DiskCommon class don't operate on
the DiskCommon object instance itself, but always on the list of
objects passed in to it. This change does not modify those functions'
behaviors but simply refactors them to live at the module level.
Change-Id: Ic0f578d94aab65a524470278f10d1fdc8b1d6392
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5138
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This is a simple sub-classing of the proper update methods to
just no-op their behavior.
Change-Id: Ib1ae5234d372cbce572da34cfe702235b78f2310
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5088
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This is a copy of the proxy unit test from OpenStack
Swift. The file has minor changes to make it compatible
with Gluster For Swift.
Change-Id: I64a0f8d274461eb2a2c38524c6282e0d3d3d1457
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5113
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Instead we save the account in the a list, where the
index to the account is the partition number.
Change-Id: Ie4abefee48a3b237306a1e301ffa798e24e3f1db
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5120
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Running the unit tests outside of tox is useful since one can use the
python debugger, pdb, to step through failing tests to find out what
is wrong. Having it conveniently generate the coverage HTML reports
avoids running tox just to get that data.
We also add support for branch coverage, which will be highlighted in
the HTML reports.
Change-Id: Iccc7cd12efff8c136702638c8cb2fdca5d5d680b
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5134
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This is quite ugly. Sorry. We ported a set of test from OpenStack
Swift's test/unit/commont/test_db.py, the testAccountBroker and
testContainerBroker, but because of the divergent nature of the
current attempt UFO (Unified File and Object) implementation, it was
not possible to use the interface calls directly.
Instead, we are using these tests to make sure most of the code paths
are exercised, and to define much of the expected behavior. Further
refactoring, unit tests and functional test work will help us bring
the code base and these tests closer to the originals in upstream
OpenStack Swift (as of Havana).
Change-Id: I095bb03619de6e7e1378b5252913e39b1ea8bf27
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5135
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
This is a follow up change to the previous commit to fs_utils unit test case
patch.
Change-Id: I677683677071efc6176ad2145d1806bc7e658c9e
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.org/5116
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
We need the code to pass pep8 tests so that we can
enable pep8 in Jenkins
Change-Id: I31e4bd10fad65735fabceaedcb1a3bd10e120153
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5114
Reviewed-by: Peter Portante <pportant@redhat.com>
Tested-by: Peter Portante <pportant@redhat.com>
Peter Portante