Commit Graph

138 Commits

Author SHA1 Message Date
Darragh Bailey a4062de4ae Switch to using fixtures-git for testing
Use library to handle setup of fixtures for testing with git.

Change-Id: I5b39f8d52beca848adb20d646c26d91c4b977b92
2020-01-23 11:23:49 +00:00
Darragh Bailey 95e6c5bea5 Align with openstack check-requirements
Replace loremipsum with faker, drop discover as unneeded and ensure any
remaining requirements match the minimum requirements.

Change-Id: Ia5c5b033173c09e899e52b23f548ff822af998f0
2018-06-29 20:19:35 +01:00
Zuul 954604e8fe Merge "Support recent git changes to merge options" 2018-04-27 17:59:00 +00:00
Darragh Bailey f54e0d120d Support recent git changes to merge options
Git 2.9+ requires an explicit option to merge unrelated
histories together. Switch all the merge commands to try
to merge without this option and call it if the stderr
indicates that it is explicitly needed.

Change-Id: I929b310126ed0ae4d6c9726e81d86c0609c3b6e2
2018-03-10 10:17:04 +00:00
Darragh Bailey 350be21af3 Handle explicit merges
GitHub projects commonly include explicit merges even when not
necessary, make sure to skip these as candidates of a previous import.

After checking the other parents of an import to see if any were an
empty merge of an obsolete commit, test whether the parents are
themselves a subset of the ancestry commits and ignore should this be
true.

Change-Id: I8b8f0b0689301cedb8b4352c3decb569c94a0e33
2018-03-06 13:51:35 +00:00
Darragh Bailey 0c33431d2e Correct typo and add a few additional debug statements
Make it easier to see what section of the code a verbose log corresponds
to.

Change-Id: I78431b45d3f78bea689cd19f4cabd127b7e8373c
2018-03-06 13:39:27 +00:00
Darragh Bailey 6415738c1c Handle multiple non-contributing commits being merged
When encountering multiple non-contributing commits included in a merge
that is TREESAME to the existing contents, ensure a list of all commits
being brought in by that merge are included in the list to be pruned
from history.

Change-Id: I643e0923f8967f9b70d10b8796688b782681bfe0
Closes-Bug: #1648029
2017-03-31 10:33:18 +00:00
Darragh Bailey ccae7b1a63 Handle non contributing merges where no import
Catch a non-contributing merge of a commit where it's parent is within
the ancestry path to the previous upstream imported.

Add it to a list of commits to be stripped to ensure when encountered
it is removed from being attempted to be re-played.

Change-Id: Ibf5034a4eec5a92097cc38120959fcc12a44af74
Related-Bug: #1648029
2017-03-31 10:32:25 +00:00
Darragh Bailey 0100a7e1d5 Find better import merge commit candidate
Continue checking until the last possible candidate has been inspected
in case we encountered a merge commit that looks a lot like an import
merge commit does.

It is possible for a merge to occur where the commit brought in does
not contribute anything to the resulting tree. In such cases the merge
commit will appear TREESAME to the previous state of the branch, and
will be mistaken to be an import merge. Ensure if there is an
additional merge commit to be considered that it is also checked.

Change-Id: If5694506b59e1288e23aecbe6e0bf8297574c72a
Related-Bug: #1648029
2017-03-27 11:47:30 +01:00
Darragh Bailey 7cffd9dc0b Include merge commit being examined in debug log
Change-Id: I3ff5dd806c076c78bfaf8afd687f2b8bf1ec31e0
2017-03-16 10:56:15 +00:00
Darragh Bailey 7220bd7994 Exclude some unnecessary merge commits from history
Removing merge commits that do not contribute to the tree during
listing, reduces the number of merge commits to be subsequently pruned.

In future any remaining merges may indicate a problem, or be used to
reduce conflict cases by preserving them should users be utilizing
manual merges to resolve patch conflicts.

Change-Id: I8118db5acace6cb5d3b056c3d2cdaf398026c84c
2017-03-09 18:26:10 +00:00
Darragh Bailey e4190ff279 Fix test and resulting random failures
Ensure the test for checking "change merge result is identical from
both upstream with carried patches and current" can be handled
correctly is building the test tree correctly using 'cherry-pick' to
duplicate the tree state.

For cherry-pick of changes to work correct, enable use of 'force
cherry-pick' through the '-x' option, to ensure a new commit is always
made instead of allowing for the possibility of a fast-forward to occur.

Include a modification to the commit message to insert the new node name
in the message (e.g "[C] Add file change" is cherry-picked as "[C1] Add
file change")

This changes exposed a random bug where the walking of the graph results
in the same node being visited twice. This in turn causes the wrong
commit id to be stored and reused to checkout the desired node when
building the tree resulting in a very broken graph.

Add a check for the _FINISHED flag being set to spot this scenario and
use this to avoid returning the same node twice in the topological
sorted graph, which in turn ensures that we do not accidentally
recreate a second commit for the same node, replacing the original node.

Change-Id: Ie1ae27196b3616979826e2df72df3cb7a517475d
2017-02-13 15:46:10 +00:00
Jenkins 2d04fcdb7f Merge "Fix searcher debug output" 2017-01-31 18:13:28 +00:00
Darragh Bailey caa7b5270d Switch to range from xrange for python 3 compatibility
xrange no longer exists in python 3, and since it is a small number of
iterations there is a limited performance impact under python 2 by
using range instead.

Add check to ensure linearisation exception does not occur under python
3, due to exception in attempting to use a non existing function.

Change-Id: I3e6165b8bce14a44459fc9419792784f64636f5e
2016-12-08 11:06:29 +00:00
Manuel Desbonnet 2c3a1d2bcf Fix searcher debug output
Convert output from git.transform_kwargs to string before intepolation.

Change-Id: Ibf3c17ea326c983343d334fe5a4c4e3623fcb4b9
2016-12-07 10:46:58 +00:00
Jenkins 34c72e9e39 Merge "Regression test for sequence editor causing hang" 2016-11-28 11:23:44 +00:00
Jenkins dc0ee3549f Merge "Simplify custom logging classes" 2016-11-22 11:54:48 +00:00
Darragh Bailey dfa212ffd0 Regression test for sequence editor causing hang
Add simple regression test to ensure that setting sequence.editor will
not cause a hang, and add a timeout to interactive mode tests to ensure
they are terminated with an error should they fail to complete
reasonably quickly (suggesting a hang).

Early release 0.9.0 had an issue where setting sequence.editor would
cause git-upstream to hang, due to git-rebase attempting to launch the
editor specified, while the console was being redirected by python's
subprocess.

At some point a behaviour change meant the config setting
sequence.editor was no longer used by the python subprocess calling
git-rebase. Instead the tool would launch the user specified editor
directly (checking the git config settings for sequence.editor, then
core.editor, and falling back to the environment variable EDITOR if
neither were set), and would instead ensure that when finally calling
git-rebase it would only use the special editor provided by
git-upstream.

However it's still important to check that setting sequence.editor
does not cause any future problems such the code be refactored again.

Change-Id: I303dced0d1a52791d18b39e8e6d342374e3d2eeb
Related-Bug: #1394553
2016-11-18 17:49:43 +00:00
Jenkins 7f8a63be12 Merge "Expand on ambiguous arguments in test description" 2016-11-18 17:25:44 +00:00
Jenkins 508409a43f Merge "Ensure logging is checked by most tests" 2016-11-16 10:00:05 +00:00
Darragh Bailey adc18a19db Expand on ambiguous arguments in test description
Make the reason to test for ambiguous arguments clearer as well as
better explain why creating a file with a name the same as a SHA1 that
will be examined by the searcher code will check that the code is
correctly delimiting between commit and path arguments.

Change-Id: I43e07d282fc3db86a1753fdbbfa395cdcc12727e
2016-11-16 09:56:41 +00:00
Darragh Bailey bc9fc649f6 Ensure correct prog name available to manpage
Building manpage will use the parser, which will need to be told that
'git-upstream' is the correct program name, otherwise it will use
sys.argv[0] and this is set to 'setup.py' when building using:

    python setup.py build_manpage

Change-Id: I6de46b99cc4e7d536dcd5e39ba8f224cc492e4d8
2016-11-01 13:03:02 +00:00
Darragh Bailey 60774fb22d Simplify custom logging classes
No need to use meta programming to support dedent of text blocks on
message emission for simpler log messages.

Change-Id: Icacb1c2f422cb6035b6b93a21c90b8896f0ac79c
2016-10-20 10:54:26 +01:00
Darragh Bailey 8d5fbf9388 Ensure logging is checked by most tests
Make sure to test that log messages are being emitted and captured
correctly across a larger number of tests to catch issues in tests
incorrectly polluting the setup of the logging capturing across other
test cases exposed by a recent patch.

Where tests are executed in separate threads it is possible for a
mistake in one thread, thrashing the logging capture to go unnoticed
unless other tests being executed in the same thread are checking for
correct log output. This can result in random failures as the test
execution order is dependent on PYTHONHASHSEED.

This change ensures that most tests being executed perform a positive
assertion for content to appear in the logged output, at one of the
standard log levels (as opposed to the custom one) to ensure that
logging is being captured correctly by the test framework.

Change-Id: I903478d6f3cb3f25dd7e2410c78155eca6a7d780
Related-Patch: https://review.openstack.org/387426
2016-10-20 10:54:16 +01:00
Darragh Bailey 7dd634a371 Fix help subcommand
Ensure help subcommand behaves as expected when given options and
a command as an argument to lookup.

Refactor to provide the parent parser as a default argument (using
weakref to avoid a reference cycle) so that the execute method can be
made consistent across all subcommands while still providing access to
the parent parser for the help subcommand.

Co-Authored-By: Paul Bourke <paul.bourke@oracle.com>
Change-Id: I9f7e10bbc2268d111dca7c3dbe55a87e3f057d87
2016-10-19 12:34:18 +01:00
Darragh Bailey 5c0a4a7e4d Remove compatibility hacks causing issues with newer GitPython
Remove some monkey patching used to allow git-upstream to work with
GitPython releases older than 0.3.2, as the project requirements now
depends on 1.0.1 or newer.

GitPython release from 2.0.9 onwards removes the '__slots__'
definition. This allowed for the git_dir attribute existence to be
checked to determine whether to patch the Commit class imported from
the GitPython project to add this attribute for older versions of the
API.

Change-Id: I3d83785b16cd0650942c7112b5cafffc21dddba7
Closes-Bug: #1634053
2016-10-17 11:52:17 +01:00
Darragh Bailey 691ae783e0 Check valid import branch argument value
Report invalid import branch to user when combined with --finish option
as the value provided to --import-branch must be a valid reference.

Change-Id: Ibd9e52bd4e387e997cb845721e49a91520989298
Closes-Bug: #1625878
2016-10-04 17:14:59 +01:00
Darragh Bailey 0e30d3898b Report import branch name used by default
To avoid users needing to assume the name of the import branch created,
ensure that by default the name is outputted as part of a message to
the user.

Change-Id: I3e633d1431d77f9c89e986755004a5904dbc0952
Related-Bug: #1625878
2016-10-04 17:09:55 +01:00
Darragh Bailey f243bd6355 Use tag for import branch naming when given
When importing using a tag for the upstream point to use, use the given
tag provided instead of running describe against it again as this might
result in the wrong tag being returned.

The documentation for git describe states that when provided a tag,
`git describe <tag>` will simply return the given tag, however
when it is not an annotated tag it is possible for there to be multiple
tags on the same commit and `git describe` may return any of them
instead of the originally given tag. Explicitly add a check to see if
the commit(ish) reference to import from is a tag and skip calling `git
describe` in such a case.

Change-Id: I7c6c3c2eb138089ccbc13d773da3d8a4a8efe673
Related-Bug: #1625878
2016-10-04 17:09:51 +01:00
Jenkins bb152bb522 Merge "Use python for pre/post scripts" 2016-10-04 10:13:14 +00:00
Darragh Bailey be781cc50b Support tags for input args
Enable users to specify tags to be used to import and reference commits.

Change-Id: If2a9a2fa52d6c34ab110ae7a7fdbd02fae82e3b9
Closes-Bug: #1547180
2016-10-03 11:36:55 +01:00
Darragh Bailey 5f4038e268 Allow SHA1 as reference for inputs
Support specifying upstream and additional branches to import via
SHA1s.

Check for pattern character '*' in provided patterns to determine which
to treat as patterns and which as explicit revisions.

Include additional test function to check converting all refs to SHA1s
as input, and an additional scenario to check passing in a relative ref
using '~' to pick a different commit to the head of the branch.

Change-Id: Ie6ad695a34307d96b95f81b017ba9f190bc258dc
Partial-Bug: #1547180
2016-10-03 11:36:55 +01:00
Darragh Bailey 7897e35491 Use python for pre/post scripts
Switch from "sh" to "python" based scripts to facilitate running the
tests on other platforms where only python may be available.

Change-Id: Ib7dea76b7cac1801521c84bce8bb61957509018f
2016-09-30 17:46:02 +01:00
Darragh Bailey 47092a9b63 Fix finish merge to ensure correct contents
Make sure to use the same variable for all git commands in the finish
phase as this has already been set to the correct branch or sha1
depending on whether the currently in a rebase or not.

In certain cases the finish steps were not using the correct references
to merge and read-tree resulting in the final merge containing the
wrong tree contents.

Add test to cover the conflict and manual resolve case, as well as
adding a check to the other tests for testing the resulting merge
contains the correct contents.

Change-Id: I2fb870ad685b7ca2543c896bb2af4410a32373d0
Closes-Bug: #1625876
2016-09-30 17:46:02 +01:00
Darragh Bailey 54d48766ae Always have rebase perform finish
To ensure consistent behaviour around encountering conflicts whether in
interactive mode or not, or whether there are changes to be carried or
not, always have rebase perform the finish step whenever the merge
behaviour is enabled.

Change-Id: I8f72ebb9fb0fa9fff4943e732cbf4f7d33672837
Related-Bug: #1625876
2016-09-30 17:46:02 +01:00
Darragh Bailey a24debcbf8 Ensure correct mode of git-rebase executed
When not using interactive mode, ensure that git rebase is called as a
subprocess to ensure git-upstream can switch the final branch to the
merged result after completion of the import.

Only where interactive mode is enabled should the branch remain on the
import branch after running to completion and leave it to the user to
switch to the target branch, as otherwise git-rebase will throw an
error.

Change-Id: I633a371775b5a177c76529a49b44fb841d9c1332
Related-Bug: #1625876
2016-09-30 17:45:59 +01:00
Jenkins db6bc6b194 Merge "Support old GitPython transform_kwargs signature" 2016-09-29 16:25:36 +00:00
Jenkins ae85853eab Merge "Refactor test to allow additional" 2016-09-29 16:25:31 +00:00
Jenkins 66846f0dc2 Merge "Ensure parent options retained for exec'ed finish" 2016-09-29 16:07:39 +00:00
Darragh Bailey d8bb479880 Ensure parent options retained for exec'ed finish
Retrieve parent parser arguments set by user, and ensure they are made
available as part of the default script cmdline. This allows the
logging from the exec'ed finish command called as the last step by
git-rebase to log to the same file automatically.

Change-Id: Ia79a54afe8ace7ade2aa5a29ad16855c9883bd54
Closes-Bug: #1625864
2016-09-29 16:54:55 +01:00
Darragh Bailey 9e077c56b9 Support old GitPython transform_kwargs signature
Change-Id: I9ce9b37bcb2227f462209bfe687a8ebe4dd74de2
2016-09-28 16:28:54 +01:00
Jenkins c5f6cc2b3a Merge "Fix access for pre-script dict member" 2016-09-28 14:33:09 +00:00
Darragh Bailey 79b31d95d1 Refactor test to allow additional
Adjust unit test checking to make way for additional tests to be added
to the same class while ensuring can verify the resulting tree while
different commit messages may be recorded.

Ensure separate parser is created for each test to prevent ensure the
'search_refs' option is isolated per test scenario as otherwise each
scenario will append entries as it is set on the class.

These lay the ground work to adding tests to check that git-upstream
can work with SHA1 and tags as inputs in addition to branches.

Change-Id: I46a27281f3f0c01848a388ad05302599f4eff1c9
Related-Bug: #1547180
2016-09-27 16:35:24 +01:00
Darragh Bailey 3b5e2415c3 Fix access for pre-script dict member
When constructing trees with the helper tool, the yaml entries are keys
in a dict rather object attributes.

Change-Id: I16b2ee4967c3f137f2b4385ec4dbc59c962ea4ef
2016-09-19 18:12:38 +01:00
Darragh Bailey 7a135a1705 Enable default GIT_PYTHON_TRACE for tests
Have tests set the GIT_PYTHON_TRACE by default, to ensure that on error
we have a detailed log of all git commands executed.

Default to setting basic tracing, and support passing through of the
environment variable via tox.ini so that users may control the level of
logging by setting the variable.

 # disable all output from GitPython
 GIT_PYTHON_TRACE= tox ...

 # log all commands executed, default for tests
 GIT_PYTHON_TRACE=1 tox ...

 # includes stdout/stderr from GitPython in logged output
 GIT_PYTHON_TRACE=full tox ...

Change-Id: I71eb8f572bc2afd70f9a44f82499565fbbd4e872
2016-09-16 14:55:42 +01:00
Darragh Bailey 26c427450b Consolidate to a single separator character
Change-Id: I4003b6a71f869b7df30f8bdf0ef219103282665a
2016-09-15 11:44:55 +01:00
Darragh Bailey c195b69c6b Fix broken interactive mode to be usable
Switch rebase editor class to exec rebase so that it takes over the
console to allow the user to continue/skip/abort as they would
normally. Update the instruction set for rebase to re-call
git-upstream as a final step for user convenience to finish the import.

When the EDITOR or GIT_EDITOR is defined to change what editor is used
to edit the sequence of instructions to git-rebase, git will recall the
same editor in those variables if one of the instructions is 'reword',
'edit', or 'squash' in order to amend the commit message. Change the
rebaseeditor script to be able to handle being called with a
COMMIT_EDITMSG file when this occurs.

This allows the 'edit', 'reword', 'squash' & 'fixup' commands to git
rebase to function as expected by the user irrespective of whether a
conflict has been encountered first or not.

Still results in git rebase outputting additional information after
the import has completed, but avoids the main issue of the interactive
mode being unusable due to ignoring the result of the git-rebase call
unless it exits with an error, which only occurs for conflicts.

Change-Id: Ic125c85b868d816ef13fac6f503092ac3142b4de
Closes-Bug: #1402032
2016-08-31 12:09:52 +01:00
Darragh Bailey 307637d8b6 Record upstream branch in import merge commit
Ensure correct upstream branch is used for the finish command when
working with branches named differently to defaults, otherwise the
merge commit message will reference the wrong branch.

Change-Id: I5b6dee8f45e86f42caf56c7e3e4c1527f89925bc
2016-08-31 12:09:52 +01:00
Darragh Bailey e231483adc Handle all local changes landed upstream
Allow importing an upstream that contains all local changes by simply
merging to replace the existing target with the upstream branch given.

In certain workflows it is possible for the local changes to have all
landed upstream. In such a case, applying of the changes carried should
detect there is nothing to apply and create import branch to point to
the upstream branch to allow the standard finish actions to complete.

Change-Id: I61ae706f2b240a0e2590e03d2e83db97dafdf962
Closes-Bug: #1367985
2016-08-31 12:09:52 +01:00
Darragh Bailey 8ab6ccc3be Detect cherry-picked changes without Change-Id
Support detecting simple cherry-picking of changes to support use with
repositories that are not using Gerrit/git-review.

Refactor the searcher list method to emulate git-rebase behaviour in
looking at a single side of the changes between two revisions and using
'--cherry-pick' to drop commits with the same patch id.

Change-Id: I75b0086203ea614c4c58873292c8dbf9eca00f64
2016-08-31 12:09:52 +01:00