openstack
/
deb-alembic
Code Issues Proposed changes

- first draft of branch docs

This commit is contained in:
Mike Bayer 2014-11-20 16:47:26 -05:00
parent 011f94e8ac
commit 8c5db73b61
3 changed files with 398 additions and 23 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
alembic/command.py
View File

@ -255,7 +255,9 @@ def heads(config, verbose=False):
script = ScriptDirectory.from_config(config)
for rev in script.get_revisions("heads"):
config.print_stdout(rev.cmd_format(verbose, short_head_status=False))
config.print_stdout(
rev.cmd_format(
verbose, include_branches=True, tree_indicators=False))
def branches(config, verbose=False):

24
alembic/script.py
View File

@ -551,34 +551,34 @@ class Script(revision.Revision):
def _head_only(
self, include_branches=False, include_doc=False,
include_parents=False):
include_parents=False, tree_indicators=True):
text = self.revision
if include_parents:
text = "%s -> %s" % (
self._format_down_revision(), text)
if include_branches and self.branch_labels:
text += " (%s)" % util.format_as_comma(self.branch_labels)
text += "%s%s%s" % (
" (head)" if self.is_head else "",
" (branchpoint)" if self.is_branch_point else "",
" (mergepoint)" if self.is_merge_point else "",
)
if tree_indicators:
text += "%s%s%s" % (
" (head)" if self.is_head else "",
" (branchpoint)" if self.is_branch_point else "",
" (mergepoint)" if self.is_merge_point else "",
)
if include_doc:
text += ", %s" % self.doc
return text
def cmd_format(
self,
verbose, short_head_status=True,
verbose,
include_branches=False, include_doc=False,
include_parents=False):
include_parents=False, tree_indicators=True):
if verbose:
return self.log_entry
elif short_head_status or include_branches or \
include_doc or include_parents:
return self._head_only(include_branches, include_doc, include_parents)
else:
return self.revision
return self._head_only(
include_branches, include_doc,
include_parents, tree_indicators)
def _format_down_revision(self):
if not self.down_revision:

393
docs/build/tutorial.rst vendored
View File

@ -1466,11 +1466,14 @@ one ``head``, the ``upgrade`` command wants us to provide more information::
The ``upgrade`` command gives us quite a few options in which we can proceed
with our upgrade, either giving it information on *which* head we'd like to upgrade
towards, or alternatively stating that we'd like *all* heads to be upgraded
towards at once.
towards at once. However, in the typical case of two source trees being
merged, we will want to pursue a third option, which is that we can **merge** these
branches.
However, in the typical case of two source trees being merged, we will
want to pursue a third option, which is that we can **merge** these
branches. An Alembic merge is a new migration file that joins two or
Merging Branches
----------------
An Alembic merge is a migration file that joins two or
more "head" files together. If the two branches we have right now can
be said to be a "tree" structure, introducing this merge file will
turn it into a "diamond" structure::
@ -1486,10 +1489,10 @@ pass to it an argument such as ``heads``, meaning we'd like to merge all
heads. Or, we can pass it individual revision numbers sequentally::
python -m alembic.config merge -m "merge ae1 and 27c" ae1027 27c6a
Generating /path_to/foo/versions/53fffde5ad5_merge_ae1_and_27c.py ... done
Generating /path/to/foo/versions/53fffde5ad5_merge_ae1_and_27c.py ... done
Looking inside the new file, we see it as a regular migration file, with
the only new twist is that the ``down_migrations`` points to both revisions::
the only new twist is that ``down_revision`` points to both revisions::
"""merge ae1 and 27c
@ -1582,7 +1585,7 @@ With a single ``head`` target, a generic ``upgrade `` can proceed::
DELETE FROM alembic_version WHERE alembic_version.version_num = 'ae1027a6acf'
UPDATE alembic_version SET version_num='53fffde5ad5' WHERE alembic_version.version_num = '27c6a30d7c24'
At the point at which both 27c6a30d7c24 and ae1027a6acf exist within our
At the point at which both ``27c6a30d7c24`` and ``ae1027a6acf`` exist within our
database, both values are present in ``alembic_version``, which now has
two rows. If we upgrade to these two versions alone, then stop and
run ``alembic current``, we will see this::
@ -1613,9 +1616,11 @@ With a single ``head`` target, a generic ``upgrade `` can proceed::
run equally well on databases that were present on version ``ae1027a6acf``
alone, versus databases that were present on version ``27c6a30d7c24`` alone;
whichever version was not yet applied, will be applied before the merge point
can be crossed. This brings forth a way of thinking about a merge file;
it's a point that cannot be crossed until all of its dependencies are
satisfied.
can be crossed. This brings forth a way of thinking about a merge file,
as well as about any Alembic revision file. As they are considered to
be "nodes" within a set that is subject to topological sorting, each
"node" is a point that cannot be crossed until all of its dependencies
are satisfied.
Prior to Alembic's support of merge points, the use case of databases
sitting on different heads was basically impossible to reconcile; having
@ -1626,9 +1631,377 @@ With a single ``head`` target, a generic ``upgrade `` can proceed::
Working with Explicit Branches
------------------------------
The ``alembic upgrade`` command hinted at other options besides merging when
dealing with multiple heads. Let's back up and assume we're back where
we have as our heads just ``ae1027a6acf`` and ``27c6a30d7c24``::
$ alembic heads
27c6a30d7c24
ae1027a6acf
Earlier, when we did ``alembic upgrade head``, it gave us an error which
suggested ``please specify a specific target revision, '<branchname>@head' to
narrow to a specific head, or 'heads' for all heads`` in order to proceed
without merging. Let's cover those cases.
Referring to all heads at once
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The ``heads`` identifier is a lot like ``head``, except it explicitly refers
to *all* heads at once. That is, it's like telling Alembic to do the operation
for both ``ae1027a6acf`` and ``27c6a30d7c24`` simultaneously. If we started
from a fresh database and ran ``upgrade heads`` we'd see::
$ alembic upgrade heads
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
INFO [alembic.migration] Running upgrade -> 1975ea83b712, add account table
INFO [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column
INFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table
Since we've upgraded to ``heads``, and we do in fact have more than one head,
that means these two distinct heads are now in our ``alembic_version`` table.
We can see this if we run ``alembic current``::
$ alembic current
ae1027a6acf (head)
27c6a30d7c24 (head)
That means there's two rows in ``alembic_version`` right now. If we downgrade
one step at a time, Alembic will **delete** from the ``alembic_version`` table
each branch that's closed out, until only one branch remains; then it will
continue updating the single value down to the previous versions::
$ alembic downgrade -1
INFO [alembic.migration] Running downgrade ae1027a6acf -> 1975ea83b712, add a column
$ alembic current
27c6a30d7c24 (head)
$ alembic downgrade -1
INFO [alembic.migration] Running downgrade 27c6a30d7c24 -> 1975ea83b712, add shopping cart table
$ alembic current
1975ea83b712 (branchpoint)
$ alembic downgrade -1
INFO [alembic.migration] Running downgrade 1975ea83b712 -> , add account table
$ alembic current
Referring to a Specific Version
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We can pass a specific version number to ``upgrade``. Alembic will ensure that
all revisions upon which this version depends are invoked, and nothing more.
So if we ``upgrade`` either to ``27c6a30d7c24`` or ``ae1027a6acf`` specifically,
it guarantees that ``1975ea83b712`` will have been applied, but not that
any "sibling" versions are applied::
$ alembic upgrade 27c6a
INFO [alembic.migration] Running upgrade -> 1975ea83b712, add account table
INFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table
With ``1975ea83b712`` and ``27c6a30d7c24`` applied, ``ae1027a6acf`` is just
a single additional step::
$ alembic upgrade ae102
INFO [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column
Working with Branch Labels
^^^^^^^^^^^^^^^^^^^^^^^^^^
To satisfy the use case where an environment has long-lived branches, especially
independent branches as will be discussed in the next section, Alembic supports
the concept of **branch labels**. These are string values that are present
within the migration file, using the new identifier ``branch_labels``.
For example, if we want to refer to the "shopping cart" branch using the name
"shoppingcart", we can add that name to our file
``27c6a30d7c24_add_shopping_cart_table.py``::
"""add shopping cart table
"""
# revision identifiers, used by Alembic.
revision = '27c6a30d7c24'
down_revision = '1975ea83b712'
branch_labels = ('shoppingcart',)
# ...
The ``branch_labels`` attribute refers to a string name, or a tuple
of names, which will now apply to this revision, all descendants of this
revision, as well as all ancestors of this revision up until the preceding
branch point, in this case ``1975ea83b712``. We can see the ``shoppingcart``
label applied to this revision::
$ alembic history
1975ea83b712 -> 27c6a30d7c24 (shoppingcart) (head), add shopping cart table
1975ea83b712 -> ae1027a6acf (head), add a column
<base> -> 1975ea83b712 (branchpoint), add account table
With the label applied the name ``shoppingcart`` now serves as an alias
for the ``27c6a30d7c24`` revision specifically. We can illustrate this
by showing it with ``alembic show``::
$ alembic show shoppingcart
Rev: 27c6a30d7c24 (head)
Parent: 1975ea83b712
Branch names: shoppingcart
Path: foo/versions/27c6a30d7c24_add_shopping_cart_table.py
add shopping cart table
Revision ID: 27c6a30d7c24
Revises: 1975ea83b712
Create Date: 2014-11-20 13:03:11.436407
However, when using branch labels, we usually want to use them using a syntax
known as "branch at" syntax; this syntax allows us to state that we want to
use a specific revision, let's say a "head" revision, in terms of a *specific*
branch. While normally, we can't refer to ``alembic upgrade head`` when
there's multiple heads, we *can* refer to this head specifcally using
``shoppingcart@head`` syntax::
$ alembic upgrade shoppingcart@head
INFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table
The ``shoppingcart@head`` syntax becomes important to us if we wish to
add new migration files to our versions directory while maintaining multiple
branches. Just like the ``upgrade`` command, if we attempted to add a new
revision file to our multiple-heads layout without a specific parent revision,
we'd get a familiar error::
$ alembic revision -m "add a shopping cart column"
FAILED: Multiple heads are present; please specify the head revision on
which the new revision should be based, or perform a merge.
The ``alembic revision`` command is pretty clear in what we need to do;
to add our new revision specifically to the ``shoppingcart`` branch,
we use the ``--head`` argument, either with the specific revision identifier
``27c6a30d7c24``, or more generically using our branchname ``shoppingcart@head``::
$ alembic revision -m "add a shopping cart column" --head shoppingcart@head
Generating /path/to/foo/versions/d747a8a8879_add_a_shopping_cart_column.py ... done
``alembic history`` shows both files now part of the ``shoppingcart`` branch::
$ alembic history
1975ea83b712 -> ae1027a6acf (head), add a column
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
<base> -> 1975ea83b712 (branchpoint), add account table
We can limit our history operation just to this branch as well::
$ alembic history -r shoppingcart:
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
If we want to illustrate the path of ``shoppingcart`` all the way from the
base, we can do that as follows::
$ alembic history -r :shoppingcart@head
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
<base> -> 1975ea83b712 (branchpoint), add account table
We can run this operation from the "base" side as well, but we get a different
result::
$ alembic history -r shoppingcart@base:
1975ea83b712 -> ae1027a6acf (head), add a column
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
<base> -> 1975ea83b712 (branchpoint), add account table
When we list from ``shoppingcart@base`` without an endpoint, it's really shorthand
for ``-r shoppingcart@base:heads``, e.g. all heads, and since ``shoppingcart@base``
is the same "base" shared by the ``ae1027a6acf`` revision, we get that
revision in our listing as well. The ``<branchname>@base`` syntax can be
useful when we are dealing with individual bases, as we'll see in the next
section.
Working with Multiple Bases
---------------------------
We've seen in the previous section that ``alembic upgrade`` is fine
if we have multiple heads, ``alembic revision`` allows us to tell it which
"head" we'd like to associate our new revision file with, and branch labels
allow us to assign names to branches that we can use in subsequent commands.
Let's put all these together and refer to a new "base", that is, a whole
new tree of revision files that will be semi-independent of the account/shopping
cart tables we've been working with.
Creating a Labeled Base Revision
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We want to create a new, labeled branch in one step. To ensure the branch can
accommodate this label, we need to ensure our ``script.py.mako`` file, used
for generating new revision files, has the appropriate substitutions present.
If Alembic version 0.7.0 or greater was used to generate the original
migration environment, this is already done. However when working with an older
environment, ``script.py.mako`` needs to have this directive added, typically
underneath the ``down_revision`` directive::
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
# add this here in order to use revision with branch_label
branch_labels = ${repr(branch_labels)}
With this in place, we can create a new revision file, starting up a branch
that will deal with database tables involving networking; we specify the
"head" version of ``base`` as well as a ``branch_label``::
$ alembic revision -m "create networking branch" --head=base --branch-label=networking
Generating /Users/classic/dev/alembic/foo/versions/3782d9986ced_create_networking_branch.py ... done
If we ran the above command and we didn't have the newer ``script.py.mako``
directive, we'd get this error::
FAILED: Version 3cac04ae8714 specified branch_labels networking, however
the migration file foo/versions/3cac04ae8714_create_networking_branch.py
does not have them; have you upgraded your script.py.mako to include the 'branch_labels'
section?
When we receive the above error, and we would like to try again, we need to
either **delete** the incorrectly generated file in order to run ``revision``
again, *or* we can edit the ``3cac04ae8714_create_networking_branch.py``
directly to add the ``branch_labels`` in of our choosing.
Running with Multiple Bases
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Once we have a new, permanent (for as long as we desire it to be)
base in our system, we'll always have multiple heads present::
$ alembic heads
3782d9986ced (networking)
ae1027a6acf
d747a8a8879 (shoppingcart)
When we want to add a new revision file to ``networking``, we specify
``networking@head`` as the ``--head``::
$ alembic revision -m "add ip number table" --head=networking@head
Generating /Users/classic/dev/alembic/foo/versions/109ec7d132bf_add_ip_number_table.py ... done
It's important that we refer to the head using ``networking@head``; if we
only refer to ``networking``, that refers to only ``3782d9986ced`` specifically;
if we specify this and it's not a head, ``alembic revision`` will make sure
we didn't mean to specify the head::
$ alembic revision -m "add DNS table" --head=networking
FAILED: Revision 3782d9986ced is not a head revision; please
specify --splice to create a new branch from this revision
The ``<branchname>@head`` format can also be used with revision numbers
instead of branch names, though this is less convenient. If we wanted to
add a new revision to our branch that includes the un-labeled ``ae1027a6acf``,
if this weren't a head already, we could ask for the "head of the branch
that includes ``ae1027a6acf``" as follows::
$ alembic revision -m "add another account column" --head ae10@head
Generating /Users/classic/dev/alembic/foo/versions/55af2cb1c267_add_another_account_column.py ... done
We have quite a lot of versioning going on, history now shows::
$ alembic history
109ec7d132bf -> 29f859a13ea (networking) (head), add DNS table
3782d9986ced -> 109ec7d132bf (networking), add ip number table
<base> -> 3782d9986ced (networking), create networking branch
ae1027a6acf -> 55af2cb1c267 (head), add another account column
1975ea83b712 -> ae1027a6acf, add a column
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
<base> -> 1975ea83b712 (branchpoint), add account table
We may now run upgrades or downgrades freely, among individual branches
(let's assume a clean database again)::
$ alembic upgrade networking@head
INFO [alembic.migration] Running upgrade -> 3782d9986ced, create networking branch
INFO [alembic.migration] Running upgrade 3782d9986ced -> 109ec7d132bf, add ip number table
INFO [alembic.migration] Running upgrade 109ec7d132bf -> 29f859a13ea, add DNS table
or against the whole thing using ``heads``::
$ alembic upgrade heads
INFO [alembic.migration] Running upgrade -> 1975ea83b712, add account table
INFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table
INFO [alembic.migration] Running upgrade 27c6a30d7c24 -> d747a8a8879, add a shopping cart column
INFO [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column
INFO [alembic.migration] Running upgrade ae1027a6acf -> 55af2cb1c267, add another account column
If you actually wanted it, all three branches can be merged::
$ alembic merge -m "merge all three branches" heads
Generating /Users/classic/dev/alembic/foo/versions/3180f4d6e81d_merge_all_three_branches.py ... done
$ alembic upgrade head
INFO [alembic.migration] Running upgrade 29f859a13ea, 55af2cb1c267, d747a8a8879 -> 3180f4d6e81d, merge all three branches
at which point, we're back to one head, but note! This head has **two** labels
now::
$ alembic heads
3180f4d6e81d (shoppingcart, networking)
$ alembic current --verbose
Current revision(s) for postgresql://scott:XXXXX@localhost/test:
Rev: 3180f4d6e81d (head) (mergepoint)
Merges: 29f859a13ea, 55af2cb1c267, d747a8a8879
Branch names: shoppingcart, networking
Path: foo/versions/3180f4d6e81d_merge_all_three_branches.py
merge all three branches
Revision ID: 3180f4d6e81d
Revises: 29f859a13ea, 55af2cb1c267, d747a8a8879
Create Date: 2014-11-20 16:27:56.395477
When labels are combined like this, it means that ``networking@head`` and
``shoppingcart@head`` are ultimately along the same branch, as is the
unnamed ``ae1027a6acf`` branch since we've merged everything together.
``alembic history`` when leading from ``networking@base:``,
``:shoppingcart@head`` or similar will show the whole tree at this point::
$ alembic history -r :shoppingcart@head
29f859a13ea, 55af2cb1c267, d747a8a8879 -> 3180f4d6e81d (networking, shoppingcart) (head) (mergepoint), merge all three branches
109ec7d132bf -> 29f859a13ea (networking), add DNS table
3782d9986ced -> 109ec7d132bf (networking), add ip number table
<base> -> 3782d9986ced (networking), create networking branch
ae1027a6acf -> 55af2cb1c267, add another account column
1975ea83b712 -> ae1027a6acf, add a column
27c6a30d7c24 -> d747a8a8879 (shoppingcart), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
<base> -> 1975ea83b712 (branchpoint), add account table
It follows then that the "branch labels" feature is useful for branches
that are **unmerged**. Once branches are merged into a single stream, the
usefulness of labels is not as apparent.
For posterity, here's the graph of the whole thing::
--- ae10 --> 55af --->--
/ \
<base> --> 1975 --> |
\ |
--- 27c6 --> d747 --> |
(shoppingcart) \ |
+--+-----> 3180
| (networking,
/ shoppingcart)
<base> --> 3782 -----> 109e ----> 29f8 --->
(networking)
If there's any point to be made here, it's if you are too freely branching, merging
and labeling, things can get pretty crazy! Hence the branching system should
be used carefully and thoughtfully for best results.
.. _building_uptodate: