107 KiB
Changelog
425
Added a new configuration option timezone
, a string
timezone name that will be applied to the create date timestamp rendered
inside the revision file as made availble to the
file_template
used to generate the revision filename. Note
this change adds the python-dateutil
package as a
dependency.
421
The autogenerate compare scheme now takes into account the name
truncation rules applied by SQLAlchemy's DDL compiler to the names of
the .Index
object,
when these names are dynamically truncated due to a too-long identifier
name. As the identifier truncation is deterministic, applying the same
rule to the metadata name allows correct comparison to the
database-derived name.
419
A warning is emitted when an object that's not a ~sqlalchemy.engine.Connection
is passed to .EnvironmentContext.configure
. For the case of a
~sqlalchemy.engine.Engine
passed, the check for "in
transaction" introduced in version 0.9.0 has been relaxed to work in the
case of an attribute error, as some users appear to be passing an ~sqlalchemy.engine.Engine
and not a ~sqlalchemy.engine.Connection
.
417
An adjustment to the bug fix for 369
to accommodate for env.py scripts that use an
enclosing transaction distinct from the one that the context provides,
so that the check for "didn't commit the transaction" doesn't trigger in
this scenario.
38
The .EnvironmentContext.configure.target_metadata
parameter may now be optionally specified as a sequence of .MetaData
objects instead
of a single .MetaData
object. The autogenerate process will process the sequence of .MetaData
objects in
order.
369
A .CommandError
is
now raised when a migration file opens a database transaction and does
not close/commit/rollback, when the backend database or environment
options also specify transactional_ddl is False. When transactional_ddl
is not in use, Alembic doesn't close any transaction so a transaction
opened by a migration file will cause the following migrations to fail
to apply.
413
The autoincrement=True
flag is now rendered within the
.Operations.alter_column
operation if the source
column indicates that this flag should be set to True. The behavior is
sensitive to the SQLAlchemy version in place, as the "auto" default
option is new in SQLAlchemy 1.1. When the source column indicates
autoincrement as True or "auto", the flag will render as True if the
original column contextually indicates that it should have
"autoincrement" keywords, and when the source column explcitly sets it
to False, this is also rendered. The behavior is intended to preserve
the AUTO_INCREMENT flag on MySQL as the column is fully recreated on
this backend. Note that this flag does not support
alteration of a column's "autoincrement" status, as this is not portable
across backends.
411
Fixed bug where Postgresql JSON/JSONB types rendered on SQLAlchemy
1.1 would render the "astext_type" argument which defaults to the
Text()
type without the module prefix, similarly to the
issue with ARRAY fixed in 85
.
85
Fixed bug where Postgresql ARRAY type would not render the import prefix for the inner type; additionally, user-defined renderers take place for the inner type as well as the outer type. Pull request courtesy Paul Brackin.
process_revision_directives_command
Added a keyword argument process_revision_directives
to
the .command.revision
API call. This function acts in the same role as the environment-level
.EnvironmentContext.configure.process_revision_directives
,
and allows API use of the command to drop in an ad-hoc directive process
function. This function can be used among other things to place a
complete .MigrationScript
structure in place.
412
Added support for Postgresql EXCLUDE constraints, including the
operation directive .Operations.create_exclude_constraints
as well as
autogenerate render support for the ExcludeConstraint
object as present in a Table
. Autogenerate detection for an
EXCLUDE constraint added or removed to/from an existing table is
not implemented as the SQLAlchemy Postgresql dialect
does not yet support reflection of EXCLUDE constraints.
Additionally, unknown constraint types now warn when encountered within an autogenerate action rather than raise.
fk_schema_compare
Fixed bug in .ops.create_foreign_key
where the internal table
representation would not be created properly if the foriegn key referred
to a table in a different schema of the same name. Pull request courtesy
Konstantin Lebedev.
406
The alembic_version table, when initially created, now establishes a
primary key constraint on the "version_num" column, to suit database
engines that don't support tables without primary keys. This behavior
can be controlled using the parameter .EnvironmentContext.configure.version_table_pk
.
Note that this change only applies to the initial creation of the
alembic_version table; it does not impact any existing alembic_version
table already present.
402
Fixed bug where doing batch_op.drop_constraint()
against
the primary key constraint would fail to remove the "primary_key" flag
from the column, resulting in the constraint being recreated.
update_uq_dedupe
Adjusted the logic originally added for 276
that detects MySQL
unique constraints which are actually unique indexes to be generalized
for any dialect that has this behavior, for SQLAlchemy version 1.0 and
greater. This is to allow for upcoming SQLAlchemy support for unique
constraint reflection for Oracle, which also has no dedicated concept of
"unique constraint" and instead establishes a unique index.
356
Added a file ignore for Python files of the form
.#<name>.py
, which are generated by the Emacs editor.
Pull request courtesy Markus Mattes.
393
Adjustment to the "please adjust!" comment in the script.py.mako template so that the generated comment starts with a single pound sign, appeasing flake8.
Batch mode will not use CAST() to copy data if type is given, however the basic type affinity matches that of the existing type. This to avoid SQLite's CAST of TIMESTAMP which results in truncation of the data, in those cases where the user needs to add redundant type for other reasons.
Continued pep8 improvements by adding appropriate whitespace in the base template for generated migrations. Pull request courtesy Markus Mattes.
Added an additional check when reading in revision files to detect if the same file is being read twice; this can occur if the same directory or a symlink equivalent is present more than once in version_locations. A warning is now emitted and the file is skipped. Pull request courtesy Jiri Kuncar.
Fixed bug where usage of a custom TypeDecorator which returns a
per-dialect type via .TypeDecorator.load_dialect_impl
that differs
significantly from the default "impl" for the type decorator would fail
to compare correctly during autogenerate.
Fixed bug in Postgresql "functional index skip" behavior where a functional index that ended in ASC/DESC wouldn't be detected as something we can't compare in autogenerate, leading to duplicate definitions in autogenerated files.
Fixed bug where the "base" specifier, as in "base:head", could not be
used explicitly when --sql
mode was present.
The imports in the default script.py.mako are now at the top so that flake8 editors don't complain by default. PR courtesy Guilherme Mansur.
Added support for the USING clause to the ALTER COLUMN operation for
Postgresql. Support is via the .op.alter_column.postgresql_using
parameter. Pull
request courtesy Frazer McLean.
Autogenerate with type comparison enabled will pick up on the timezone setting changing between DateTime types. Pull request courtesy David Szotten.
Fixed bug where upgrading to the head of a branch which is already present would fail, only if that head were also the dependency of a different branch that is also upgraded, as the revision system would see this as trying to go in the wrong direction. The check here has been refined to distinguish between same-branch revisions out of order vs. movement along sibling branches.
Adjusted the version traversal on downgrade such that we can
downgrade to a version that is a dependency for a version in a different
branch, without needing to remove that dependent version as
well. Previously, the target version would be seen as a "merge point"
for it's normal up-revision as well as the dependency. This integrates
with the changes for 377
and 378
to improve treatment of branches with
dependencies overall.
Fixed bug where a downgrade to a version that is also a dependency to a different branch would fail, as the system attempted to treat this as an "unmerge" of a merge point, when in fact it doesn't have the other side of the merge point available for update.
Fixed bug where the "alembic current" command wouldn't show a revision as a current head if it were also a dependency of a version in a different branch that's also applied. Extra logic is added to extract "implied" versions of different branches from the top-level versions listed in the alembic_version table.
Fixed bug where a repr() or str() of a Script object would fail if the script had multiple dependencies.
Fixed bug in autogen where if the DB connection sends the default schema as "None", this "None" would be removed from the list of schemas to check if include_schemas were set. This could possibly impact using include_schemas with SQLite.
Small adjustment made to the batch handling for reflected CHECK constraints to accommodate for SQLAlchemy 1.1 now reflecting these. Batch mode still does not support CHECK constraints from the reflected table as these can't be easily differentiated from the ones created by types such as Boolean.
Errors which occur within the Mako render step are now intercepted and raised as CommandErrors like other failure cases; the Mako exception itself is written using template-line formatting to a temporary file which is named in the exception message.
Added a fix to Postgresql server default comparison which first checks if the text of the default is identical to the original, before attempting to actually run the default. This accomodates for default-generation functions that generate a new value each time such as a uuid function.
Fixed bug introduced by the fix for 338
in version 0.8.4 where a server default could
no longer be dropped in batch mode. Pull request courtesy Martin
Domke.
Fixed bug where SQL Server arguments for drop_column() would not be propagated when running under a batch block. Pull request courtesy Michal Petrucha.
Fixed bug where the columns rendered in a
PrimaryKeyConstraint
in autogenerate would inappropriately
render the "key" of the column, not the name. Pull request courtesy
Jesse Dhillon.
Repaired batch migration support for "schema" types which generate
constraints, in particular the Boolean
datatype which
generates a CHECK constraint. Previously, an alter column operation with
this type would fail to correctly accommodate for the CHECK constraint
on change both from and to this type. In the former case the operation
would fail entirely, in the latter, the CHECK constraint would not get
generated. Both of these issues are repaired.
Changing a schema type such as Boolean
to a non-schema
type would emit a drop constraint operation which emits
NotImplementedError
for the MySQL dialect. This drop
constraint operation is now skipped when the constraint originates from
a schema type.
A major improvement to the hash id generation function, which for some reason used an awkward arithmetic formula against uuid4() that produced values that tended to start with the digits 1-4. Replaced with a simple substring approach which provides an even distribution. Pull request courtesy Antti Haapala.
Added an autogenerate renderer for the .ExecuteSQLOp
operation object; only renders if
given a plain SQL string, otherwise raises NotImplementedError. Can be
of help with custom autogenerate sequences that includes straight SQL
execution. Pull request courtesy Jacob Magnusson.
Batch mode generates a FOREIGN KEY constraint that is
self-referential using the ultimate table name, rather than
_alembic_batch_temp
. When the table is renamed from
_alembic_batch_temp
back to the original name, the FK now
points to the right name. This will not work if
referential integrity is being enforced (eg. SQLite "PRAGMA
FOREIGN_KEYS=ON") since the original table is dropped and the new table
then renamed to that name, however this is now consistent with how
foreign key constraints on other tables already operate
with batch mode; these don't support batch mode if referential integrity
is enabled in any case.
Added a type-level comparator that distinguishes .Integer
, .BigInteger
, and .SmallInteger
types and
dialect-specific types; these all have "Integer" affinity so previously
all compared as the same.
Fixed bug where the server_default
parameter of
alter_column()
would not function correctly in batch
mode.
Adjusted the rendering for index expressions such that a .Column
object present in
the source .Index
will not be rendered as table-qualified; e.g. the column name will be
rendered alone. Table-qualified names here were failing on systems such
as Postgresql.
Fixed an 0.8 regression whereby the "imports" dictionary member of
the autogen context was removed; this collection is documented in the
"render custom type" documentation as a place to add new imports. The
member is now known as .AutogenContext.imports
and the documentation is
repaired.
Fixed bug in batch mode where a table that had pre-existing indexes would create the same index on the new table with the same name, which on SQLite produces a naming conflict as index names are in a global namespace on that backend. Batch mode now defers the production of both existing and new indexes until after the entire table transfer operation is complete, which also means those indexes no longer take effect during the INSERT from SELECT section as well; the indexes are applied in a single step afterwards.
Added "pytest-xdist" as a tox dependency, so that the -n flag in the test command works if this is not already installed. Pull request courtesy Julien Danjou.
Fixed issue in PG server default comparison where model-side defaults
configured with Python unicode literals would leak the "u" character
from a repr()
into the SQL used for comparison, creating an
invalid SQL expression, as the server-side comparison feature in PG
currently repurposes the autogenerate Python rendering feature to get a
quoted version of a plain string default.
Added workaround in new foreign key option detection feature for MySQL's consideration of the "RESTRICT" option being the default, for which no value is reported from the database; the MySQL impl now corrects for when the model reports RESTRICT but the database reports nothing. A similar rule is in the default FK comparison to accommodate for the default "NO ACTION" setting being present in the model but not necessarily reported by the database, or vice versa.
A custom .EnvironmentContext.configure.process_revision_directives
hook can now generate op directives within the .UpgradeOps
and .DowngradeOps
containers
that will be generated as Python code even when the
--autogenerate
flag is False; provided that
revision_environment=True
, the full render operation will
be run even in "offline" mode.
Repaired the render operation for the .ops.AlterColumnOp
object to succeed when the
"existing_type" field was not present.
Fixed a regression 0.8 whereby the "multidb" environment template
failed to produce independent migration script segments for the output
template. This was due to the reorganization of the script rendering
system for 0.8. To accommodate this change, the .MigrationScript
structure
will in the case of multiple calls to .MigrationContext.run_migrations
produce lists for
the .MigrationScript.upgrade_ops
and .MigrationScript.downgrade_ops
attributes; each .UpgradeOps
and .DowngradeOps
instance
keeps track of its own upgrade_token
and
downgrade_token
, and each are rendered individually.
autogen_customizing_multiengine_revision
- additional
detail on the workings of the .EnvironmentContext.configure.process_revision_directives
parameter when multiple calls to .MigrationContext.run_migrations
are made.
Implemented support for autogenerate detection of changes in the
ondelete
, onupdate
, initially
and
deferrable
attributes of .ForeignKeyConstraint
objects on SQLAlchemy backends
that support these on reflection (as of SQLAlchemy 1.0.8 currently
Postgresql for all four, MySQL for ondelete
and
onupdate
only). A constraint object that modifies these
values will be reported as a "diff" and come out as a drop/create of the
constraint with the modified values. The fields are ignored for backends
which don't reflect these attributes (as of SQLA 1.0.8 this includes
SQLite, Oracle, SQL Server, others).
Fixed bug in batch mode where the
batch_op.create_foreign_key()
directive would be
incorrectly rendered with the source table and schema names in the
argument list.
Added new command alembic edit
. This command takes the
same arguments as alembic show
, however runs the target
script file within $EDITOR. Makes use of the python-editor
library in order to facilitate the handling of $EDITOR with reasonable
default behaviors across platforms. Pull request courtesy Michel
Albert.
Added new multiple-capable argument --depends-on
to the
alembic revision
command, allowing depends_on
to be established at the command line level rather than having to edit
the file after the fact. depends_on
identifiers may also be
specified as branch names at the command line or directly within the
migration file. The values may be specified as partial revision numbers
from the command line which will be resolved to full revision numbers in
the output file.
A range of positional argument names have been changed to be clearer
and more consistent across methods within the .Operations
namespace. The
most prevalent form of name change is that the descriptive names
constraint_name
and table_name
are now used
where previously the name name
would be used. This is in
support of the newly modularized and extensible system of operation
objects in alembic.operations.ops
. An argument translation layer
is in place across the alembic.op
namespace that will
ensure that named argument calling styles that use the old names will
continue to function by transparently translating to the new names, also
emitting a warning. This, along with the fact that these arguments are
positional in any case and aren't normally passed with an explicit name,
should ensure that the overwhelming majority of applications should be
unaffected by this change. The only applications that are
impacted are those that:
- use the
.Operations
object directly in some way, rather than calling upon thealembic.op
namespace, and - invoke the methods on
.Operations
using named keyword arguments for positional arguments liketable_name
,constraint_name
, etc., which commonly were namedname
as of 0.7.6. - any application that is using named keyword arguments in place of
positional argument for the recently added
.BatchOperations
object may also be affected.
The naming changes are documented as "versionchanged" for 0.8.0:
.BatchOperations.create_check_constraint
.BatchOperations.create_foreign_key
.BatchOperations.create_index
.BatchOperations.create_unique_constraint
.BatchOperations.drop_constraint
.BatchOperations.drop_index
.Operations.create_check_constraint
.Operations.create_foreign_key
.Operations.create_primary_key
.Operations.create_index
.Operations.create_table
.Operations.create_unique_constraint
.Operations.drop_constraint
.Operations.drop_index
.Operations.drop_table
The default test runner via "python setup.py test" is now py.test. nose still works via run_tests.py.
The internal system for Alembic operations has been reworked to now
build upon an extensible system of operation objects. New operations can
be added to the op.
namespace, including that they are
available in custom autogenerate schemes.
operation_plugins
The internal system for autogenerate been reworked to build upon the
extensible system of operation objects present in 302
. As part of this
change, autogenerate now produces a full object graph representing a
list of migration scripts to be written as well as operation objects
that will render all the Python code within them; a new hook .EnvironmentContext.configure.process_revision_directives
allows end-user code to fully customize what autogenerate will do,
including not just full manipulation of the Python steps to take but
also what file or files will be written and where. Additionally,
autogenerate is now extensible as far as database objects compared and
rendered into scripts; any new operation directive can also be
registered into a series of hooks that allow custom database/model
comparison functions to run as well as to render new operation
directives into autogenerate scripts.
alembic.autogenerate.toplevel
Fixed bug where in the erroneous case that alembic_version contains duplicate revisions, some commands would fail to process the version history correctly and end up with a KeyError. The fix allows the versioning logic to proceed, however a clear error is emitted later when attempting to update the alembic_version table.
Fixed critical issue where a complex series of branches/merges would bog down the iteration algorithm working over redundant nodes for millions of cycles. An internal adjustment has been made so that duplicate nodes are skipped within this iteration.
Implemented support for .BatchOperations.create_primary_key
and .BatchOperations.create_check_constraint
.
Additionally, table keyword arguments are copied from the original
reflected table, such as the "mysql_engine" keyword argument.
The .MigrationContext.stamp
method, added as part of the
versioning refactor in 0.7 as a more granular version of .command.stamp
, now includes
the "create the alembic_version table if not present" step in the same
way as the command version, which was previously omitted.
Fixed bug where foreign key options including "onupdate", "ondelete"
would not render within the op.create_foreign_key()
directive, even though they render within a full
ForeignKeyConstraint
directive.
Repaired warnings that occur when running unit tests against SQLAlchemy 1.0.5 or greater involving the "legacy_schema_aliasing" flag.
Fixed bug where the case of multiple mergepoints that all have the identical set of ancestor revisions would fail to be upgradable, producing an assertion failure. Merge points were previously assumed to always require at least an UPDATE in alembic_revision from one of the previous revs to the new one, however in this case, if one of the mergepoints has already been reached, the remaining mergepoints have no row to UPDATE therefore they must do an INSERT of their target version.
Added support for type comparison functions to be not just per
environment, but also present on the custom types themselves, by
supplying a method compare_against_backend
. Added a new
documentation section compare_types
describing type comparison fully.
Added a new option .EnvironmentContext.configure.literal_binds
,
which will pass the literal_binds
flag into the compilation
of SQL constructs when using "offline" mode. This has the effect that
SQL objects like inserts, updates, deletes as well as textual statements
sent using text()
will be compiled such that the dialect
will attempt to render literal values "inline" automatically. Only a
subset of types is typically supported; the .Operations.inline_literal
construct remains as the construct used to force a specific literal
representation of a value. The .EnvironmentContext.configure.literal_binds
flag
is added to the "offline" section of the env.py
files
generated in new environments.
Fully implemented the ~.Operations.batch_alter_table.copy_from
parameter for batch mode, which previously was not functioning. This
allows "batch mode" to be usable in conjunction with
--sql
.
Repaired support for the .BatchOperations.create_index
directive, which was
mis-named internally such that the operation within a batch context
could not proceed. The create index operation will proceed as part of a
larger "batch table recreate" operation only if ~.Operations.batch_alter_table.recreate
is set to
"always", or if the batch operation includes other instructions that
require a table recreate.
The --autogenerate
option is not valid when used in
conjunction with "offline" mode, e.g. --sql
. This now
raises a CommandError
, rather than failing more deeply
later on. Pull request courtesy Johannes Erdfelt.
Fixed bug where the mssql DROP COLUMN directive failed to include modifiers such as "schema" when emitting the DDL.
Postgresql "functional" indexes are necessarily skipped from the autogenerate process, as the SQLAlchemy backend currently does not support reflection of these structures. A warning is emitted both from the SQLAlchemy backend as well as from the Alembic backend for Postgresql when such an index is detected.
Fixed bug where MySQL backend would report dropped unique indexes and/or constraints as both at the same time. This is because MySQL doesn't actually have a "unique constraint" construct that reports differently than a "unique index", so it is present in both lists. The net effect though is that the MySQL backend will report a dropped unique index/constraint as an index in cases where the object was first created as a unique constraint, if no other information is available to make the decision. This differs from other backends like Postgresql which can report on unique constraints and unique indexes separately.
Fixed bug where using a partial revision identifier as the "starting
revision" in --sql
mode in a downgrade operation would fail
to resolve properly.
As a side effect of this change, the .EnvironmentContext.get_starting_revision_argument
method will return the "starting" revision in its originally-given
"partial" form in all cases, whereas previously when running within the
.command.stamp
command, it would have been resolved to a full number before passing it
to the .EnvironmentContext
. The resolution of this value to
a real revision number has basically been moved to a more fundamental
level within the offline migration process.
Added a new feature .Config.attributes
, to help with the use case of
sharing state such as engines and connections on the outside with a
series of Alembic API calls; also added a new cookbook section to
describe this simple but pretty important use case.
connection_sharing
The format of the default env.py
script has been refined
a bit; it now uses context managers not only for the scope of the
transaction, but also for connectivity from the starting engine. The
engine is also now called a "connectable" in support of the use case of
an external connection being passed in.
Added support for "alembic stamp" to work when given "heads" as an argument, when multiple heads are present.
Repaired issue where a server default specified without
text()
that represented a numeric or floating point (e.g.
with decimal places) value would fail in the Postgresql-specific check
for "compare server default"; as PG accepts the value with quotes in the
table specification, it's still valid. Pull request courtesy Dimitris
Theodorou.
The rendering of a ~sqlalchemy.schema.ForeignKeyConstraint
will now
ensure that the names of the source and target columns are the
database-side name of each column, and not the value of the
.key
attribute as may be set only on the Python side. This
is because Alembic generates the DDL for constraints as standalone
objects without the need to actually refer to an in-Python ~sqlalchemy.schema.Table
object, so there's no step that would resolve these Python-only key
names to database column names.
Fixed bug in foreign key autogenerate where if the in-Python table
used custom column keys (e.g. using the key='foo'
kwarg to
Column
), the comparison of existing foreign keys to those
specified in the metadata would fail, as the reflected table would not
have these keys available which to match up. Foreign key comparison for
autogenerate now ensures it's looking at the database-side names of the
columns in all cases; this matches the same functionality within unique
constraints and indexes.
Fixed issue in autogenerate type rendering where types that belong to
modules that have the name "sqlalchemy" in them would be mistaken as
being part of the sqlalchemy.
namespace. Pull req courtesy
Bartosz Burclaf.
Fixed regression in new versioning system where upgrade / history operation would fail on AttributeError if no version files were present at all.
Adjusted the SQLite backend regarding autogen of unique constraints to work fully with the current SQLAlchemy 1.0, which now will report on UNIQUE constraints that have no name.
Fixed bug in batch where if the target table contained multiple foreign keys to the same target table, the batch mechanics would fail with a "table already exists" error. Thanks for the help on this from Lucas Kahlert.
Fixed an issue where the MySQL routine to skip foreign-key-implicit indexes would also catch unnamed unique indexes, as they would be named after the column and look like the FK indexes. Pull request courtesy Johannes Erdfelt.
Repaired a regression in both the MSSQL and Oracle dialects whereby
the overridden _exec()
method failed to return a value, as
is needed now in the 0.7 series.
The render_as_batch
flag was inadvertently hardcoded to
True
, so all autogenerates were spitting out batch
mode...this has been fixed so that batch mode again is only when
selected in env.py.
Support for autogenerate of FOREIGN KEY constraints has been added.
These are delivered within the autogenerate process in the same manner
as UNIQUE constraints, including include_object
support.
Big thanks to Ann Kamyshnikova for doing the heavy lifting here.
Added ~.Operations.batch_alter_table.naming_convention
argument to .Operations.batch_alter_table
, as this is necessary
in order to drop foreign key constraints; these are often unnamed on the
target database, and in the case that they are named, SQLAlchemy is as
of the 0.9 series not including these names yet.
dropping_sqlite_foreign_keys
Fixed bug where the "source_schema" argument was not correctly passed
when calling .BatchOperations.create_foreign_key
. Pull request
courtesy Malte Marquarding.
Repaired the inspection, copying and rendering of CHECK constraints and so-called "schema" types such as Boolean, Enum within the batch copy system; the CHECK constraint will not be "doubled" when the table is copied, and additionally the inspection of the CHECK constraint for its member columns will no longer fail with an attribute error.
Added two new arguments .Operations.batch_alter_table.reflect_args
and
.Operations.batch_alter_table.reflect_kwargs
, so
that arguments may be passed directly to suit the ~.sqlalchemy.schema.Table
object that will be reflected.
batch_controlling_table_reflection
The "multiple heads / branches" feature has now landed. This is by
far the most significant change Alembic has seen since its inception;
while the workflow of most commands hasn't changed, and the format of
version files and the alembic_version
table are unchanged
as well, a new suite of features opens up in the case where multiple
version files refer to the same parent, or to the "base". Merging of
branches, operating across distinct named heads, and multiple
independent bases are now all supported. The feature incurs radical
changes to the internals of versioning and traversal, and should be
treated as "beta mode" for the next several subsequent releases within
0.7.
branches
In conjunction with support for multiple independent bases, the specific version directories are now also configurable to include multiple, user-defined directories. When multiple directories exist, the creation of a revision file with no down revision requires that the starting directory is indicated; the creation of subsequent revisions along that lineage will then automatically use that directory for new files.
multiple_version_directories
Added "move and copy" workflow, where a table to be altered is copied
to a new one with the new structure and the old one dropped, is now
implemented for SQLite as well as all database backends in general using
the new .Operations.batch_alter_table
system. This directive
provides a table-specific operations context which gathers column- and
constraint-level mutations specific to that table, and at the end of the
context creates a new table combining the structure of the old one with
the given changes, copies data from old table to new, and finally drops
the old table, renaming the new one to the existing name. This is
required for fully featured SQLite migrations, as SQLite has very little
support for the traditional ALTER directive. The batch directive is
intended to produce code that is still compatible with other databases,
in that the "move and copy" process only occurs for SQLite by default,
while still providing some level of sanity to SQLite's requirement by
allowing multiple table mutation operations to proceed within one "move
and copy" as well as providing explicit control over when this operation
actually occurs. The "move and copy" feature may be optionally applied
to other backends as well, however dealing with referential integrity
constraints from other tables must still be handled explicitly.
batch_migrations
Relative revision identifiers as used with
alembic upgrade
, alembic downgrade
and
alembic history
can be combined with specific revisions as
well, e.g. alembic upgrade ae10+3
, to produce a migration
target relative to the given exact version.
The alembic revision
command accepts the
--sql
option to suit some very obscure use case where the
revision_environment
flag is set up, so that
env.py
is run when alembic revision
is run
even though autogenerate isn't specified. As this flag is otherwise
confusing, error messages are now raised if
alembic revision
is invoked with both --sql
and --autogenerate
or with --sql
without
revision_environment
being set.
Added a rule for Postgresql to not render a "drop unique" and "drop index" given the same name; for now it is assumed that the "index" is the implicit one Postgreql generates. Future integration with new SQLAlchemy 1.0 features will improve this to be more resilient.
A change in the ordering when columns and constraints are dropped; autogenerate will now place the "drop constraint" calls before the "drop column" calls, so that columns involved in those constraints still exist when the constraint is dropped.
New commands added: alembic show
,
alembic heads
and alembic merge
. Also, a new
option --verbose
has been added to several informational
commands, such as alembic history
,
alembic current
, alembic branches
, and
alembic heads
. alembic revision
also contains
several new options used within the new branch management system. The
output of commands has been altered in many cases to support new fields
and attributes; the history
command in particular now
returns it's "verbose" output only if --verbose
is sent;
without this flag it reverts to it's older behavior of short line items
(which was never changed in the docs).
The --head_only
option to the
alembic current
command is deprecated; the
current
command now lists just the version numbers alone by
default; use --verbose
to get at additional output.
Added new argument .Config.config_args
, allows a dictionary of
replacement variables to be passed which will serve as substitution
values when an API-produced .Config
consumes the .ini
file. Pull
request courtesy Noufal Ibrahim.
The Oracle dialect sets "transactional DDL" to False by default, as Oracle does not support transactional DDL.
Fixed a variety of issues surrounding rendering of Python code that
contains unicode literals. The first is that the "quoted_name" construct
that SQLAlchemy uses to represent table and column names as well as
schema names does not repr()
correctly on Py2K when the
value contains unicode characters; therefore an explicit stringification
is added to these. Additionally, SQL expressions such as server defaults
were not being generated in a unicode-safe fashion leading to decode
errors if server defaults contained non-ascii characters.
The .Operations.add_column
directive will now
additionally emit the appropriate CREATE INDEX
statement if
the ~sqlalchemy.schema.Column
object specifies
index=True
. Pull request courtesy David Szotten.
The ~sqlalchemy.schema.Table
object is now returned when
the .Operations.create_table
method is used. This
Table
is suitable for use in subsequent SQL operations, in
particular the .Operations.bulk_insert
operation.
Indexes and unique constraints are now included in the .EnvironmentContext.configure.include_object
hook. Indexes are sent with type "index"
and unique
constraints with type "unique_constraint"
.
Bound parameters are now resolved as "literal" values within the SQL expression inside of a CheckConstraint(), when rendering the SQL as a text string; supported for SQLAlchemy 0.8.0 and forward.
Added a workaround for SQLAlchemy issue #3023 (fixed in 0.9.5) where a column that's part of an explicit PrimaryKeyConstraint would not have its "nullable" flag set to False, thus producing a false autogenerate. Also added a related correction to MySQL which will correct for MySQL's implicit server default of '0' when a NULL integer column is turned into a primary key column.
Repaired issue related to the fix for #208 and others; a composite foreign key reported by MySQL would cause a KeyError as Alembic attempted to remove MySQL's implicitly generated indexes from the autogenerate list.
If the "alembic_version" table is present in the target metadata, autogenerate will skip this also. Pull request courtesy Dj Gilcrease.
The .EnvironmentContext.configure.version_table
and
.EnvironmentContext.configure.version_table_schema
arguments are now honored during the autogenerate process, such that
these names will be used as the "skip" names on both the database
reflection and target metadata sides.
The default value of the .EnvironmentContext.configure.user_module_prefix
parameter is no longer the same as the SQLAlchemy
prefix. When omitted, user-defined types will now use the
__module__
attribute of the type class itself when
rendering in an autogenerated module.
Revision files are now written out using the 'wb'
modifier to open()
, since Mako reads the templates with
'rb'
, thus preventing CRs from being doubled up as has been
observed on windows. The encoding of the output now defaults to 'utf-8',
which can be configured using a newly added config file parameter
output_encoding
.
Added support for use of the ~sqlalchemy.sql.elements.quoted_name
construct when
using the schema
argument within operations. This allows a
name containing a dot to be fully quoted, as well as to provide
configurable quoting on a per-name basis.
Added a routine by which the Postgresql Alembic dialect inspects the
server default of INTEGER/BIGINT columns as they are reflected during
autogenerate for the pattern nextval(<name>...)
containing a potential sequence name, then queries
pg_catalog
to see if this sequence is "owned" by the column
being reflected; if so, it assumes this is a SERIAL or BIGSERIAL column
and the server default is omitted from the column reflection as well as
any kind of server_default comparison or rendering, along with an INFO
message in the logs indicating this has taken place. This allows
SERIAL/BIGSERIAL columns to keep the SEQUENCE from being unnecessarily
present within the autogenerate operation.
The system by which autogenerate renders expressions within a ~sqlalchemy.schema.Index
,
the server_default
of ~sqlalchemy.schema.Column
, and the
existing_server_default
of .Operations.alter_column
has been overhauled to
anticipate arbitrary SQLAlchemy SQL constructs, such as
func.somefunction()
, cast()
,
desc()
, and others. The system does not, as might be
preferred, render the full-blown Python expression as originally created
within the application's source code, as this would be exceedingly
complex and difficult. Instead, it renders the SQL expression against
the target backend that's subject to the autogenerate, and then renders
that SQL inside of a ~sqlalchemy.sql.expression.text
construct as a
literal SQL string. This approach still has the downside that the
rendered SQL construct may not be backend-agnostic in all cases, so
there is still a need for manual intervention in that small number of
cases, but overall the majority of cases should work correctly now. Big
thanks to Carlos Rivera for pull requests and support on this.
SQLAlchemy's testing infrastructure is now used to run tests. This system supports both nose and pytest and opens the way for Alembic testing to support any number of backends, parallel testing, and 3rd party dialect testing.
Minimum SQLAlchemy version is now 0.7.6, however at least 0.8.4 is strongly recommended. The overhaul of the test suite allows for fully passing tests on all SQLAlchemy versions from 0.7.6 on forward.
The "match" keyword is not sent to .ForeignKeyConstraint
by .Operations.create_foreign_key
when SQLAlchemy 0.7 is
in use; this keyword was added to SQLAlchemy as of 0.8.0.
Fixed bug in MSSQL dialect where "rename table" wasn't using
sp_rename()
as is required on SQL Server. Pull request
courtesy Łukasz Bołdys.
Added support for functional indexes when using the .Operations.create_index
directive. Within the list of columns, the SQLAlchemy
text()
construct can be sent, embedding a literal SQL
expression; the .Operations.create_index
will perform some hackery
behind the scenes to get the .Index
construct to cooperate. This works around
some current limitations in .Index
which should be resolved on the SQLAlchemy
side at some point.
A file named __init__.py
in the versions/
directory is now ignored by Alembic when the collection of version files
is retrieved. Pull request courtesy Michael Floering.
Fixed Py3K bug where an attempt would be made to sort None against string values when autogenerate would detect tables across multiple schemas, including the default schema. Pull request courtesy paradoxxxzero.
Autogenerate render will render the arguments within a Table
construct using *[...]
when the number of columns/elements
is greater than 255. Pull request courtesy Ryan P. Kelly.
Fixed bug where foreign key constraints would fail to render in autogenerate when a schema name was present. Pull request courtesy Andreas Zeidler.
Some deep-in-the-weeds fixes to try to get "server default" comparison working better across platforms and expressions, in particular on the Postgresql backend, mostly dealing with quoting/not quoting of various expressions at the appropriate time and on a per-backend basis. Repaired and tested support for such defaults as Postgresql interval and array defaults.
When a run of Alembic command line fails due to
CommandError
, the output now prefixes the string with
"FAILED:"
, and the error is also written to the log output
using log.error()
.
Liberalized even more the check for MySQL indexes that shouldn't be counted in autogenerate as "drops"; this time it's been reported that an implicitly created index might be named the same as a composite foreign key constraint, and not the actual columns, so we now skip those when detected as well.
Added a new accessor .MigrationContext.config
, when used in conjunction
with a .EnvironmentContext
and .Config
, this config will
be returned. Patch courtesy Marc Abramowitz.
This releases' "autogenerate index detection" bug, when a MySQL table includes an Index with the same name as a column, autogenerate reported it as an "add" even though its not; this is because we ignore reflected indexes of this nature due to MySQL creating them implicitly. Indexes that are named the same as a column are now ignored on MySQL if we see that the backend is reporting that it already exists; this indicates that we can still detect additions of these indexes but not drops, as we cannot distinguish a backend index same-named as the column as one that is user generated or mysql-generated.
Added new feature .EnvironmentContext.configure.transaction_per_migration
,
which when True causes the BEGIN/COMMIT pair to incur for each migration
individually, rather than for the whole series of migrations. This is to
assist with some database directives that need to be within individual
transactions, without the need to disable transactional DDL
entirely.
Fixed bug where the include_object()
filter would not
receive the original .Column
object when evaluating a database-only
column to be dropped; the object would not include the parent .Table
nor other aspects of
the column that are important for generating the "downgrade" case where
the column is recreated.
Fixed bug where .EnvironmentContext.get_x_argument
would fail if the
.Config
in use didn't
actually originate from a command line call.
Fixed another bug regarding naming conventions, continuing from 183
, where add_index()
drop_index() directives would not correctly render the f()
construct when the index contained a convention-driven name.
Added quoting to the table name when the special EXEC is run to drop
any existing server defaults or constraints when the .drop_column.mssql_drop_check
or .drop_column.mssql_drop_default
arguments are
used.
Added/fixed support for MySQL "SET DEFAULT" / "DROP DEFAULT" phrases, which will now be rendered if only the server default is changing or being dropped (e.g. specify None to alter_column() to indicate "DROP DEFAULT"). Also added support for rendering MODIFY rather than CHANGE when the column name isn't changing.
Added support for the initially
, match
keyword arguments as well as dialect-specific keyword arguments to .Operations.create_foreign_key
.
- tags
-
feature
- tickets
-
163
Altered the support for "sourceless" migration files (e.g. only .pyc or .pyo present) so that the flag "sourceless=true" needs to be in alembic.ini for this behavior to take effect.
The feature that keeps on giving, index/unique constraint autogenerate detection, has even more fixes, this time to accommodate database dialects that both don't yet report on unique constraints, but the backend does report unique constraints as indexes. The logic Alembic uses to distinguish between "this is an index!" vs. "this is a unique constraint that is also reported as an index!" has now been further enhanced to not produce unwanted migrations when the dialect is observed to not yet implement get_unique_constraints() (e.g. mssql). Note that such a backend will no longer report index drops for unique indexes, as these cannot be distinguished from an unreported unique index.
Extensive changes have been made to more fully support SQLAlchemy's new naming conventions feature. Note that while SQLAlchemy has added this feature as of 0.9.2, some additional fixes in 0.9.4 are needed to resolve some of the issues:
- The
.Operations
object now takes into account the naming conventions that are present on the.MetaData
object that's associated using~.EnvironmentContext.configure.target_metadata
. When.Operations
renders a constraint directive likeADD CONSTRAINT
, it now will make use of this naming convention when it produces its own temporary.MetaData
object. - Note however that the autogenerate feature in most cases generates constraints like foreign keys and unique constraints with the final names intact; the only exception are the constraints implicit with a schema-type like Boolean or Enum. In most of these cases, the naming convention feature will not take effect for these constraints and will instead use the given name as is, with one exception....
- Naming conventions which use the
"%(constraint_name)s"
token, that is, produce a new name that uses the original name as a component, will still be pulled into the naming convention converter and be converted. The problem arises when autogenerate renders a constraint with it's already-generated name present in the migration file's source code, the name will be doubled up at render time due to the combination of #1 and #2. So to work around this, autogenerate now renders these already-tokenized names using the new.Operations.f
component. This component is only generated if SQLAlchemy 0.9.4 or greater is in use.
Therefore it is highly recommended that an upgrade to Alembic 0.6.4 be accompanied by an upgrade of SQLAlchemy 0.9.4, if the new naming conventions feature is used.
autogen_naming_conventions
Suppressed IOErrors which can raise when program output pipe is
closed under a program like head
; however this only works
on Python 2. On Python 3, there is not yet a known way to suppress the
BrokenPipeError warnings without prematurely terminating the program via
signals.
Fixed bug where .Operations.bulk_insert
would not function properly
when .Operations.inline_literal
values were used, either
in --sql or non-sql mode. The values will now render directly in --sql
mode. For compatibility with "online" mode, a new flag ~.Operations.bulk_insert.multiinsert
can be set
to False which will cause each parameter set to be compiled and executed
with individual INSERT statements.
Fixed a failure of the system that allows "legacy keyword arguments" to be understood, which arose as of a change in Python 3.4 regarding decorators. A workaround is applied that allows the code to work across Python 3 versions.
The .command.revision
command now returns the .Script
object
corresponding to the newly generated revision. From this structure, one
can get the revision id, the module documentation, and everything else,
for use in scripts that call upon this command. Pull request courtesy
Robbie Coomber.
Added a workaround for when we call fcntl.ioctl()
to get
at TERMWIDTH
; if the function returns zero, as is reported
to occur in some pseudo-ttys, the message wrapping system is disabled in
the same way as if ioctl()
failed.
Added new argument .EnvironmentContext.configure.user_module_prefix
.
This prefix is applied when autogenerate renders a user-defined type,
which here is defined as any type that is from a module outside of the
sqlalchemy.
hierarchy. This prefix defaults to
None
, in which case the .EnvironmentContext.configure.sqlalchemy_module_prefix
is used, thus preserving the current behavior.
Added support for autogenerate covering the use case where .Table
objects specified in
the metadata have an explicit schema
attribute whose name
matches that of the connection's default schema (e.g. "public" for
Postgresql). Previously, it was assumed that "schema" was
None
when it matched the "default" schema, now the
comparison adjusts for this.
The .compare_metadata
public API function now takes into
account the settings for .EnvironmentContext.configure.include_object
,
.EnvironmentContext.configure.include_symbol
, and
.EnvironmentContext.configure.include_schemas
, in
the same way that the --autogenerate
command does. Pull
request courtesy Roman Podoliaka.
Calling .bulk_insert
with an empty list will not emit any
commands on the current connection. This was already the case with
--sql
mode, so is now the case with "online" mode.
Enabled schema support for index and unique constraint autodetection; previously these were non-functional and could in some cases lead to attribute errors. Pull request courtesy Dimitris Theodorou.
More fixes to index autodetection; indexes created with expressions like DESC or functional indexes will no longer cause AttributeError exceptions when attempting to compare the columns.
The .ScriptDirectory
system that loads migration files
from a versions/
directory now supports so-called
"sourceless" operation, where the .py
files are not present
and instead .pyc
or .pyo
files are directly
present where the .py
files should be. Note that while
Python 3.3 has a new system of locating
.pyc
/.pyo
files within a directory called
__pycache__
(e.g. PEP-3147), PEP-3147 maintains support for
the "source-less imports" use case, where the
.pyc
/.pyo
are in present in the "old"
location, e.g. next to the .py
file; this is the usage
that's supported even when running Python3.3.
Autogenerate for op.create_table()
will not include a
PrimaryKeyConstraint()
that has no columns.
Fixed bug in the not-internally-used .ScriptDirectory.get_base
method which would fail if
called on an empty versions directory.
An almost-rewrite of the new unique constraint/index autogenerate detection, to accommodate a variety of issues. The emphasis is on not generating false positives for those cases where no net change is present, as these errors are the ones that impact all autogenerate runs:
- Fixed an issue with unique constraint autogenerate detection where a named
UniqueConstraint
on both sides with column changes would render with the "add" operation before the "drop", requiring the user to reverse the order manually.- Corrected for MySQL's apparent addition of an implicit index for a foreign key column, so that it doesn't show up as "removed". This required that the index/constraint autogen system query the dialect-specific implementation for special exceptions.
- reworked the "dedupe" logic to accommodate MySQL's bi-directional duplication of unique indexes as unique constraints, and unique constraints as unique indexes. Postgresql's slightly different logic of duplicating unique constraints into unique indexes continues to be accommodated as well. Note that a unique index or unique constraint removal on a backend that duplicates these may show up as a distinct "remove_constraint()" / "remove_index()" pair, which may need to be corrected in the post-autogenerate if multiple backends are being supported.
- added another dialect-specific exception to the SQLite backend when dealing with unnamed unique constraints, as the backend can't currently report on constraints that were made with this technique, hence they'd come out as "added" on every run.
- the
op.create_table()
directive will be auto-generated with theUniqueConstraint
objects inline, but will not double them up with a separatecreate_unique_constraint()
call, which may have been occurring. Indexes still get rendered as distinctop.create_index()
calls even when the corresponding table was created in the same script.- the inline
UniqueConstraint
withinop.create_table()
includes all the options likedeferrable
,initially
, etc. Previously these weren't rendering.
Added new argument mssql_drop_foreign_key
to .Operations.drop_column
.
Like mssql_drop_default
and mssql_drop_check
,
will do an inline lookup for a single foreign key which applies to this
column, and drop it. For a column with more than one FK, you'd still
need to explicitly use .Operations.drop_constraint
given the name, even
though only MSSQL has this limitation in the first place.
The MSSQL backend will add the batch separator (e.g.
"GO"
) in --sql
mode after the final
COMMIT
statement, to ensure that statement is also
processed in batch mode. Courtesy Derek Harland.
Fixed bug where .op.alter_column
in the MySQL dialect would fail to
apply quotes to column names that had mixed casing or spaces.
Expanded the size of the "slug" generated by "revision" to 40
characters, which is also configurable by new field
truncate_slug_length
; and also split on the word rather
than the character; courtesy Frozenball.
Fixed the output wrapping for Alembic message output, so that we either get the terminal width for "pretty printing" with indentation, or if not we just output the text as is; in any case the text won't be wrapped too short.
Fixes to Py3k in-place compatibity regarding output encoding and related; the use of the new io.* package introduced some incompatibilities on Py2k. These should be resolved, due to the introduction of new adapter types for translating from io.* to Py2k file types, StringIO types. Thanks to Javier Santacruz for help with this.
Fixed py3k bug where the wrong form of next()
was being
called when using the list_templates command. Courtesy Chris Wilkes.
Support for autogeneration detection and rendering of indexes and unique constraints has been added. The logic goes through some effort in order to differentiate between true unique constraints and unique indexes, where there are some quirks on backends like Postgresql. The effort here in producing the feature and tests is courtesy of IJL.
Fixed bug introduced by new include_object
argument
where the inspected column would be misinterpreted when using a
user-defined type comparison function, causing a KeyError or similar
expression-related error. Fix courtesy Maarten van Schaik.
Added the "deferrable" keyword argument to .op.create_foreign_key
so
that DEFERRABLE
constraint generation is supported;
courtesy Pedro Romano.
Ensured that strings going to stdout go through an encode/decode phase, so that any non-ASCII characters get to the output stream correctly in both Py2k and Py3k. Also added source encoding detection using Mako's parse_encoding() routine in Py2k so that the __doc__ of a non-ascii revision file can be treated as unicode in Py2k.
Added new kw argument to .EnvironmentContext.configure
include_object
. This is a more flexible version of the
include_symbol
argument which allows filtering of columns
as well as tables from the autogenerate process, and in the future will
also work for types, constraints and other constructs. The fully
constructed schema object is passed, including its name and type as well
as a flag indicating if the object is from the local application
metadata or is reflected.
The output of the alembic history
command is now
expanded to show information about each change on multiple lines,
including the full top message, resembling the formatting of git
log.
Added alembic.config.Config.cmd_opts
attribute, allows
access to the argparse
options passed to the
alembic
runner.
Added new command line argument -x
, allows extra
arguments to be appended to the command line which can be consumed
within an env.py
script by looking at
context.config.cmd_opts.x
, or more simply a new method
.EnvironmentContext.get_x_argument
.
Added support for options like "name" etc. to be rendered within CHECK constraints in autogenerate. Courtesy Sok Ann Yap.
Source repository has been moved from Mercurial to Git.
Repaired autogenerate rendering of ForeignKeyConstraint to include use_alter argument, if present.
Added -r
argument to alembic history
command, allows specification of [start]:[end]
to view a
slice of history. Accepts revision numbers, symbols "base", "head", a
new symbol "current" representing the current migration, as well as
relative ranges for one side at a time (i.e. -r-5:head
,
-rcurrent:+3
). Courtesy Atsushi Odagiri for this
feature.
Source base is now in-place for Python 2.6 through 3.3, without the need for 2to3. Support for Python 2.5 and below has been dropped. Huge thanks to Hong Minhee for all the effort on this!
Note
Alembic 0.5.0 now requires at least version 0.7.3 of SQLAlchemy to run properly. Support for 0.6 has been dropped.
Added version_table_schema
argument to .EnvironmentContext.configure
, complements the
version_table
argument to set an optional remote schema for
the version table. Courtesy Christian Blume.
Fixed format of RENAME for table that includes schema with Postgresql; the schema name shouldn't be in the "TO" field.
Added output_encoding
option to .EnvironmentContext.configure
, used with
--sql
mode to apply an encoding to the output stream.
Added .Operations.create_primary_key
operation, will
genenerate an ADD CONSTRAINT for a primary key.
Fixed bug whereby double quoting would be applied to target column
name during an sp_rename
operation.
transactional_ddl flag for SQLite, MySQL dialects set to False. MySQL doesn't support it, SQLite does but current pysqlite driver does not.
upgrade and downgrade commands will list the first line of the docstring out next to the version number. Courtesy Hong Minhee.
Added --head-only option to "alembic current", will print current version plus the symbol "(head)" if this version is the head or not. Courtesy Charles-Axel Dein.
Autogenerate will render additional table keyword arguments like "mysql_engine" and others within op.create_table().
The rendering of any construct during autogenerate can be customized,
in particular to allow special rendering for user-defined column,
constraint subclasses, using new render_item
argument to
.EnvironmentContext.configure
.
Fixed bug whereby create_index() would include in the constraint columns that are added to all Table objects using events, externally to the generation of the constraint. This is the same issue that was fixed for unique constraints in version 0.3.2.
Worked around a backwards-incompatible regression in Python3.3 regarding argparse; running "alembic" with no arguments now yields an informative error in py3.3 as with all previous versions. Courtesy Andrey Antukh.
SQLAlchemy 0.6 is no longer supported by Alembic - minimum version is 0.7.3, full support is as of 0.7.9.
A host of argument name changes within migration operations for consistency. Keyword arguments will continue to work on the old name for backwards compatibility, however required positional arguments will not:
.Operations.alter_column
-name
->new_column_name
- old name will work for backwards compatibility.
.Operations.create_index
-tablename
->table_name
-argument is positional.
.Operations.drop_index
-tablename
->table_name
- old name will work for backwards compatibility.
.Operations.drop_constraint
-tablename
->table_name
-argument is positional.
.Operations.drop_constraint
-type
->type_
- old name will work for backwards compatibility
Fixed bug where autogenerate would fail if a Column to be added to a table made use of the ".key" paramter.
The "implicit" constraint generated by a type such as Boolean or Enum will not generate an ALTER statement when run on SQlite, which does not support ALTER for the purpose of adding/removing constraints separate from the column def itself. While SQLite supports adding a CHECK constraint at the column level, SQLAlchemy would need modification to support this. A warning is emitted indicating this constraint cannot be added in this scenario.
Added a workaround to setup.py to prevent "NoneType" error from occuring when "setup.py test" is run.
Added an append_constraint() step to each condition within test_autogenerate:AutogenRenderTest.test_render_fk_constraint_kwarg if the SQLAlchemy version is less than 0.8, as ForeignKeyConstraint does not auto-append prior to 0.8.
Added a README.unittests with instructions for running the test suite fully.
Added support for autogenerate render of ForeignKeyConstraint options onupdate, ondelete, initially, and deferred.
Autogenerate will include "autoincrement=False" in the rendered table
metadata if this flag was set to false on the source .Column
object.
Explicit error message describing the case when downgrade --sql is used without specifying specific start/end versions.
Removed erroneous "emit_events" attribute from operations.create_table() documentation.
Fixed the minute component in file_template which returned the month part of the create date.
Support for tables in alternate schemas has been added fully to all operations, as well as to the autogenerate feature. When using autogenerate, specifying the flag include_schemas=True to Environment.configure() will also cause autogenerate to scan all schemas located by Inspector.get_schema_names(), which is supported by some (but not all) SQLAlchemy dialects including Postgresql. Enormous thanks to Bruno Binet for a huge effort in implementing as well as writing tests. .
The command line runner has been organized into a reusable CommandLine object, so that other front-ends can re-use the argument parsing built in.
Added "stdout" option to Config, provides control over where the "print" output of commands like "history", "init", "current" etc. are sent.
Fixed the "multidb" template which was badly out of date. It now generates revision files using the configuration to determine the different upgrade<xyz>() methods needed as well, instead of needing to hardcode these. Huge thanks to BryceLohr for doing the heavy lifting here.
Fixed the regexp that was checking for .py files in the version directory to allow any .py file through. Previously it was doing some kind of defensive checking, probably from some early notions of how this directory works, that was prohibiting various filename patterns such as those which begin with numbers.
Fixed MySQL rendering for server_default which didn't work if the server_default was a generated SQL expression. Courtesy Moriyoshi Koizumi.
Added support for alteration of MySQL columns that have AUTO_INCREMENT, as well as enabling this flag. Courtesy Moriyoshi Koizumi.
Added include_symbol option to EnvironmentContext.configure(), specifies a callable which will include/exclude tables in their entirety from the autogeneration process based on name.
Added year, month, day, hour, minute, second variables to file_template.
Added 'primary' to the list of constraint types recognized for MySQL drop_constraint().
Added --sql argument to the "revision" command, for the use case where the "revision_environment" config option is being used but SQL access isn't desired.
Repaired create_foreign_key() for self-referential foreign keys, which weren't working at all.
'alembic' command reports an informative error message when the configuration is missing the 'script_directory' key.
Fixes made to the constraints created/dropped alongside so-called "schema" types such as Boolean and Enum. The create/drop constraint logic does not kick in when using a dialect that doesn't use constraints for these types, such as postgresql, even when existing_type is specified to alter_column(). Additionally, the constraints are not affected if existing_type is passed but type_ is not, i.e. there's no net change in type.
Improved error message when specifiying non-ordered revision identifiers to cover the case when the "higher" rev is None, improved message overall.
Fixed issue whereby reflected server defaults wouldn't be quoted correctly; uses repr() now.
Fixed issue whereby when autogenerate would render create_table() on the upgrade side for a table that has a Boolean type, an unnecessary CheckConstraint() would be generated.
Implemented SQL rendering for CheckConstraint() within autogenerate upgrade, including for literal SQL as well as SQL Expression Language expressions.
Fixed command-line bug introduced by the "revision_environment" feature.
New config argument "revision_environment=true", causes env.py to be run unconditionally when the "revision" command is run, to support script.py.mako templates with dependencies on custom "template_args".
Added "template_args" option to configure() so that an env.py can add additional arguments to the template context when running the "revision" command. This requires either --autogenerate or the configuration directive "revision_environment=true".
Added "type" argument to op.drop_constraint(), and implemented full constraint drop support for MySQL. CHECK and undefined raise an error. MySQL needs the constraint type in order to emit a DROP CONSTRAINT.
Added version_table argument to EnvironmentContext.configure(), allowing for the configuration of the version table name.
Added support for "relative" migration identifiers, i.e. "alembic upgrade +2", "alembic downgrade -1". Courtesy Atsushi Odagiri for this feature.
Fixed bug whereby directories inside of the template directories, such as __pycache__ on Pypy, would mistakenly be interpreted as files which are part of the template.
Basic support for Oracle added, courtesy shgoh.
Added support for UniqueConstraint in autogenerate, courtesy Atsushi Odagiri
Fixed support of schema-qualified ForeignKey target in column alter operations, courtesy Alexander Kolov.
Fixed bug whereby create_unique_constraint() would include in the constraint columns that are added to all Table objects using events, externally to the generation of the constraint.
bulk_insert() fixes:
- bulk_insert() operation was not working most likely since the 0.2 series when used with an engine.
- Repaired bulk_insert() to complete when used against a lower-case-t table and executing with only one set of parameters, working around SQLAlchemy bug #2461 in this regard.
- bulk_insert() uses "inline=True" so that phrases like RETURNING and such don't get invoked for single-row bulk inserts.
- bulk_insert() will check that you're passing a list of dictionaries in, raises TypeError if not detected.
The focus of 0.3 is to clean up and more fully document the public API of Alembic, including better accessors on the MigrationContext and ScriptDirectory objects. Methods that are not considered to be public on these objects have been underscored, and methods which should be public have been cleaned up and documented, including:
MigrationContext.get_current_revision() ScriptDirectory.iterate_revisions() ScriptDirectory.get_current_head() ScriptDirectory.get_heads() ScriptDirectory.get_base() ScriptDirectory.generate_revision()
Added a bit of autogenerate to the public API in the form of the function alembic.autogenerate.compare_metadata.
Informative error message when op.XYZ directives are invoked at module import time.
Fixed inappropriate direct call to util.err() and therefore sys.exit() when Config failed to locate the config file within library usage.
Autogenerate will emit CREATE TABLE and DROP TABLE directives according to foreign key dependency order.
implement 'tablename' parameter on drop_index() as this is needed by some backends.
Added execution_options parameter to op.execute(), will call execution_options() on the Connection before executing.
The immediate use case here is to allow access to the new no_parameters option in SQLAlchemy 0.7.6, which allows some DBAPIs (psycopg2, MySQLdb) to allow percent signs straight through without escaping, thus providing cross-compatible operation with DBAPI execution and static script generation.
setup.py won't install argparse if on Python 2.7/3.2
script_location can be interpreted by pkg_resources.resource_filename(), if it is a non-absolute URI that contains colons. This scheme is the same one used by Pyramid.
added missing support for onupdate/ondelete flags for ForeignKeyConstraint, courtesy Giacomo Bagnoli
fixed a regression regarding an autogenerate error message, as well as various glitches in the Pylons sample template. The Pylons sample template requires that you tell it where to get the Engine from now. courtesy Marcin Kuzminski
drop_index() ensures a dummy column is added when it calls "Index", as SQLAlchemy 0.7.6 will warn on index with no column names.
Fixed the generation of CHECK constraint, regression from 0.2.0
API rearrangement allows everything Alembic does to be represented by contextual objects, including EnvironmentContext, MigrationContext, and Operations. Other libraries and applications can now use things like "alembic.op" without relying upon global configuration variables. The rearrangement was done such that existing migrations should be OK, as long as they use the pattern of "from alembic import context" and "from alembic import op", as these are now contextual objects, not modules.
The naming of revision files can now be customized to be some combination of "rev id" and "slug", the latter of which is based on the revision message. By default, the pattern "<rev>_<slug>" is used for new files. New script files should include the "revision" variable for this to work, which is part of the newer script.py.mako scripts.
env.py templates call connection.close() to better support programmatic usage of commands; use NullPool in conjunction with create_engine() as well so that no connection resources remain afterwards.
fix the config.main() function to honor the arguments passed, remove no longer used "scripts/alembic" as setuptools creates this for us.
Fixed alteration of column type on MSSQL to not include the keyword "TYPE".
Can create alembic.config.Config with no filename, use set_main_option() to add values. Also added set_section_option() which will add sections.
Clean up file write operations so that file handles are closed.
PyPy is supported.
Python 2.5 is supported, needs __future__.with_statement
Fix autogenerate so that "pass" is generated between the two comments if no net migrations were present.
Fix autogenerate bug that prevented correct reflection of a foreign-key referenced table in the list of "to remove".
Fix bug where create_table() didn't handle self-referential foreign key correctly
Default prefix for autogenerate directives is "op.", matching the mako templates.
Add alembic_module_prefix argument to configure() to complement sqlalchemy_module_prefix.
fix quotes not being rendered in ForeignKeConstraint during autogenerate
Initial release. Status of features:
Alembic is used in at least one production environment, but should still be considered ALPHA LEVEL SOFTWARE as of this release, particularly in that many features are expected to be missing / unimplemented. Major API changes are not anticipated but for the moment nothing should be assumed.
The author asks that you please report all issues, missing features, workarounds etc. to the bugtracker, at https://bitbucket.org/zzzeek/alembic/issues/new .
Python 3 is supported and has been tested.
The "Pylons" and "MultiDB" environment templates have not been directly tested - these should be considered to be samples to be modified as needed. Multiple database support itself is well tested, however.
Postgresql and MS SQL Server environments have been tested for several weeks in a production environment. In particular, some involved workarounds were implemented to allow fully-automated dropping of default- or constraint-holding columns with SQL Server.
MySQL support has also been implemented to a basic degree, including MySQL's awkward style of modifying columns being accommodated.
Other database environments not included among those three have not been tested, at all. This includes Firebird, Oracle, Sybase. Adding support for these backends should be straightforward. Please report all missing/ incorrect behaviors to the bugtracker! Patches are welcome here but are optional - please just indicate the exact format expected by the target database.
SQLite, as a backend, has almost no support for schema alterations to existing databases. The author would strongly recommend that SQLite not be used in a migration context - just dump your SQLite database into an intermediary format, then dump it back into a new schema. For dev environments, the dev installer should be building the whole DB from scratch. Or just use Postgresql, which is a much better database for non-trivial schemas. Requests for full ALTER support on SQLite should be reported to SQLite's bug tracker at http://www.sqlite.org/src/wiki?name=Bug+Reports, as Alembic will not be implementing the "rename the table to a temptable then copy the data into a new table" workaround. Note that Alembic will at some point offer an extensible API so that you can implement commands like this yourself.
Well-tested directives include add/drop table, add/drop column, including support for SQLAlchemy "schema" types which generate additional CHECK constraints, i.e. Boolean, Enum. Other directives not included here have not been strongly tested in production, i.e. rename table, etc.
Both "online" and "offline" migrations, the latter being generated SQL scripts to hand off to a DBA, have been strongly production tested against Postgresql and SQL Server.
Modify column type, default status, nullable, is functional and tested across PG, MSSQL, MySQL, but not yet widely tested in production usage.
Many migrations are still outright missing, i.e. create/add sequences, etc. As a workaround, execute() can be used for those which are missing, though posting of tickets for new features/missing behaviors is strongly encouraged.
Autogenerate feature is implemented and has been tested, though only a little bit in a production setting. In particular, detection of type and server default changes are optional and are off by default; they can also be customized by a callable. Both features work but can have surprises particularly the disparity between BIT/TINYINT and boolean, which hasn't yet been worked around, as well as format changes performed by the database on defaults when it reports back. When enabled, the PG dialect will execute the two defaults to be compared to see if they are equivalent. Other backends may need to do the same thing.
The autogenerate feature only generates "candidate" commands which must be hand-tailored in any case, so is still a useful feature and is safe to use. Please report missing/broken features of autogenerate! This will be a great feature and will also improve SQLAlchemy's reflection services.
Support for non-ASCII table, column and constraint names is mostly nonexistent. This is also a straightforward feature add as SQLAlchemy itself supports unicode identifiers; Alembic itself will likely need fixes to logging, column identification by key, etc. for full support here.