Attention

This is the documentation for HARP Proxy, actually published as an early preview. Both the software and documentation are a work in progress, and although we already use it on various production servers, they may contain inaccuracies, typographical errors, huge mistakes and empty pages. We work hard to eradicate all mistakes and implement stuff, but it is a long and tedious process. We appreciate your patience and understanding. Of course, any help will be greatly appreciated.

Database Migrations

Database migrations are managed by alembic, a database migration library for sqlalchemy. For now, we only have models in the sqlalchemy_storage application, and the implementation shared between this application and the command line package in the main harp package.

We use custom wrappers around the alembic CLI because we need to handle migrations slightly differently for different database engines.

Running Migrations

To run migrations, you can either start your harp server with the sqlalchemy_storage application enabled and the storage.migrate = true setting (which are the default), or use the dedicated command line tool:

# migrate to the latest revision, or «head»
harp db:migrate up head

# migrate down to a specific revision
harp db:migrate down <revision>

# migrate up to a specific revision
harp db:migrate up <revision>

# migrate down to the initial empty state
harp db:migrate down base

Note that to run migrations, you’ll need a valid storage configured.

To list available database versions, you can run:

harp db:history

Note

Migrations have no effect for sqlite setups.

Configuration

By default, the storage application will migrate to the latest database revision on startup. This is convenient for most cases, as you’ll want the latest version of the database schema to be available with the latest harp version installed, but it may not be desirable in some cases.

You can disable the automatic migrations by using the storage.migrate = false setting. This can be activated on command line by adding --set storage.migrate=false to the command line arguments, using a configuration file, or a mix of both (for example to have a good default and override it manually when you need it.

storage.migrate: false

Dialect-specific migrations and features

Some database implementations allows us to implement specific optimizations.

MySQL (and MariaDB)

MySQL and MariaDB will add full-text indexes to the searchable columns. This is done without any additional configuration.

PostgreSQL (and TimescaleDB)

PostgreSQL can use the pg_trgm extension to implement full-text search. This extension is not enabled by default, but you can install it with the associated GIN indexes using:

harp db:feature add pg_trgm ...settings...

Writing migrations

To write migrations, the easiest way is to update models then run the autogeneration command:

harp db:create-migration 'some short description'

You’ll be able to edit the newly added migration file and then, you can migrate your database to apply changes. It is recommended to squash migrations before submitting a merge request.

Note

This is only available in development environments.