Michal Iskierko
28
vendor/github.com/status-im/migrate/v4/database/postgres/README.md
generated
vendored
Normal file
28
vendor/github.com/status-im/migrate/v4/database/postgres/README.md
generated
vendored
Normal file
@@ -0,0 +1,28 @@
|
||||
# postgres
|
||||
|
||||
`postgres://user:password@host:port/dbname?query` (`postgresql://` works, too)
|
||||
|
||||
| URL Query | WithInstance Config | Description |
|
||||
|------------|---------------------|-------------|
|
||||
| `x-migrations-table` | `MigrationsTable` | Name of the migrations table |
|
||||
| `dbname` | `DatabaseName` | The name of the database to connect to |
|
||||
| `search_path` | | This variable specifies the order in which schemas are searched when an object is referenced by a simple name with no schema specified. |
|
||||
| `user` | | The user to sign in as |
|
||||
| `password` | | The user's password |
|
||||
| `host` | | The host to connect to. Values that start with / are for unix domain sockets. (default is localhost) |
|
||||
| `port` | | The port to bind to. (default is 5432) |
|
||||
| `fallback_application_name` | | An application_name to fall back to if one isn't provided. |
|
||||
| `connect_timeout` | | Maximum wait for connection, in seconds. Zero or not specified means wait indefinitely. |
|
||||
| `sslcert` | | Cert file location. The file must contain PEM encoded data. |
|
||||
| `sslkey` | | Key file location. The file must contain PEM encoded data. |
|
||||
| `sslrootcert` | | The location of the root certificate file. The file must contain PEM encoded data. |
|
||||
| `sslmode` | | Whether or not to use SSL (disable\|require\|verify-ca\|verify-full) |
|
||||
|
||||
|
||||
## Upgrading from v1
|
||||
|
||||
1. Write down the current migration version from schema_migrations
|
||||
1. `DROP TABLE schema_migrations`
|
||||
2. Wrap your existing migrations in transactions ([BEGIN/COMMIT](https://www.postgresql.org/docs/current/static/transaction-iso.html)) if you use multiple statements within one migration.
|
||||
3. Download and install the latest migrate version.
|
||||
4. Force the current migration version with `migrate force <current_version>`.
|
||||
148
vendor/github.com/status-im/migrate/v4/database/postgres/TUTORIAL.md
generated
vendored
Normal file
148
vendor/github.com/status-im/migrate/v4/database/postgres/TUTORIAL.md
generated
vendored
Normal file
@@ -0,0 +1,148 @@
|
||||
# PostgreSQL tutorial for beginners
|
||||
|
||||
## Create/configure database
|
||||
|
||||
For the purpose of this tutorial let's create PostgreSQL database called `example`.
|
||||
Our user here is `postgres`, password `password`, and host is `localhost`.
|
||||
```
|
||||
psql -h localhost -U postgres -w -c "create database example;"
|
||||
```
|
||||
When using Migrate CLI we need to pass to database URL. Let's export it to a variable for convienience:
|
||||
```
|
||||
export POSTGRESQL_URL=postgres://postgres:password@localhost:5432/example?sslmode=disable
|
||||
```
|
||||
`sslmode=disable` means that the connection with our database will not be encrypted. Enabling it is left as an exercise.
|
||||
|
||||
You can find further description of database URLs [here](README.md#database-urls).
|
||||
|
||||
## Create migrations
|
||||
Let's create table called `users`:
|
||||
```
|
||||
migrate create -ext sql -dir db/migrations -seq create_users_table
|
||||
```
|
||||
If there were no errors, we should have two files available under `db/migrations` folder:
|
||||
- 000001_create_users_table.down.sql
|
||||
- 000001_create_users_table.up.sql
|
||||
|
||||
Note the `sql` extension that we provided.
|
||||
|
||||
In the `.up.sql` file let's create the table:
|
||||
```
|
||||
CREATE TABLE IF NOT EXISTS users(
|
||||
user_id serial PRIMARY KEY,
|
||||
username VARCHAR (50) UNIQUE NOT NULL,
|
||||
password VARCHAR (50) NOT NULL,
|
||||
email VARCHAR (300) UNIQUE NOT NULL
|
||||
);
|
||||
```
|
||||
And in the `.down.sql` let's delete it:
|
||||
```
|
||||
DROP TABLE IF EXISTS users;
|
||||
```
|
||||
By adding `IF EXISTS/IF NOT EXISTS` we are making migrations idempotent - you can read more about idempotency in [getting started](GETTING_STARTED.md#create-migrations)
|
||||
|
||||
## Run migrations
|
||||
```
|
||||
migrate -database ${POSTGRESQL_URL} -path db/migrations up
|
||||
```
|
||||
Let's check if the table was created properly by running `psql example -c "\d users"`.
|
||||
The output you are supposed to see:
|
||||
```
|
||||
Table "public.users"
|
||||
Column | Type | Modifiers
|
||||
----------+------------------------+---------------------------------------------------------
|
||||
user_id | integer | not null default nextval('users_user_id_seq'::regclass)
|
||||
username | character varying(50) | not null
|
||||
password | character varying(50) | not null
|
||||
email | character varying(300) | not null
|
||||
Indexes:
|
||||
"users_pkey" PRIMARY KEY, btree (user_id)
|
||||
"users_email_key" UNIQUE CONSTRAINT, btree (email)
|
||||
"users_username_key" UNIQUE CONSTRAINT, btree (username)
|
||||
```
|
||||
Great! Now let's check if running reverse migration also works:
|
||||
```
|
||||
migrate -database ${POSTGRESQL_URL} -path db/migrations down
|
||||
```
|
||||
Make sure to check if your database changed as expected in this case as well.
|
||||
|
||||
## Database transactions
|
||||
|
||||
To show database transactions usage, let's create another set of migrations by running:
|
||||
```
|
||||
migrate create -ext sql -dir db/migrations -seq add_mood_to_users
|
||||
```
|
||||
Again, it should create for us two migrations files:
|
||||
- 000002_add_mood_to_users.down.sql
|
||||
- 000002_add_mood_to_users.up.sql
|
||||
|
||||
In Postgres, when we want our queries to be done in a transaction, we need to wrap it with `BEGIN` and `COMMIT` commands.
|
||||
In our example, we are going to add a column to our database that can only accept enumerable values or NULL.
|
||||
Migration up:
|
||||
```
|
||||
BEGIN;
|
||||
|
||||
CREATE TYPE enum_mood AS ENUM (
|
||||
'happy',
|
||||
'sad',
|
||||
'neutral'
|
||||
);
|
||||
ALTER TABLE users ADD COLUMN mood enum_mood;
|
||||
|
||||
COMMIT;
|
||||
```
|
||||
Migration down:
|
||||
```
|
||||
BEGIN;
|
||||
|
||||
ALTER TABLE users DROP COLUMN mood;
|
||||
DROP TYPE enum_mood;
|
||||
|
||||
COMMIT;
|
||||
```
|
||||
|
||||
Now we can run our new migration and check the database:
|
||||
```
|
||||
migrate -database ${POSTGRESQL_URL} -path db/migrations up
|
||||
psql example -c "\d users"
|
||||
```
|
||||
Expected output:
|
||||
```
|
||||
Table "public.users"
|
||||
Column | Type | Modifiers
|
||||
----------+------------------------+---------------------------------------------------------
|
||||
user_id | integer | not null default nextval('users_user_id_seq'::regclass)
|
||||
username | character varying(50) | not null
|
||||
password | character varying(50) | not null
|
||||
email | character varying(300) | not null
|
||||
mood | enum_mood |
|
||||
Indexes:
|
||||
"users_pkey" PRIMARY KEY, btree (user_id)
|
||||
"users_email_key" UNIQUE CONSTRAINT, btree (email)
|
||||
"users_username_key" UNIQUE CONSTRAINT, btree (username)
|
||||
```
|
||||
|
||||
## Optional: Run migrations within your Go app
|
||||
Here is a very simple app running migrations for the above configuration:
|
||||
```
|
||||
import (
|
||||
"log"
|
||||
|
||||
"github.com/golang-migrate/migrate/v4"
|
||||
_ "github.com/golang-migrate/migrate/v4/database/postgres"
|
||||
_ "github.com/golang-migrate/migrate/v4/source/file"
|
||||
)
|
||||
|
||||
func main() {
|
||||
m, err := migrate.New(
|
||||
"file://db/migrations",
|
||||
"postgres://postgres:postgres@localhost:5432/example?sslmode=disable")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
if err := m.Up(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
```
|
||||
You can find details [here](README.md#use-in-your-go-project)
|
||||
362
vendor/github.com/status-im/migrate/v4/database/postgres/postgres.go
generated
vendored
Normal file
362
vendor/github.com/status-im/migrate/v4/database/postgres/postgres.go
generated
vendored
Normal file
@@ -0,0 +1,362 @@
|
||||
// +build go1.9
|
||||
|
||||
package postgres
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
nurl "net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/golang-migrate/migrate/v4"
|
||||
"github.com/golang-migrate/migrate/v4/database"
|
||||
multierror "github.com/hashicorp/go-multierror"
|
||||
"github.com/lib/pq"
|
||||
)
|
||||
|
||||
func init() {
|
||||
db := Postgres{}
|
||||
database.Register("postgres", &db)
|
||||
database.Register("postgresql", &db)
|
||||
}
|
||||
|
||||
var DefaultMigrationsTable = "schema_migrations"
|
||||
|
||||
var (
|
||||
ErrNilConfig = fmt.Errorf("no config")
|
||||
ErrNoDatabaseName = fmt.Errorf("no database name")
|
||||
ErrNoSchema = fmt.Errorf("no schema")
|
||||
ErrDatabaseDirty = fmt.Errorf("database is dirty")
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
MigrationsTable string
|
||||
DatabaseName string
|
||||
SchemaName string
|
||||
}
|
||||
|
||||
type Postgres struct {
|
||||
// Locking and unlocking need to use the same connection
|
||||
conn *sql.Conn
|
||||
db *sql.DB
|
||||
isLocked bool
|
||||
|
||||
// Open and WithInstance need to guarantee that config is never nil
|
||||
config *Config
|
||||
}
|
||||
|
||||
func WithInstance(instance *sql.DB, config *Config) (database.Driver, error) {
|
||||
if config == nil {
|
||||
return nil, ErrNilConfig
|
||||
}
|
||||
|
||||
if err := instance.Ping(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
query := `SELECT CURRENT_DATABASE()`
|
||||
var databaseName string
|
||||
if err := instance.QueryRow(query).Scan(&databaseName); err != nil {
|
||||
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
if len(databaseName) == 0 {
|
||||
return nil, ErrNoDatabaseName
|
||||
}
|
||||
|
||||
config.DatabaseName = databaseName
|
||||
|
||||
query = `SELECT CURRENT_SCHEMA()`
|
||||
var schemaName string
|
||||
if err := instance.QueryRow(query).Scan(&schemaName); err != nil {
|
||||
return nil, &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
if len(schemaName) == 0 {
|
||||
return nil, ErrNoSchema
|
||||
}
|
||||
|
||||
config.SchemaName = schemaName
|
||||
|
||||
if len(config.MigrationsTable) == 0 {
|
||||
config.MigrationsTable = DefaultMigrationsTable
|
||||
}
|
||||
|
||||
conn, err := instance.Conn(context.Background())
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
px := &Postgres{
|
||||
conn: conn,
|
||||
db: instance,
|
||||
config: config,
|
||||
}
|
||||
|
||||
if err := px.ensureVersionTable(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return px, nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Open(url string) (database.Driver, error) {
|
||||
purl, err := nurl.Parse(url)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
db, err := sql.Open("postgres", migrate.FilterCustomQuery(purl).String())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
migrationsTable := purl.Query().Get("x-migrations-table")
|
||||
|
||||
px, err := WithInstance(db, &Config{
|
||||
DatabaseName: purl.Path,
|
||||
MigrationsTable: migrationsTable,
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return px, nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Close() error {
|
||||
connErr := p.conn.Close()
|
||||
dbErr := p.db.Close()
|
||||
if connErr != nil || dbErr != nil {
|
||||
return fmt.Errorf("conn: %v, db: %v", connErr, dbErr)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// https://www.postgresql.org/docs/9.6/static/explicit-locking.html#ADVISORY-LOCKS
|
||||
func (p *Postgres) Lock() error {
|
||||
if p.isLocked {
|
||||
return database.ErrLocked
|
||||
}
|
||||
|
||||
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.SchemaName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// This will wait indefinitely until the lock can be acquired.
|
||||
query := `SELECT pg_advisory_lock($1)`
|
||||
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
|
||||
return &database.Error{OrigErr: err, Err: "try lock failed", Query: []byte(query)}
|
||||
}
|
||||
|
||||
p.isLocked = true
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Unlock() error {
|
||||
if !p.isLocked {
|
||||
return nil
|
||||
}
|
||||
|
||||
aid, err := database.GenerateAdvisoryLockId(p.config.DatabaseName, p.config.SchemaName)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
query := `SELECT pg_advisory_unlock($1)`
|
||||
if _, err := p.conn.ExecContext(context.Background(), query, aid); err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
p.isLocked = false
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Run(migration io.Reader) error {
|
||||
migr, err := ioutil.ReadAll(migration)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// run migration
|
||||
query := string(migr[:])
|
||||
if _, err := p.conn.ExecContext(context.Background(), query); err != nil {
|
||||
if pgErr, ok := err.(*pq.Error); ok {
|
||||
var line uint
|
||||
var col uint
|
||||
var lineColOK bool
|
||||
if pgErr.Position != "" {
|
||||
if pos, err := strconv.ParseUint(pgErr.Position, 10, 64); err == nil {
|
||||
line, col, lineColOK = computeLineFromPos(query, int(pos))
|
||||
}
|
||||
}
|
||||
message := fmt.Sprintf("migration failed: %s", pgErr.Message)
|
||||
if lineColOK {
|
||||
message = fmt.Sprintf("%s (column %d)", message, col)
|
||||
}
|
||||
if pgErr.Detail != "" {
|
||||
message = fmt.Sprintf("%s, %s", message, pgErr.Detail)
|
||||
}
|
||||
return database.Error{OrigErr: err, Err: message, Query: migr, Line: line}
|
||||
}
|
||||
return database.Error{OrigErr: err, Err: "migration failed", Query: migr}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func computeLineFromPos(s string, pos int) (line uint, col uint, ok bool) {
|
||||
// replace crlf with lf
|
||||
s = strings.Replace(s, "\r\n", "\n", -1)
|
||||
// pg docs: pos uses index 1 for the first character, and positions are measured in characters not bytes
|
||||
runes := []rune(s)
|
||||
if pos > len(runes) {
|
||||
return 0, 0, false
|
||||
}
|
||||
sel := runes[:pos]
|
||||
line = uint(runesCount(sel, newLine) + 1)
|
||||
col = uint(pos - 1 - runesLastIndex(sel, newLine))
|
||||
return line, col, true
|
||||
}
|
||||
|
||||
const newLine = '\n'
|
||||
|
||||
func runesCount(input []rune, target rune) int {
|
||||
var count int
|
||||
for _, r := range input {
|
||||
if r == target {
|
||||
count++
|
||||
}
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
func runesLastIndex(input []rune, target rune) int {
|
||||
for i := len(input) - 1; i >= 0; i-- {
|
||||
if input[i] == target {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
func (p *Postgres) SetVersion(version int, dirty bool) error {
|
||||
tx, err := p.conn.BeginTx(context.Background(), &sql.TxOptions{})
|
||||
if err != nil {
|
||||
return &database.Error{OrigErr: err, Err: "transaction start failed"}
|
||||
}
|
||||
|
||||
query := `TRUNCATE ` + pq.QuoteIdentifier(p.config.MigrationsTable)
|
||||
if _, err := tx.Exec(query); err != nil {
|
||||
if errRollback := tx.Rollback(); errRollback != nil {
|
||||
err = multierror.Append(err, errRollback)
|
||||
}
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
if version >= 0 {
|
||||
query = `INSERT INTO ` + pq.QuoteIdentifier(p.config.MigrationsTable) + ` (version, dirty) VALUES ($1, $2)`
|
||||
if _, err := tx.Exec(query, version, dirty); err != nil {
|
||||
if errRollback := tx.Rollback(); errRollback != nil {
|
||||
err = multierror.Append(err, errRollback)
|
||||
}
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
}
|
||||
|
||||
if err := tx.Commit(); err != nil {
|
||||
return &database.Error{OrigErr: err, Err: "transaction commit failed"}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (p *Postgres) Version() (version int, dirty bool, err error) {
|
||||
query := `SELECT version, dirty FROM ` + pq.QuoteIdentifier(p.config.MigrationsTable) + ` LIMIT 1`
|
||||
err = p.conn.QueryRowContext(context.Background(), query).Scan(&version, &dirty)
|
||||
switch {
|
||||
case err == sql.ErrNoRows:
|
||||
return database.NilVersion, false, nil
|
||||
|
||||
case err != nil:
|
||||
if e, ok := err.(*pq.Error); ok {
|
||||
if e.Code.Name() == "undefined_table" {
|
||||
return database.NilVersion, false, nil
|
||||
}
|
||||
}
|
||||
return 0, false, &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
|
||||
default:
|
||||
return version, dirty, nil
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Postgres) Drop() (err error) {
|
||||
// select all tables in current schema
|
||||
query := `SELECT table_name FROM information_schema.tables WHERE table_schema=(SELECT current_schema()) AND table_type='BASE TABLE'`
|
||||
tables, err := p.conn.QueryContext(context.Background(), query)
|
||||
if err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
defer func() {
|
||||
if errClose := tables.Close(); errClose != nil {
|
||||
err = multierror.Append(err, errClose)
|
||||
}
|
||||
}()
|
||||
|
||||
// delete one table after another
|
||||
tableNames := make([]string, 0)
|
||||
for tables.Next() {
|
||||
var tableName string
|
||||
if err := tables.Scan(&tableName); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(tableName) > 0 {
|
||||
tableNames = append(tableNames, tableName)
|
||||
}
|
||||
}
|
||||
|
||||
if len(tableNames) > 0 {
|
||||
// delete one by one ...
|
||||
for _, t := range tableNames {
|
||||
query = `DROP TABLE IF EXISTS ` + pq.QuoteIdentifier(t) + ` CASCADE`
|
||||
if _, err := p.conn.ExecContext(context.Background(), query); err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ensureVersionTable checks if versions table exists and, if not, creates it.
|
||||
// Note that this function locks the database, which deviates from the usual
|
||||
// convention of "caller locks" in the Postgres type.
|
||||
func (p *Postgres) ensureVersionTable() (err error) {
|
||||
if err = p.Lock(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
defer func() {
|
||||
if e := p.Unlock(); e != nil {
|
||||
if err == nil {
|
||||
err = e
|
||||
} else {
|
||||
err = multierror.Append(err, e)
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
query := `CREATE TABLE IF NOT EXISTS ` + pq.QuoteIdentifier(p.config.MigrationsTable) + ` (version bigint not null primary key, dirty boolean not null)`
|
||||
if _, err = p.conn.ExecContext(context.Background(), query); err != nil {
|
||||
return &database.Error{OrigErr: err, Query: []byte(query)}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user