GeniePy comes with built-in support for replicating SQLite databases using Litestream.
Description
Litestream is a tool for replicating SQLite databases to remote backends.
The core idea is that most user-facing applications can be sufficiently served using SQLite as their main data store. Adding replication to the equation makes for an even stronger case for using SQLite in production. This is exactly where Litestream comes in.
Setup
The first step is to install Litestream, if you don't have it installed already. Follow the installation instructions on their website to set this up, depending on the OS you're running.
Configuration
The DB_URL should be set to use sqlite. For instance, setting this to a
value like sqlite:///db.sqlite would make sure that GeniePy uses the
db.sqlite file in the current working directory as its main data store.
Amazon S3
Specify valid values for the following environment variables:
LITESTREAM_URL: the path inside the S3 bucket where the replication should happen (eg.s3://<bucket_name>/<path>)LITESTREAM_S3_ACCESS_KEY_ID: your API access key IDLITESTREAM_S3_SECRET_ACCESS_KEY: your API secret access key
Google Cloud Storage
Specifiy valid values for the following environment variables:
LITESTREAM_URL: the path inside the GCS bucket where the replication should happen (eg.gcs://<bucket_name>/<path>)LITESTREAM_GCS_APPLICATION_CREDENTIALS: the path to the application credentials file issued by Google that is allowed to access data on your bucket
Usage
Run poetry run task litestream to start the litestream replication process.
~/W/p/geniepy/code main ❯ poetry run task litestream
litestream v0.3.9
initialized db: /Users/siddhant/Work/projects/geniepy/code/db.sqlite
replicating to: name="s3" type="s3" bucket="siddhantgoel" path="geniepy" region="" endpoint="" sync-interval=1s
Running this command should start replicating the local SQLite database file
(what DB_URL points to) to the remote backend specified in LITESTREAM_URL.