lab
lab

Model Repository

Lab uses minio to store Projects. Minio is a high performance distributed object storage server, designed for large-scale private cloud infrastructure. This makes it a great fit as a storage environment for multiple Lab Projects and Experiments. Lab makes it trivial to back up completed Projects and share them across teams.

_images/lab-uml.png

Configuring minio server

There are a number of ways to install minio on a wide range of operating systems. See more details installation instructions in minio documentation pages.

Setting up Lab minio interface

Once minio is up and running, you will need to make a note of the endpoint, access key, and secret key. Lab supports multiple minio configurations through a convenient tagging system. Each configuration can be set up through CLI:

lab config minio --tag [MINIO_TAG] -- endpoint [TEXT] --accesskey [TEXT] --secretkey [TEXT]

Note that the endpoint is simply an IP address and port of a minio host, e.g. 192.168.1.50:9000.

Storing Lab Projects

Lab Projects can be pushed to a specific minio host by running a simple command from the Project root folder:

lab push --tag [MINIO_TAG] --bucket [TEXT] .

Here, --tag specifies a nickname of an exisiting minio connection and --bucket refers to a unique destination name on minio host, analogous to an S3 bucket.

Each project contains a .labignore file that specifies intentionally untracked files to ignore during a push. A default .labignore will omit the virtual environment directory .venv. Further omitions can be specified on each line:

.venv
data
experiments/abcdefgh/model.joblib

Pruning remote repository

Sometimes it may be desirable to prune a remote repository. Pruning simply replaces the entire content of a remote repository with local files. The user is warned just before proceding, as this operation can have undersirable consequences.

lab push --tag [MINIO_TAG] --bucket [TEXT] --force .

Pulling from a remote repository

To retrieve a Lab Project from a minio host, run a simple command from folder into which you’d like to pull the Project:

lab pull --tag [MINIO_TAG] --bucket [TEXT] --project [TEXT].

In cases where connection with minio has already been establish, a project can be pushed/pulled directly from the project directory via lab push or lab pull without further options.