Docker-Services-CLI

https://github.com/inveniosoftware/docker-services-cli/workflows/CI/badge.svg

Python module to start external services needed to tests Invenio modules and applications.

User’s Guide

This part of the documentation will show you how to get started in using docker-services-cli.

Installation

Docker-Services-CLI can be installed from PyPI.

$ pip install docker-services-cli

Configuration

Configuration module.

Configuration values (e.g. service configuration) need to be set through environment variables. However, sane defaults are provided below.

The list of services to be configured is taken from SERVICES. Each one should contain a <SERVICE_NAME>_VERSION varaible.

Service’s version are treated slightly different:

  • If the variable is not found in the environment, it will use the set default.

  • If the variable is set with a version number (e.g. 10, 10.7) it will use said value.

  • If the variable is set with a string point to one of the configured latests it will load the value of said latest and use it.

This means that the environment set/load logic will first set the default versions before loading a given service’s version.

docker_services_cli.config.DOCKER_SERVICES_FILEPATH = 'docker-services.yml'

Docker services file default path.

docker_services_cli.config.ELASTICSEARCH = {'DEFAULT_VERSIONS': {'ELASTICSEARCH_7_LATEST': '7.10.2'}, 'ELASTICSEARCH_VERSION': 'ELASTICSEARCH_7_LATEST'}

Elasticsearch service configuration.

docker_services_cli.config.MYSQL = {'CONTAINER_CONFIG_ENVIRONMENT_VARIABLES': {'MYSQL_DB': 'invenio', 'MYSQL_PASSWORD': 'invenio', 'MYSQL_ROOT_PASSWORD': 'invenio', 'MYSQL_USER': 'invenio'}, 'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'db': {'SQLALCHEMY_DATABASE_URI': 'mysql+pymysql://invenio:invenio@localhost:3306/invenio'}}, 'DEFAULT_VERSIONS': {'MYSQL_5_LATEST': '5.7', 'MYSQL_8_LATEST': '8.1'}, 'MYSQL_VERSION': 'MYSQL_8_LATEST'}

MySQL service configuration.

docker_services_cli.config.OPENSEARCH = {'DEFAULT_VERSIONS': {'OPENSEARCH_1_LATEST': '1.3.5', 'OPENSEARCH_2_LATEST': '2.3.0'}, 'OPENSEARCH_VERSION': 'OPENSEARCH_2_LATEST'}

Opensearch service configuration.

docker_services_cli.config.POSTGRESQL = {'CONTAINER_CONFIG_ENVIRONMENT_VARIABLES': {'POSTGRESQL_DB': 'invenio', 'POSTGRESQL_PASSWORD': 'invenio', 'POSTGRESQL_USER': 'invenio'}, 'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'db': {'SQLALCHEMY_DATABASE_URI': 'postgresql+psycopg2://invenio:invenio@localhost:5432/invenio'}}, 'DEFAULT_VERSIONS': {'POSTGRESQL_11_LATEST': '11.21', 'POSTGRESQL_12_LATEST': '12.16', 'POSTGRESQL_13_LATEST': '13.12', 'POSTGRESQL_14_LATEST': '14.9', 'POSTGRESQL_15_LATEST': '15.4'}, 'POSTGRESQL_VERSION': 'POSTGRESQL_14_LATEST'}

Postgresql service configuration.

docker_services_cli.config.RABBITMQ = {'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'mq': {'BROKER_URL': 'amqp://localhost:5672//'}}, 'DEFAULT_VERSIONS': {'RABBITMQ_3_LATEST': '3'}, 'RABBITMQ_VERSION': 'RABBITMQ_3_LATEST'}

RabbitMQ service configuration.

docker_services_cli.config.REDIS = {'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'cache': {'CACHE_TYPE': 'redis'}, 'mq': {'BROKER_URL': 'redis://localhost:6379/0'}}, 'DEFAULT_VERSIONS': {'REDIS_6_LATEST': '6', 'REDIS_7_LATEST': '7'}, 'REDIS_VERSION': 'REDIS_7_LATEST'}

Redis service configuration.

docker_services_cli.config.SERVICES = {'elasticsearch': {'DEFAULT_VERSIONS': {'ELASTICSEARCH_7_LATEST': '7.10.2'}, 'ELASTICSEARCH_VERSION': 'ELASTICSEARCH_7_LATEST'}, 'mysql': {'CONTAINER_CONFIG_ENVIRONMENT_VARIABLES': {'MYSQL_DB': 'invenio', 'MYSQL_PASSWORD': 'invenio', 'MYSQL_ROOT_PASSWORD': 'invenio', 'MYSQL_USER': 'invenio'}, 'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'db': {'SQLALCHEMY_DATABASE_URI': 'mysql+pymysql://invenio:invenio@localhost:3306/invenio'}}, 'DEFAULT_VERSIONS': {'MYSQL_5_LATEST': '5.7', 'MYSQL_8_LATEST': '8.1'}, 'MYSQL_VERSION': 'MYSQL_8_LATEST'}, 'opensearch': {'DEFAULT_VERSIONS': {'OPENSEARCH_1_LATEST': '1.3.5', 'OPENSEARCH_2_LATEST': '2.3.0'}, 'OPENSEARCH_VERSION': 'OPENSEARCH_2_LATEST'}, 'postgresql': {'CONTAINER_CONFIG_ENVIRONMENT_VARIABLES': {'POSTGRESQL_DB': 'invenio', 'POSTGRESQL_PASSWORD': 'invenio', 'POSTGRESQL_USER': 'invenio'}, 'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'db': {'SQLALCHEMY_DATABASE_URI': 'postgresql+psycopg2://invenio:invenio@localhost:5432/invenio'}}, 'DEFAULT_VERSIONS': {'POSTGRESQL_11_LATEST': '11.21', 'POSTGRESQL_12_LATEST': '12.16', 'POSTGRESQL_13_LATEST': '13.12', 'POSTGRESQL_14_LATEST': '14.9', 'POSTGRESQL_15_LATEST': '15.4'}, 'POSTGRESQL_VERSION': 'POSTGRESQL_14_LATEST'}, 'rabbitmq': {'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'mq': {'BROKER_URL': 'amqp://localhost:5672//'}}, 'DEFAULT_VERSIONS': {'RABBITMQ_3_LATEST': '3'}, 'RABBITMQ_VERSION': 'RABBITMQ_3_LATEST'}, 'redis': {'CONTAINER_CONNECTION_ENVIRONMENT_VARIABLES': {'cache': {'CACHE_TYPE': 'redis'}, 'mq': {'BROKER_URL': 'redis://localhost:6379/0'}}, 'DEFAULT_VERSIONS': {'REDIS_6_LATEST': '6', 'REDIS_7_LATEST': '7'}, 'REDIS_VERSION': 'REDIS_7_LATEST'}}

List of services to configure.

docker_services_cli.config.SERVICES_ALL_DEFAULT_VERSIONS = {'ELASTICSEARCH_7_LATEST': '7.10.2', 'MYSQL_5_LATEST': '5.7', 'MYSQL_8_LATEST': '8.1', 'OPENSEARCH_1_LATEST': '1.3.5', 'OPENSEARCH_2_LATEST': '2.3.0', 'POSTGRESQL_11_LATEST': '11.21', 'POSTGRESQL_12_LATEST': '12.16', 'POSTGRESQL_13_LATEST': '13.12', 'POSTGRESQL_14_LATEST': '14.9', 'POSTGRESQL_15_LATEST': '15.4', 'RABBITMQ_3_LATEST': '3', 'REDIS_6_LATEST': '6', 'REDIS_7_LATEST': '7'}

Services default latest versions.

docker_services_cli.config.SERVICE_TYPES = {'cache': ['redis'], 'db': ['mysql', 'postgresql'], 'mq': ['rabbitmq', 'redis'], 'search': ['opensearch', 'elasticsearch']}

Types of offered services.

Usage

Module to ease the creation and management of services.

The specific version for the services can be set through environment variables

$ export OPENSEARCH_VERSION=2.3.0

It can also use the centrally managed (supported) major version:

$ export OPENSEARCH_VERSION=OPENSEARCH_2_LATEST

Then it simply needs to boot up the services. Note that if no version was exported in the environment, the CLI will use the default values set in env.py.

$ docker-services-cli up --search opensearch --db postgresql --cache redis

And turn them of once they are not needed anymore:

$ docker-services-cli down

API Reference

If you are looking for information on a specific function, class or method, this part of the documentation is for you.

API Docs

CLI

CLI module.

class docker_services_cli.cli.ServicesCtx(filepath, verbose)[source]

Context class for docker services cli.

Constructor.

docker_services_cli.cli.env_output(env_set_command)[source]

Decorate command to print exportable environment settings.

docker_services_cli.cli.services_by_type(func)[source]

Decorate command adding all service types as options.

Parameters

func – The function that implements the Click command to which the service types options will be added.

Returns

A wrapped function around the passed Click command which exposes all config.SERVICES_TYPES as Click options. The list of services by type is injected as services keyword argument.

Additional Notes

Notes on how to contribute, legal information and changes are here for the interested.

Contributing

Changes

License

MIT License

Copyright (c) 2020 CERN

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Note

In applying this license, CERN does not waive the privileges and immunities granted to it by virtue of its status as an Intergovernmental Organization or submit itself to any jurisdiction.

Authors