3.0 configuration by andreyaksenov · Pull Request #3825 · tarantool/doc

andreyaksenov · 2023-10-31T07:54:26Z

Base topics:

Configuration - a base topic (READY)
Storing configuration in etcd (READY)
Configuration in code (READY, mostly old content with some cosmetic updates)
Migrating to declarative configuration (TODO: [Config] Migrating configuration from 2.11 to 3.х #3661)

Reference:

Configuration reference YAML (IN PROGRESS, part of [Config] Configuration reference (YAML) #3664)
Configuration reference box.cfg (READY, old content)

Tooling:

tt cluster (TODO: New tt CLI module - cluster #3725)

API:

Module config (TODO: [Config] Document the config module API #3662)

p7nov

Some comments from my side.

p7nov · 2023-11-10T06:31:10Z

+Configuration
+=============
+
+There are two approaches to configuring Tarantool:


I'd prefer having an intro: a couple of sentences about what is Tarantool configuration. What's its purpose? Can I just run Tarantool without it?

Agree, will add

p7nov · 2023-11-10T06:32:12Z

+
+*   *Since version 3.0*: In a YAML file.
+
+    In a YAML file, you can provide the full cluster topology and specify all configuration options.


Repeating In a YAML file. Let's rephase.

p7nov · 2023-11-10T06:36:11Z

+*   *Since version 3.0*: In a YAML file.
+
+    In a YAML file, you can provide the full cluster topology and specify all configuration options.
+    You can also use :ref:`etcd <configuration_etcd_overview>` to store configuration data in one reliable place.


This step from YAML to etcd is surprising. If I understand correctly, etcd doesn't store YAML data. It stores the same declarative config, but has nothing to do with YAML format.
Maybe rename this list item to Declarative configuration and say that it can be in YAML or in etcd?

etcd doesn't store YAML data

etcd stores configuration data using YAML. So, it is not a local YAML file but YAML data stored under the specified key on an etcd server. Maybe, worth changing to more general In a YAML format, smth like this:

YAML configuration allows you to provide the full cluster topology and specify all configuration options.
You can use local configuration in a YAML file for each instance or store configuration data in one reliable place using :ref:etcd <configuration_etcd_overview>.

p7nov · 2023-11-10T06:44:48Z

+*   *In version 2.11 and earlier*: :ref:`In code <configuration_code>` using the ``box.cfg`` API.
+
+    In this case, configuration is provided in a Lua initialization script.
+    Starting with the 3.0 version, configuring Tarantool in code is considered a legacy approach.


Maybe convert to an admonition for more visibility?

p7nov · 2023-11-10T06:48:47Z

+Configuration overview
+----------------------
+
+A YAML configuration file describes the full topology of a Tarantool cluster.


It's not always YAML file. To me, a configuration and its format look like separate entities.

It's not only topology. This may be important in the introductory paragraphs (readers don't know the details yet)

Changed to:

YAML configuration describes the full topology of a Tarantool cluster.

Added some information to the intro.

p7nov · 2023-11-10T07:07:59Z

+-   ``replicaset_name``
+-   ``group_name``
+
+To reference such variables in a configuration file, use double curly braces.


Suggested change

To reference such variables in a configuration file, use double curly braces.

To reference these variables in a configuration file, enclose them in double curly braces with whitespaces.

Seems more accurate

p7nov · 2023-11-10T07:12:27Z

+*   Variables whose names start with ``TT_`` are used to substitute parameters specified in a configuration file.
+    This means that these variables have a higher :ref:`priority <configuration_precedence>` than the options specified in a configuration file.
+
+*   Variables whose names start with ``TT_`` and end with ``_DEFAULT`` are used to specify default values for parameters missing in a configuration file.


Schematic variable name with placeholder can help shorten the description:

two sets .. of variables: - TT_<CONFIG_PARAMETER>. These variables are used ... - TT_<CONFIG_PARAMETER>_DEFAULT. These variables are used...

p7nov · 2023-11-10T07:24:32Z

+        $ export TT_LOG_LEVEL=3
+
+*   (*Array*) The examples below show how to set the ``TT_SHARDING_ROLES`` variable that accepts an array value.
+    Arrays can be passed in a *simple* ...


Not sure unfinished sentence with ellipsis is good here, especially with italic. I couldn't understand right away.
in a simple or JSON format: <two examples>.

p7nov · 2023-11-10T07:27:41Z

+
+Below are a few examples that show how to set environment variables of different types, like *string*, *number*, *array*, or *map*:
+
+*   (*String*) In the example below, ``TT_IPROTO_LISTEN`` is used to specify a :ref:`listening host and port <configuration_options_connection>` values:


Punctuation in this list is confusing me. I don't get these brackets and italic. Why not just

- String. In the example ... - Number. In this example... - Array. ...

p7nov · 2023-11-10T07:32:08Z

+
+.. _configuration_command_options:
+
+Command-line options


This is definitely not the place for tarantool binary options reference. Need to find place somewhere in reference or future Tools section and link to it.

p7nov

LGTM, just a couple more suggestions

p7nov · 2023-11-13T04:51:47Z

+Configuration
+=============
+
+Tarantool provides the ability to configure the full topology of a cluster and set parameters specific for concrete instances, like connection settings, memory used to store data, logging, and snapshot settings.


Suggested change

Tarantool provides the ability to configure the full topology of a cluster and set parameters specific for concrete instances, like connection settings, memory used to store data, logging, and snapshot settings.

Tarantool provides the ability to configure the full topology of a cluster and set parameters specific for concrete instances, such as connection settings, memory used to store data, logging, and snapshot settings.

"like" == comparison, similar thing
"such as" == example, items of the set you describe

p7nov · 2023-11-13T04:51:56Z

+
+There are two approaches to configuring Tarantool:
+
+*   *Since version 3.0*: In a YAML format.


Suggested change

* *Since version 3.0*: In a YAML format.

* *Since version 3.0*: In the YAML format.

p7nov · 2023-11-13T04:57:45Z

+    It stores data or might act as a router for handling CRUD requests in a :ref:`sharded <sharding>` cluster.
+-   ``replicasets``
+
+    A *replica set* is a pack of instances that operate on copies of the same databases.


Suggested change

A *replica set* is a pack of instances that operate on copies of the same databases.

A *replica set* is a pack of instances that operate on same data chunks.

"copies of the same database" may be confusing, because the term "database" can refer to everything stored in the cluster (all buckets of all spaces is one whole database).
Just an suggestion, the wording may be different.

I'd use data set here. To me, data chunk sounds like a small piece of data passed over a network.

p7nov · 2023-11-13T05:07:21Z

+Access control
+~~~~~~~~~~~~~~
+
+The ``credentials`` section allows you to grant the specified privileges to users.


It may be helpful to say here that this config creates the users. It's not obvious. I thought that perhaps I should create them somehow and then run this config to grant roles.

p7nov · 2023-11-13T05:47:04Z

+      - http://localhost:2379
+    prefix: /example
+    username: testuser
+    password: foobar


Minor: avoid when possible https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/f/foo-foobar-fubar

xuniq · 2023-11-09T14:20:21Z

+    }
+    print('Starting ', arg[1])
+
+and suppose the environment variable LISTEN_URI contains 3301,


Suggested change

and suppose the environment variable LISTEN_URI contains 3301,

and suppose the environment variable ``LISTEN_URI`` contains 3301,

xuniq · 2023-11-13T07:44:58Z

+
+.. box_cfg_legacy_note_end
+
+This topic covers specifics of configuring Tarantool in code using the ``box.cfg`` API.


Suggested change

This topic covers specifics of configuring Tarantool in code using the ``box.cfg`` API.

This topic covers the specifics of configuring Tarantool in code using the ``box.cfg`` API.

xuniq · 2023-11-13T07:45:52Z

+
+    export TT_LISTEN="localhost:3301?param1=value1&param2=value2"
+
+An empty variable (``TT_LISTEN=``) has the same effect as an unset one meaning that the corresponding configuration parameter won't be set when calling ``box.cfg{}``.


Suggested change

An empty variable (``TT_LISTEN=``) has the same effect as an unset one meaning that the corresponding configuration parameter won't be set when calling ``box.cfg{}``.

An empty variable (``TT_LISTEN=``) has the same effect as an unset one, meaning that the corresponding configuration parameter won't be set when calling ``box.cfg{}``.

xuniq · 2023-11-13T07:52:07Z

+* ``TT_LISTEN`` -- corresponds to the ``box.cfg.listen`` option.
+* ``TT_MEMTX_DIR`` -- corresponds to the ``box.cfg.memtx_dir`` option.


Suggested change

* ``TT_LISTEN`` -- corresponds to the ``box.cfg.listen`` option.

* ``TT_MEMTX_DIR`` -- corresponds to the ``box.cfg.memtx_dir`` option.

* ``TT_LISTEN`` -- corresponds to the ``box.cfg.listen`` option.

* ``TT_MEMTX_DIR`` -- corresponds to the ``box.cfg.memtx_dir`` option.

I would add the links to these options

xuniq · 2023-11-13T08:26:11Z

+
+Formally, the URI
+syntax is ``[host:]port`` or ``[username:password@]host:port``.
+If a host is omitted, then "0.0.0.0" or "[::]" is assumed


Suggested change

If a host is omitted, then "0.0.0.0" or "[::]" is assumed

If a host is omitted, then "0.0.0.0" or "[::]" is assumed,

xuniq · 2023-11-13T08:30:25Z

+where a URI is expected, for example, "unix/:/tmp/unix_domain_socket.sock" or
+simply "/tmp/unix_domain_socket.sock".


Suggested change

where a URI is expected, for example, "unix/:/tmp/unix_domain_socket.sock" or

simply "/tmp/unix_domain_socket.sock".

where a URI is expected, for example, ``"unix/:/tmp/unix_domain_socket.sock"`` or

simply ``"/tmp/unix_domain_socket.sock"``.

It's a bit difficult to read these sockets (and find them in text) without any code formatting.

xuniq · 2023-11-13T08:50:07Z

+..  literalinclude:: /code_snippets/snippets/config/instances.enabled/etcd/config.yaml
+    :language: yaml
+    :dedent:
+


If this port (2379) is default for etcd, it might be useful to mention that.

xuniq · 2023-11-13T08:54:14Z

+In this example, the following options are configured in addition to an etcd endpoint and key prefix:
+


Suggested change

In this example, the following options are configured in addition to an etcd endpoint and key prefix:

Repeats the previous paragraphs

Document a new declarative configuration approach: local YAML file and etcd config

andreyaksenov linked an issue Oct 31, 2023 that may be closed by this pull request

[Config] Storing configuration in 'etcd' #3658

Closed

andreyaksenov force-pushed the 3.0-config-etcd branch 13 times, most recently from 05caf81 to 02ea158 Compare November 1, 2023 11:59

andreyaksenov requested a review from Totktonada November 1, 2023 13:32

andreyaksenov changed the base branch from 3.0-config to 3.0 November 3, 2023 07:04

andreyaksenov changed the base branch from 3.0 to 3.0-config November 3, 2023 07:05

andreyaksenov changed the base branch from 3.0-config to 3.0 November 3, 2023 08:07

andreyaksenov changed the base branch from 3.0 to 3.0-config November 3, 2023 08:08

andreyaksenov changed the base branch from 3.0-config to 3.0 November 3, 2023 08:08

andreyaksenov force-pushed the 3.0-config-etcd branch 2 times, most recently from e906c23 to 6107111 Compare November 3, 2023 08:10

andreyaksenov changed the title ~~3.0 etcd config~~ 3.0 configuration Nov 3, 2023

andreyaksenov force-pushed the 3.0-config-etcd branch from 6107111 to d5024c7 Compare November 3, 2023 08:36

andreyaksenov mentioned this pull request Nov 3, 2023

Update CLI options documentation #3832

Closed

andreyaksenov force-pushed the 3.0-config-etcd branch 8 times, most recently from 82bc916 to c3e8597 Compare November 8, 2023 12:16

andreyaksenov requested review from p7nov and xuniq November 8, 2023 14:36

andreyaksenov force-pushed the 3.0-config-etcd branch from c3e8597 to 5cbe38a Compare November 8, 2023 15:26

p7nov reviewed Nov 10, 2023

View reviewed changes

andreyaksenov force-pushed the 3.0-config-etcd branch 2 times, most recently from fdf8244 to 0c5d37c Compare November 10, 2023 10:48

andreyaksenov force-pushed the 3.0 branch from e160f44 to 7b587a0 Compare November 10, 2023 13:00

andreyaksenov added 3 commits November 10, 2023 16:00

3.0 config

49fa44e

3.0 etcd config

a37f513

3.0 config - update per review

382b36b

andreyaksenov force-pushed the 3.0-config-etcd branch from 0c5d37c to c24f5b0 Compare November 10, 2023 13:01

p7nov approved these changes Nov 13, 2023

View reviewed changes

p7nov reviewed Nov 13, 2023

View reviewed changes

3.0 config - update per TW review

bb1784e

andreyaksenov force-pushed the 3.0-config-etcd branch from c24f5b0 to bb1784e Compare November 13, 2023 07:20

xuniq reviewed Nov 13, 2023

View reviewed changes

3.0 config - update per TW review 2

26b2d78

xuniq approved these changes Nov 13, 2023

View reviewed changes

andreyaksenov merged commit f1277cb into 3.0 Nov 13, 2023

andreyaksenov deleted the 3.0-config-etcd branch November 13, 2023 10:17

andreyaksenov added a commit that referenced this pull request Nov 21, 2023

3.0 configuration (#3825)

ad4dcc7

Document a new declarative configuration approach: local YAML file and etcd config

andreyaksenov added a commit that referenced this pull request Dec 6, 2023

3.0 configuration (#3825)

289f260

Document a new declarative configuration approach: local YAML file and etcd config


		* Since version 3.0: In a YAML file.

		In a YAML file, you can provide the full cluster topology and specify all configuration options.

	To reference such variables in a configuration file, use double curly braces.
	To reference these variables in a configuration file, enclose them in double curly braces with whitespaces.


		Below are a few examples that show how to set environment variables of different types, like string, number, array, or map:

		* (String) In the example below, ``TT_IPROTO_LISTEN`` is used to specify a :ref:`listening host and port <configuration_options_connection>` values:

	Tarantool provides the ability to configure the full topology of a cluster and set parameters specific for concrete instances, like connection settings, memory used to store data, logging, and snapshot settings.
	Tarantool provides the ability to configure the full topology of a cluster and set parameters specific for concrete instances, such as connection settings, memory used to store data, logging, and snapshot settings.


		There are two approaches to configuring Tarantool:

		* Since version 3.0: In a YAML format.

	* Since version 3.0: In a YAML format.
	* Since version 3.0: In the YAML format.

	A replica set is a pack of instances that operate on copies of the same databases.
	A replica set is a pack of instances that operate on same data chunks.

	and suppose the environment variable LISTEN_URI contains 3301,
	and suppose the environment variable ``LISTEN_URI`` contains 3301,


		.. box_cfg_legacy_note_end

		This topic covers specifics of configuring Tarantool in code using the ``box.cfg`` API.

	This topic covers specifics of configuring Tarantool in code using the ``box.cfg`` API.
	This topic covers the specifics of configuring Tarantool in code using the ``box.cfg`` API.


		export TT_LISTEN="localhost:3301?param1=value1&param2=value2"

		An empty variable (``TT_LISTEN=``) has the same effect as an unset one meaning that the corresponding configuration parameter won't be set when calling ``box.cfg{}``.

		* ``TT_LISTEN`` -- corresponds to the ``box.cfg.listen`` option.
		* ``TT_MEMTX_DIR`` -- corresponds to the ``box.cfg.memtx_dir`` option.

	If a host is omitted, then "0.0.0.0" or "[::]" is assumed
	If a host is omitted, then "0.0.0.0" or "[::]" is assumed,

		where a URI is expected, for example, "unix/:/tmp/unix_domain_socket.sock" or
		simply "/tmp/unix_domain_socket.sock".

		In this example, the following options are configured in addition to an etcd endpoint and key prefix:

Conversation

andreyaksenov commented Oct 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

p7nov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andreyaksenov Nov 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

p7nov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andreyaksenov Nov 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andreyaksenov commented Oct 31, 2023 •

edited

Loading

andreyaksenov Nov 10, 2023 •

edited

Loading

andreyaksenov Nov 13, 2023 •

edited

Loading