Skip to content

New Instructions for installing snap including enabling interfaces #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sklassen wants to merge 19 commits into from
Closed

New Instructions for installing snap including enabling interfaces #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
63a7025
Add files via upload
sklassen Nov 5, 2018
3cb2eef
Update HOWTO.md
wohali Nov 5, 2018
12e4bf9
Update README.md
wohali Nov 5, 2018
35ac4ad
Update README.md
wohali Nov 5, 2018
c6f4903
Typo: was /etc/default.d/local.ini should just be /etc/local.ini; add…
sklassen Nov 5, 2018
cd61d05
Added --edge to installation
sklassen Nov 5, 2018
ca78777
Changed couchdb.ini from local.d to default.d
sklassen Nov 5, 2018
aee12ce
Switched config order to standard: default.ini, default.d, local.ini,…
sklassen Nov 5, 2018
33ea336
couchdb.ini has been moved from local.d to default.d
sklassen Nov 5, 2018
c32bc00
Re-wrote configuration section to reflect standard order.
sklassen Nov 5, 2018
1d1c4f6
Added a 90-override.ini file to ensure HTTP changes go in the last file
Nov 6, 2018
3af3924
Pared back the list of settable options to bare min
Nov 6, 2018
36cd4ba
Rewrote README and HOWTO to reflect new order of configuration
Nov 6, 2018
4b9b5e7
md formating changes
sklassen Nov 7, 2018
327eb25
Corrected white spacing
sklassen Nov 7, 2018
618f8c1
white spacing
sklassen Nov 7, 2018
c1393c1
Emphasized the q=1 parameter
sklassen Nov 7, 2018
dd32cf0
On fresh installation, copy the local.ini from the rel directory
Nov 7, 2018
f5b6463
Merge branch 'snap_instructions' of https://github.com/sklassen/couch…
Nov 7, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions snap/BUILD.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Building snaps

## Prerequisites

CouchDB requires Ubuntu 16.04. If building on 18.04, then LXD might be useful.

1. `lxc launch ubuntu:16.04 couchdb-pkg`
1. `lxc exec couchdb-pkg bash`
1. `sudo apt update`
1. `sudo apt install snapd snapcraft`

1. `git clone https://github.com/couchdb/couchdb-pkg.git`
1. `cd couchdb-pkg`

## How to do it

1. Edit `snap/snapcraft.yaml` to point to the correct tag (e.g. `2.2.0`)
1. `snapcraft`

## Instalation

You may need to pull the LXD file to the host system.

$ lxc file pull couchdb-pkg/root/couchdb-pkg/couchdb_2.2.0_amd64.snap /tmp/couchdb_2.2.0_amd64.snap

The self crafted snap will need to be installed in devmode

$ sudo snap install /tmp/couchdb_2.2.0_amd64.snap --devmode


172 changes: 85 additions & 87 deletions snap/HOWTO.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,109 +1,107 @@
# HOW TO install a cluster using snap

# Create three machines

In the instruction below, we are going to set up a three -- the miniumn number needed to gain performace improvement -- Couch cluster database. In this potted example we will be using LXD.

We launch a new container and install couchdb on one machine

1. localhost> `lxc launch ubuntu:18.04 couchdb-c1`
1. localhost> `lxc exec couchdb-c1 bash`
1. couchdb-c1> `apt update`
1. couchdb-c1> `snap install couchdb`
1. couchdb-c1> `logout`

Here we use LXD copy function to speed up the test
```
lxc copy couchdb-c1 couchdb-c2
lxc copy couchdb-c1 couchdb-c3
lxc copy couchdb-c1 cdb-backup
lxc start couchdb-c2
lxc start couchdb-c3
lxc start cdb-backup
```

# Configure CouchDB (using the snap tool)

We are going to need the IP addresses. You can find them here.
```
lxc list
```

Now lets use the snap configuration tool to set the configuration files.
```
lxc exec couchdb-c1 snap set couchdb name=couchdb@10.210.199.199 setcookie=monster admin=password bind-address=0.0.0.0
lxc exec couchdb-c2 snap set couchdb name=couchdb@10.210.199.254 setcookie=monster admin=password bind-address=0.0.0.0
lxc exec couchdb-c3 snap set couchdb name=couchdb@10.210.199.24 setcookie=monster admin=password bind-address=0.0.0.0
```
The backup machine we will leave as a single instance and no sharding.
```
lxc exec cdb-backup snap set couchdb name=couchdb@127.0.0.1 setcookie=monster admin=password bind-address=0.0.0.0 n=1 q=1
```

The snap must be restarted for the new configurations to take affect.
```
lxc exec couchdb-c1 snap restart couchdb
lxc exec couchdb-c2 snap restart couchdb
lxc exec couchdb-c3 snap restart couchdb
lxc exec cdb-backup snap restart couchdb
## Create three nodes

In the example below, we are going to set up a three node CouchDB cluster. (Three is the minimum number needed to support clustering features.) We'll also set up a separate, single machine for making backups. In this example we will be using LXD.

We launch a (single) new container, install couchdb via snap from the store and enable interfaces.
```bash
1. localhost> lxc launch ubuntu:18.04 couchdb-c1
1. localhost> lxc exec couchdb-c1 bash
1. couchdb-c1> apt update
1. couchdb-c1> snap install couchdb --edge
1. couchdb-c1> snap connect couchdb:mount-observe
1. couchdb-c1> snap connect couchdb:process-control
1. couchdb-c1> logout
```
Back on localhost, we can then use the LXD copy function to speed up installation:
```bash
$ lxc copy couchdb-c1 couchdb-c2
$ lxc copy couchdb-c1 couchdb-c3
$ lxc copy couchdb-c1 cdb-backup
$ lxc start couchdb-c2
$ lxc start couchdb-c3
$ lxc start cdb-backup
```
## Configure CouchDB using the snap tool

We are going to need the IP addresses:
```bash
$ lxc list
```
Now, again from localhost, and using the `lxc exec` commond, we will use the snap configuration tool to set the
various configuration files.
```bash
$ lxc exec couchdb-c1 snap set couchdb name=couchdb@10.210.199.199 setcookie=monster admin=Be1stDB bind-address=0.0.0.0
$ lxc exec couchdb-c2 snap set couchdb name=couchdb@10.210.199.254 setcookie=monster admin=Be1stDB bind-address=0.0.0.0
$ lxc exec couchdb-c3 snap set couchdb name=couchdb@10.210.199.24 setcookie=monster admin=Be1stDB bind-address=0.0.0.0
```
The backup machine we will configure as a single instance (n=1) and a single shard (q=1).
```bash
$ lxc exec cdb-backup snap set couchdb name=couchdb@127.0.0.1 setcookie=monster admin=Be1stDB bind-address=0.0.0.0 n=1 q=1
```
Each snap must be restarted for the new configurations to take affect.
```bash
$ lxc exec couchdb-c1 snap restart couchdb
$ lxc exec couchdb-c2 snap restart couchdb
$ lxc exec couchdb-c3 snap restart couchdb
$ lxc exec cdb-backup snap restart couchdb
```
The configuration files are stored here.
```
lxc exec cdb-backup cat /var/snap/couchdb/current/etc/vm.args
lxc exec cdb-backup cat /var/snap/couchdb/current/etc/local.d/*
```bash
$ lxc exec cdb-backup cat /var/snap/couchdb/current/etc/vm.args
$ lxc exec cdb-backup cat /var/snap/couchdb/current/etc/local.d/*
```
Any changes to couchdb from the http configutation tool are made here
```
lxc exec cdb-backup cat /var/snap/couchdb/current/etc/local.d/local.ini
```bash
$ lxc exec cdb-backup cat /var/snap/couchdb/current/etc/local.ini
```

# Configure CouchDB Cluster (using the http interface)
## Configure CouchDB Cluster (using the http interface)

Now we set up the cluster via the http front-end. This only needs to be run once on the first machine. The last command syncs with the other nodes and creates the standard databases.
```
curl -X POST -H "Content-Type: application/json" http://admin:password@10.210.199.199:5984/_cluster_setup -d '{"action": "add_node", "host":"10.210.199.254", "port": "5984", "username": "admin", "password":"password"}'
curl -X POST -H "Content-Type: application/json" http://admin:password@10.210.199.199:5984/_cluster_setup -d '{"action": "add_node", "host":"10.210.199.24", "port": "5984", "username": "admin", "password":"password"}'
curl -X POST -H "Content-Type: application/json" http://admin:password@10.210.199.199:5984/_cluster_setup -d '{"action": "finish_cluster"}'
Now we set up the cluster via the http front-end. This only needs to be run once on the first machine. The last command
syncs with the other nodes and creates the standard databases.
```bash
$ curl -X POST -H "Content-Type: application/json" http://admin:Be1stDB@10.210.199.199:5984/_cluster_setup -d '{"action": "add_node", "host":"10.210.199.254", "port": "5984", "username": "admin", "password":"Be1stDB"}'
$ curl -X POST -H "Content-Type: application/json" http://admin:Be1stDB@10.210.199.199:5984/_cluster_setup -d '{"action": "add_node", "host":"10.210.199.24", "port": "5984", "username": "admin", "password":"Be1stDB"}'
$ curl -X POST -H "Content-Type: application/json" http://admin:Be1stDB@10.210.199.199:5984/_cluster_setup -d '{"action": "finish_cluster"}'
```
Now we have a functioning three node cluster.

# An Example Database
## An Example Database

Let's create an example database ...
```bash
$ curl -X PUT http://admin:password@10.210.199.199:5984/example
$ curl -X PUT http://admin:password@10.210.199.199:5984/example/aaa -d '{"test":1}' -H "Content-Type: application/json"
$ curl -X PUT http://admin:password@10.210.199.199:5984/example/aab -d '{"test":2}' -H "Content-Type: application/json"
$ curl -X PUT http://admin:password@10.210.199.199:5984/example/aac -d '{"test":3}' -H "Content-Type: application/json"
```
curl -X PUT http://admin:password@10.210.199.199:5984/example
curl -X PUT http://admin:password@10.210.199.199:5984/example/aaa -d '{"test":1}' -H "Content-Type: application/json"
curl -X PUT http://admin:password@10.210.199.199:5984/example/aab -d '{"test":2}' -H "Content-Type: application/json"
curl -X PUT http://admin:password@10.210.199.199:5984/example/aac -d '{"test":3}' -H "Content-Type: application/json"
```
... And see that it is sync'd accross the three nodes.
```
curl -X GET http://admin:password@10.210.199.199:5984/example/_all_docs
curl -X GET http://admin:password@10.210.199.254:5984/example/_all_docs
curl -X GET http://admin:password@10.210.199.24:5984/example/_all_docs
... And see that it is created on all three nodes.
```bash
$ curl -X GET http://admin:password@10.210.199.199:5984/example/_all_docs
$ curl -X GET http://admin:password@10.210.199.254:5984/example/_all_docs
$ curl -X GET http://admin:password@10.210.199.24:5984/example/_all_docs
```
# Backing Up CouchDB
## Backing Up CouchDB

Our back up server is on 10.210.199.242. We will manually replicate this from one (anyone) of the nodes.
Our backup server is on 10.210.199.242. We will manually replicate to this from one (can be any one) of the nodes.
```bash
$ curl -X POST http://admin:Be1stDB@10.210.199.242:5984/_replicate -d '{"source":"http://10.210.199.199:5984/example", "target":"example", "continuous":false,"create_target":true}' -H "Content-Type: application/json"
$ curl -X GET http://admin:Be1stDB@10.210.199.242:5984/example/_all_docs
```
curl -X POST http://admin:password@10.210.199.242:5984/_replicate -d '{"source":"http://10.210.199.199:5984/example", "target":"example", "continuous":false,"create_target":true}' -H "Content-Type: application/json"
curl -X GET http://admin:password@10.210.199.242:5984/example/_all_docs
Whereas the data store for the clusters nodes is sharded:
```bash
$ lxc exec couchdb-c1 ls /var/snap/couchdb/common/data/shards/
```
The data store for the clusters nodes are sharded
```
lxc exec couchdb-c1 ls /var/snap/couchdb/common/2.x/data/shards/
```

The backup database is a single file.
```
lxc exec cdb-backup ls /var/snap/couchdb/common/2.x/data/shards/00000000-ffffffff/
The backup database is a single directory:
```bash
$ lxc exec cdb-backup ls /var/snap/couchdb/common/data/shards/
```

# Monitoring CouchDB

The logs, by default, are captured by journald
```
lxc exec couchdb-c1 bash
journalctl -u snap.couchdb -f
```
## Monitoring CouchDB

The logs, by default, are captured by journald. First connect to the node in question:
`$ lxc exec couchdb-c1 bash`
Then, show logs as usual. couchdb is likely prefixed with 'snap'
`$ journalctl -u snap.couchdb -f`
101 changes: 62 additions & 39 deletions snap/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,61 +1,84 @@
# Building snaps

## Prerequisites

CouchDB requires Ubuntu 16.04. If building on 18.04, then LXD might be useful.

1. `lxc launch ubuntu:16.04 couchdb-pkg`
1. `lxc exec couchdb-pkg bash`
1. `sudo apt update`
1. `sudo apt install snapd snapcraft`

1. `git clone https://github.com/couchdb/couchdb-pkg.git`
1. `cd couchdb-pkg`

## How to do it
# Snap Instalation

1. Edit `snap/snapcraft.yaml` to point to the correct tag (e.g. `2.2.0`)
1. `snapcraft`
## Downloading from the snap store

# Snap Instalation
The snap can be installed from a file or directly from the snap store. It is, for the moment, listed in the edge channel.

You may need to pull the LXD file to the host system.
```
$ sudo snap install couchdb --edge
```
## Enable snap permissions

$ lxc file pull couchdb-pkg/root/couchdb-pkg/couchdb_2.2.0_amd64.snap /tmp/couchdb_2.2.0_amd64.snap
The snap installation uses AppArmor to protect your system. CouchDB requests access to two interfaces: mount-observe, which
is used by the disk compactor to know when to initiate a cleanup; and process-control, which is used by the indexer to set
the priority of couchjs to 'nice'. These two interfaces, while not required, are useful. If they are not enabled, CouchDB will
still run, but you will need to run the compactor manually and couchjs may put a heavy load on the system when indexing.

The self crafted snap will need to be installed in devmode
To connect the interfaces type:
```
$ sudo snap connect couchdb:mount-observe
$ sudo snap connect couchdb:process-control
```
## Snap configuration

$ sudo snap install /tmp/couchdb_2.2.0_amd64.snap --devmode
There are two levels of hierarchy within couchdb configuration.

# Snap Configuration
The default layer is stored in /snap/couchdb/current/rel/couchdb/etc/ and the default.ini is
referred to before the default.d directory. In the snap installation this is mounted read-only.

There are two levels of erlang and couchdb configuration hierarchy.
The local layer is stored in /var/snap/couchdb/current/etc/ on the writable /var mount.
Within this second layer, configurations are set local.ini (single file co-mingled sections) or
the local.d directory (one file per section). Configuration management tools (like puppet, chef,
ansible, salt) often use the former (local.ini). The "snap set" command works within the
latter (local.d). Entries in local.d supersede those in the local.ini directory.

The default layer is stored in /snap/couchdb/current/rel/couchdb/etc/ and is read only.
The user override layer, is stored in /var/snap/couchdb/current/etc/ and is writable.
Within this second layer, configurations are set with the local.d directory (one file
per section) or the local.ini (co-mingled). The "snap set" command works with the
former (local.d) and couchdb http configuration overwrites the latter (local.ini).
Entries in local.ini supersede those in the local.d directory.
Changes made via the http configuration will write into local.d/90-override.ini.

The name of the erlang process and the security cookie used is set in vm.args file.
This can be set through the snap native configuration. For example, when setting up
This should be set through the snap native configuration. For example, when setting up
a cluster over several machines the convention is to set the erlang
name to couchdb@your.ip.address. Both erlang and couchdb configuration changes can be
made at the same time.
name to couchdb@your.ip.address.

Both erlang and couchdb configuration changes can be made at the same time.

```
$ sudo snap set couchdb name=couchdb@216.3.128.12 setcookie=cutter admin=Be1stDB bind-address=0.0.0.0
```

Snap set variable can not contain underscore character, but any dashes are converted to underscore when
writing to file. Wrap double quotes around any bracets and avoid spaces.

$ sudo snap set couchdb delayed-commits=true erlang="{couch_native_process,start_link,[]}"
writing to file. Wrap double quotes around any brackets or spaces.

```
$ sudo snap set couchdb erlang="{couch_native_process,start_link,[]}"
```
Snap Native Configuration changes only come into effect after a restart


```
$ sudo snap restart couchdb
```
Snap Native Configuration has only been enabled for a few options essential to inital installation or items
that are not white-listed for configuration over HTTP. Other options that can be set via snap are: CHTTPD's "port";
Cluster options: "n", "q"; the log options "writer","file","level". And the Native Query Servers
Options "query" and "erlang".

Other options can be set via configuration over HTTP, as below.

```
$ curl -X PUT http://admin:Be1stDB@216.3.128.12:5984/_node/_local/_config/couchdb/delayed-commits -d '"true"'
```
Which has the advantage of not requiring restarting the application.

For anything not covered by snap set or the configuration over http, you can edit
the /var/snap/couchdb/current/etc files by hand.

## Example Cluster

See the [HOWTO][1] file to see an example of a three node cluster and further notes.

## Building a Private Snap

# Example Cluster
If you want to build your own snap file from source see the [BUILD][2] for instructions.

See the HOWTO.md file to see an example of a three node cluster.
[1]: HOWTO.md
[2]: BUILD.md

Loading