From c04f34d1c1bf4596168e29d91b7c1e431038c689 Mon Sep 17 00:00:00 2001 From: Jan Lehnardt Date: Wed, 16 May 2018 18:14:36 +0200 Subject: [PATCH] remove mentions on external process shenanigans --- src/api/ddoc/views.rst | 4 - src/config/externals.rst | 187 ---------------------------- src/config/index.rst | 1 - src/config/services.rst | 7 -- src/externals.rst | 259 --------------------------------------- src/index.rst | 1 - 6 files changed, 459 deletions(-) delete mode 100644 src/config/externals.rst delete mode 100644 src/externals.rst diff --git a/src/api/ddoc/views.rst b/src/api/ddoc/views.rst index 006e1372..6bcb6020 100644 --- a/src/api/ddoc/views.rst +++ b/src/api/ddoc/views.rst @@ -279,10 +279,6 @@ issues. These include: database and then access the view to force the corresponding view index to be updated. -- Use a monitor with the :ref:`update notification ` - section of the CouchDB configuration file to monitor for changes to your - database, and trigger a view query to force the view to be updated. - None of these can completely eliminate the need for the indexes to be rebuilt or updated when the view is accessed, but they may lessen the effects on end-users of the index update affecting the user experience. diff --git a/src/config/externals.rst b/src/config/externals.rst deleted file mode 100644 index 32f5bf39..00000000 --- a/src/config/externals.rst +++ /dev/null @@ -1,187 +0,0 @@ -.. Licensed under the Apache License, Version 2.0 (the "License"); you may not -.. use this file except in compliance with the License. You may obtain a copy of -.. the License at -.. -.. http://www.apache.org/licenses/LICENSE-2.0 -.. -.. Unless required by applicable law or agreed to in writing, software -.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -.. License for the specific language governing permissions and limitations under -.. the License. - -.. default-domain:: config -.. highlight:: ini - -================== -External Processes -================== - -.. _config/os_daemons: - -OS Daemons -========== - -.. config:section:: os_daemons :: OS Daemons - - This is a simple feature that allows users to configure CouchDB so that it - maintains a given OS level process alive. If the process dies for any - reason, CouchDB will restart it. If the process restarts too often, then - CouchDB will mark it has halted and not attempt to restart it. The default - max restart rate is ``3`` times in the last ``5`` seconds. These parameters - are :section:`adjustable `. - - Commands that are started in this manner will have access to a simple API - over stdio to request configuration parameters or to add log statements to - CouchDB's logs. - - To configure an OS process as a CouchDB os_daemon, create a section in your - `local.ini` like such:: - - [os_daemons] - daemon_name = /path/to/command -with args - - This will make CouchDB bring up the command and attempt to keep it alive. - To request a configuration parameter, an `os_daemon` can write a simple - JSON message to stdout like such: - - .. code-block:: none - - ["get", "os_daemons"]\n - - which would return: - - .. code-block:: none - - {"daemon_name": "/path/to/command -with args"}\n - - Or: - - .. code-block:: none - - ["get", "os_daemons", "daemon_name"]\n - - which would return: - - .. code-block:: none - - "/path/to/command -with args"\n - - There's no restriction on what configuration variables are visible. There's - also no method for altering the configuration. - - If you would like your OS daemon to be restarted in the event that the - configuration changes, you can send the following messages: - - .. code-block:: none - - ["register", $(SECTION)]\n - - When anything in that section changes, your OS process will be rebooted so - it can pick up the new configuration settings. If you want to listen for - changes on a specific key, you can send something like: - - .. code-block:: none - - ["register", $(SECTION), $(KEY)]\n - - In this case, CouchDB will only restart your daemon if that exact - section/key pair changes, instead of anything in that entire section. - - Logging commands look like: - - .. code-block:: none - - ["log", $(JSON_MESSAGE)]\n - - Where ``$(JSON_MESSAGE)`` is arbitrary JSON data. These messages are logged - at the 'info' level. If you want to log at a different level you can pass - messages like such: - - .. code-block:: none - - ["log", $(JSON_MESSAGE), {"level": $(LEVEL)}]\n - - Where ``$(LEVEL)`` is one of "debug", "info", or "error". - - When implementing a daemon process to be managed by CouchDB you should - remember to use a method like checking the parent process id or if stdin - has been closed. These flags can tell you if your daemon process has been - orphaned so you can exit cleanly. - - There is no interactivity between CouchDB and the running process, but you - can use the OS Daemons service to create new HTTP servers and responders - and then use the new proxy service to redirect requests and output to the - CouchDB managed service. For more information on proxying, see - :ref:`http-proxying`. For further background on the OS Daemon service, - see :ref:`externals`. - -.. _config/os_daemon_settings: - -OS Daemons settings -=================== - -.. config:section:: os_daemon_settings :: OS Daemons settings - - .. config:option:: max_retries :: Maximum restart retries - - Specifies maximum attempts to run :section:`os_daemons` before mark - them halted:: - - [os_daemon_settings] - max_retries = 3 - - .. config:option:: retry_time :: Delay between restart attempts - - Delay in seconds between :section:`os_daemons` restarts:: - - [os_daemon_settings] - retry_time = 5 - -.. _update-notifications: -.. _config/update_notification: - -Update notifications -==================== - -.. config:section:: update_notification :: Update notifications - - CouchDB is able to spawn OS processes to notify them about recent databases - updates. The notifications are in form of JSON messages sent as a line of - text, terminated by ``CR`` (``\n``) character, to the OS processes through - `stdout`:: - - [update_notification] - ;unique notifier name=/full/path/to/exe -with "cmd line arg" - index_updater = ruby /usr/local/bin/index_updater.rb - - The update notification messages are depend upon of event type: - - - **Database created**: - - .. code-block:: javascript - - {"type":"created","db":"dbname"} - - - **Database updated**: this event raises when any document gets updated - for specified database: - - .. code-block:: javascript - - {"type":"updated","db":"dbname"} - - - **Design document updated**: for design document updates there is special - event raised in additional to regular db update one: - - .. code-block:: javascript - - {"type":"ddoc_updated","db":"dbname","id":"_design/ddoc_name"} - - - **Database deleted**: - - .. code-block:: javascript - - {"type":"deleted","db":"dbname"} - - .. note:: - New line (``\n``) trailing character was removed from examples. diff --git a/src/config/index.rst b/src/config/index.rst index 93dca8b2..439a9808 100644 --- a/src/config/index.rst +++ b/src/config/index.rst @@ -29,7 +29,6 @@ Configuring CouchDB logging replicator query-servers - externals http-handlers services misc diff --git a/src/config/services.rst b/src/config/services.rst index 90f48f28..5556e872 100644 --- a/src/config/services.rst +++ b/src/config/services.rst @@ -71,13 +71,6 @@ CouchDB Daemonized Mini Apps [daemons] index_server={couch_index_server, start_link, []} - .. config:option:: os_daemons - - :ref:`OS Daemons ` manager:: - - [daemons] - os_daemons={couch_os_daemons, start_link, []} - .. config:option:: query_servers :ref:`Query servers ` manager:: diff --git a/src/externals.rst b/src/externals.rst deleted file mode 100644 index 2ec16436..00000000 --- a/src/externals.rst +++ /dev/null @@ -1,259 +0,0 @@ -.. Licensed under the Apache License, Version 2.0 (the "License"); you may not -.. use this file except in compliance with the License. You may obtain a copy of -.. the License at -.. -.. http://www.apache.org/licenses/LICENSE-2.0 -.. -.. Unless required by applicable law or agreed to in writing, software -.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -.. License for the specific language governing permissions and limitations under -.. the License. - -.. _externals: - -===================== -CouchDB Externals API -===================== - -:Author: Paul Joseph Davis -:Date: 2010-09-26 -:Source: http://davispj.com/2010/09/26/new-couchdb-externals-api.html - -For a bit of background, CouchDB has had an API for managing `external OS -processes`_ that are capable of handling HTTP requests for a given -URL prefix. These OS processes communicate with CouchDB using JSON over -stdio. They're dead simple to write and provide CouchDB users an easy way to -extend CouchDB functionality. - -Even though they're dead simple to write, there are a few issues. The -implementation in CouchDB does not provide fancy pooling semantics. The -current API is explicitly synchronous which prevents people from writing -event driven code in an external handler. In the end, they may be simple, -but their simplicity is also quite limiting. - -During CouchCamp a few weeks ago I had multiple discussions with various people -that wanted to see the _externals API modified in slight ways that weren't -mutually compatible. After having multiple discussions with multiple people -we formed a general consensus on what a new API could look like. - -The New Hotness -=============== - -So the first idea for improving the _external API was to make CouchDB act as -a reverse proxy. This would allow people to write an HTTP server that was as -simple or as complicated as they wanted. It will allow people to change their -networking configuration more easily and also allow for external processes to -be hosted on nodes other than the one running CouchDB. Bottom line, it not -only allows us to have similar semantics as _externals, it provides a lot more -fringe benefits as well. I'm always a fan of extra awesomeness. - -After hitting on the idea of adding a reverse proxy, people quickly pointed -out that it would require users to start manually managing their external -processes using something like `Runit`_ or `Supervisord`_. After some -more discussions I ran into people that wanted something like _externals that -didn't handle HTTP requests. After that it was easy to see that adding a second -feature that managed OS processes was the way to go. - -I spent this weekend implementing both of these features. Both are at the stage -of working but requiring more testing. In the case of the HTTP proxy I have no -tests because I can't decide how to test the thing. If you have ideas, I'd -sure like to hear them. - -**[Update]**: I woke up the other morning realizing that I was being an idiot -and that Erlang is awesome. There's no reason that I can't have an HTTP client, -proxy, and server all hosted in the same process. So that's what I did. It -turns out to be a fairly nice way of configuring matching assertions between -the client and the server to test the proxy transmissions. - -How does it work? - HTTP Proxying -================================= - -To configure a :ref:`proxy handler `, edit your `local.ini` and -add a section like such:: - - [httpd_global_handlers] - _fti = {couch_httpd_proxy, handle_proxy_req, <<"http://127.0.0.1:5985">>} - -This would be approximately what you'd need to do to get `CouchDB-Lucene`_ -handled through this interface. The URL you use to access a query would be: - - http://127.0.0.1:5984/_fti/db_name/_design/foo/by_content?q=hello - -A couple things to note here. Anything in the path after the configured proxy -name ("_fti" in this case) will be appended to the configured destination URL -("http://127.0.0.1:5985" in this case). The query string and any associated -body will also be proxied transparently. - -Also, of note is that there's nothing that limits on what resources can be -proxied. You're free to choose any destination that the CouchDB node is capable -of communicating with. - -How does it work? - OS Daemons -============================== - -The second part of the new API gives CouchDB simple OS process management. When -CouchDB boots it will start each configured OS daemon. If one of these daemons -fails at some point, it will be restarted. If one of these daemons fails too -often, CouchDB will stop attempting to start it. - -OS daemons are one-to-one. For each daemon, CouchDB will make sure that exactly -one instance of it is alive. If you have something where you want multiple -processes, you need to either tell CouchDB about each one, or have a main -process that forks off the required sub-processes. - -To configure an :config:section:`OS daemon `, add this to your -`local.ini`:: - - [os_daemons] - my_daemon = /path/to/command -with args - -Configuration API ------------------ - -As an added benefit, because stdio is now free, I implemented a simple API -that OS daemons can use to read the configuration of their CouchDB host. This -way you can have them store their configuration inside CouchDB's config system -if you desire. Or they can peek at things like the -:config:option:`httpd/bind_address` and :config:option:`httpd/port` that CouchDB -is using. - -A request for a config section looks like this:: - - ["get", "os_daemons"]\n - -And the response:: - - {"my_daemon": "/path/to/command -with args"}\n - -Or to get a specific key:: - - ["get", "os_daemons", "my_daemon"]\n - -And the response:: - - "/path/to/command -with args"\n - -All requests and responses are terminated with a newline (indicated by ``\n``). - -Logging API ------------ - -There's also an API for adding messages to CouchDB's logs. Its simply:: - - ["log", $MESG]\n - -Where ``$MESG`` is any arbitrary JSON. There is no response from this command. As -with the config API, the trailing ``\n`` represents a newline byte. - -Dynamic Daemons ---------------- - -The OS daemons react in real time to changes to the configuration system. If -you set or delete keys in the :config:section:`os_daemons` section, -the corresponding daemons will be started or killed as appropriate. - -Neat. But So What? -================== - -It was suggested that a good first demo would be a `Node.js`_ handler. So, I -present to you a "Hello, World" Node.js handler. Also, remember that this -currently relies on code in my fork on `GitHub`_. - -File `node-hello-world.js`: - -.. code-block:: javascript - - var http = require('http'); - var sys = require('sys'); - - // Send a log message to be included in CouchDB's - // log files. - - var log = function(mesg) { - console.log(JSON.stringify(["log", mesg])); - } - - // The Node.js example HTTP server - - var server = http.createServer(function (req, resp) { - resp.writeHead(200, {'Content-Type': 'text/plain'}); - resp.end('Hello World\n'); - log(req.method + " " + req.url); - }) - - // We use stdin in a couple ways. First, we - // listen for data that will be the requested - // port information. We also listen for it - // to close which indicates that CouchDB has - // exited and that means its time for us to - // exit as well. - - var stdin = process.openStdin(); - - stdin.on('data', function(d) { - server.listen(parseInt(JSON.parse(d))); - }); - - stdin.on('end', function () { - process.exit(0); - }); - - // Send the request for the port to listen on. - - console.log(JSON.stringify(["get", "node_hello", "port"])); - -File `local.ini` (Just add these to what you have): - -.. code-block:: ini - - [log] - level = info - - [os_daemons] - node_hello = /path/to/node-hello-world.js - - [node_hello] - port = 8000 - - [httpd_global_handlers] - _hello = {couch_httpd_proxy, handle_proxy_req, <<"http://127.0.0.1:8000">>} - -And then start CouchDB and try: - -.. code-block:: bash - - $ curl -v http://127.0.0.1:5984/_hello - * About to connect() to 127.0.0.1 port 5984 (#0) - * Trying 127.0.0.1... connected - * Connected to 127.0.0.1 (127.0.0.1) port 5984 (#0) - > GET /_hello HTTP/1.1 - > User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3 - > Host: 127.0.0.1:5984 - > Accept: */* - > - < HTTP/1.1 200 - < Transfer-Encoding: chunked - < Server: CouchDB (Erlang/OTP) - < Date: Mon, 27 Sep 2010 01:13:37 GMT - < Content-Type: text/plain - < Connection: keep-alive - < - Hello World - * Connection #0 to host 127.0.0.1 left intact - * Closing connection #0 - -The corresponding CouchDB logs look like:: - - Apache CouchDB 1.5.0 (LogLevel=info) is starting. - Apache CouchDB has started. Time to relax. - [info] [<0.31.0>] Apache CouchDB has started on http://127.0.0.1:5984/ - [info] [<0.105.0>] 127.0.0.1 - - 'GET' /_hello 200 - [info] [<0.95.0>] Daemon "node-hello" :: GET / - -.. _external OS processes: http://wiki.apache.org/couchdb/ExternalProcesses -.. _Runit: http://smarden.org/runit/ -.. _Supervisord: http://supervisord.org/ -.. _Node.js: http://nodejs.org/ -.. _GitHub: http://github.com/davisp/couchdb/tree/new_externals -.. _CouchDB-Lucene: https://github.com/rnewson/couchdb-lucene diff --git a/src/index.rst b/src/index.rst index 09b8f3da..57069220 100644 --- a/src/index.rst +++ b/src/index.rst @@ -30,7 +30,6 @@ Table of Contents replication/index maintenance/index ddocs/index - externals query-server/index fauxton/index best-practices/index