From ba55ae1d5e626504bbd07eb55f664c8fd19b799b Mon Sep 17 00:00:00 2001 From: Nathaniel Freeman Date: Fri, 17 Apr 2020 11:16:37 -0400 Subject: [PATCH 1/5] Initial commit of the work to stand up an accumulo-proxy inside a docker container. I have only implemented support for Accumulo 2.x and by default this first commit contains: * Accumulo 2.0.0 * Hadoop 3.2.1 * Zookeeper 3.5.7 A new document (DOCKER.md) has been created to start to document the implementation and usage guide which should allow others to test this if they so wished. A number of outstanding questions will be posted on the issue, there is also a number of TODOs still required to be implemented that I'm tracking. --- DOCKER.md | 53 +++++++++++++++++++++++++++++++++ Dockerfile | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 140 insertions(+) create mode 100644 DOCKER.md create mode 100644 Dockerfile diff --git a/DOCKER.md b/DOCKER.md new file mode 100644 index 00000000..480a68a1 --- /dev/null +++ b/DOCKER.md @@ -0,0 +1,53 @@ + + +# accumulo-proxy-docker + +A temporary guide on how to run this up in Docker. + +## Build the image using +Invoke the docker build command to create a container image. +```commandline +docker build -t accumulo-proxy:latest . +``` + +## Default Configuration +By default, the container image expects the following to be true: +1. Your accumulo instance is named "myinstance" +2. Your zookeeper is available (and reachable from the container) at localhost:2181 + +## Custom proxy.properties +If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is: +```commandline +docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest +``` + +## Accumulo Instance Configuration +In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS. + +If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry. + +```commandline +docker run --rm -d -p 42424:42424 --add-host "FQDN:IP" --name accumulo-proxy accumulo-proxy:latest +``` + +If you run your Accumulo instance inside a container you can link the containers together using the legacy `--link` approach or place them in the same network ([see official docs](https://docs.docker.com/network/links/)) + +Cleanup using: +```commandline +docker stop accumulo-proxy; +``` \ No newline at end of file diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 00000000..ff39779c --- /dev/null +++ b/Dockerfile @@ -0,0 +1,87 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +FROM openjdk:8 + +EXPOSE 42424 + +WORKDIR /opt/accumulo-proxy + +ARG HADOOP_VERSION=3.2.1 +ARG ZOOKEEPER_VERSION=3.5.7 +ARG ACCUMULO_VERSION=2.0.0 +ARG ACCUMULO_PROXY_VERSION=2.0.0-SNAPSHOT + +ARG HADOOP_HASH=a57962a24d178193349917730bf95cdc99bde9df +ARG ZOOKEEPER_HASH=619928c8553b62775119e3d7d143a4714a160365 +ARG ACCUMULO_HASH=b72bf5c3dcaa25387933a032925046234f30e17a + +# Download from Apache mirrors instead of archive #9 +ENV APACHE_DIST_URLS \ + https://www.apache.org/dyn/closer.cgi?action=download&filename= \ +# if the version is outdated (or we're grabbing the .asc file), we might have to pull from the dist/archive :/ + https://www-us.apache.org/dist/ \ + https://www.apache.org/dist/ \ + https://archive.apache.org/dist/ + +RUN set -eux; \ + download_bin() { \ + local f="$1"; shift; \ + local hash="$1"; shift; \ + local distFile="$1"; shift; \ + local success=; \ + local distUrl=; \ + for distUrl in ${APACHE_DIST_URLS}; do \ + if wget -nv -O "/tmp/${f}" "${distUrl}${distFile}"; then \ + success=1; \ + # Checksum the download + echo "${hash}" "/tmp/${f}" | sha1sum -c -; \ + break; \ + fi; \ + done; \ + [ -n "${success}" ]; \ + };\ + \ + download_bin "apache-zookeeper.tar.gz" "${ZOOKEEPER_HASH}" "zookeeper/zookeeper-${ZOOKEEPER_VERSION}/apache-zookeeper-${ZOOKEEPER_VERSION}-bin.tar.gz"; \ + download_bin "hadoop.tar.gz" "$HADOOP_HASH" "hadoop/core/hadoop-${HADOOP_VERSION}/hadoop-$HADOOP_VERSION.tar.gz"; \ + download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz"; + +# Install the dependencies into /opt/ +RUN mkdir /opt/hadoop && tar xzf /tmp/hadoop.tar.gz -C /opt/hadoop --strip 1 +RUN mkdir /opt/apache-zookeeper && tar xzf /tmp/apache-zookeeper.tar.gz -C /opt/apache-zookeeper --strip 1 +RUN mkdir /opt/accumulo && tar xzf /tmp/accumulo.tar.gz -C /opt/accumulo --strip 1 + +ENV HADOOP_HOME /opt/hadoop +ENV ZOOKEEPER_HOME /opt/apache-zookeeper +ENV ACCUMULO_HOME /opt/accumulo + +# Add some useful readme files +COPY README.md DOCKER.md /tmp/ + +# Add the proxy binary +COPY target/accumulo-proxy-${ACCUMULO_PROXY_VERSION}-bin.tar.gz /tmp/ +RUN tar xzf /tmp/accumulo-proxy-${ACCUMULO_PROXY_VERSION}-bin.tar.gz -C /opt/accumulo-proxy --strip 1 +ENV ACCUMULO_PROXY_HOME /opt/accumulo-proxy + +# Sort out PATH and CLASSPATH configuration +ENV PATH "${PATH}:${ACCUMULO_HOME}/bin" +ENV CLASSPATH=/opt/apache-zookeeper/lib/* + +# TEMPORARY FOR SPEED DURING DEVELOPMENT +#RUN apt-get update && apt-get install vim telnet -y && rm -rf /var/lib/apt/lists/* +#RUN sed -i 's/localhost:2181/host.docker.internal:2181/g' /opt/accumulo-proxy/conf/proxy.properties +#RUN sed -i 's/myinstance/uno/' /opt/accumulo-proxy/conf/proxy.properties + +CMD ["/opt/accumulo-proxy/bin/accumulo-proxy", "-p", "/opt/accumulo-proxy/conf/proxy.properties"] From e620d36d17878fcbe2d74f1d189b6fa3570acff7 Mon Sep 17 00:00:00 2001 From: Nathaniel Freeman Date: Tue, 28 Apr 2020 13:40:49 -0400 Subject: [PATCH 2/5] This commit updates some aspects based on pull-request feedback but also tidies up a number of other aspects. * These are now configured in the same mechanism as accumulo-docker see: https://github.com/apache/accumulo-docker/pull/12/commits/ff8cedf711e6283251d1aaafd67c17386917f6ef * The CLASSPATH env override has been removed. * The DOCKER.md and README.md file has been removed from the container. * Updated the DOCKER.md to try and be more explicit about networking choices depending upon your environment. * I've updated the install process for hadoop, zookeeper and accumulo to retain their version numbers in their install paths. * Symlinks have been added to avoid the need to update references should dependency versions change. --- DOCKER.md | 75 +++++++++++++++++++++++++++++++++++++++++++++++------- Dockerfile | 17 +++---------- 2 files changed, 70 insertions(+), 22 deletions(-) diff --git a/DOCKER.md b/DOCKER.md index 480a68a1..57dde552 100644 --- a/DOCKER.md +++ b/DOCKER.md @@ -25,29 +25,86 @@ Invoke the docker build command to create a container image. docker build -t accumulo-proxy:latest . ``` -## Default Configuration +## Default Configuration and Quickstart By default, the container image expects the following to be true: -1. Your accumulo instance is named "myinstance" +1. Your Accumulo instance name is "myinstance" 2. Your zookeeper is available (and reachable from the container) at localhost:2181 +You can start the proxy using: +```commandline +docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest; +``` + ## Custom proxy.properties If you wish to create advanced proxy.properties configuration changes, you should look to volume mount these in when you invoke the `docker run` command, an example is: ```commandline -docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --add-host"FQDN:IP" --name accumulo-proxy accumulo-proxy:latest +docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-proxy/conf/proxy.properties --network="host" --name accumulo-proxy accumulo-proxy:latest ``` -## Accumulo Instance Configuration -In order for Thrift to communicate with the Accumulo instance you will need to provide the container with the FQDN and IP of the Accumulo instance and its relevant servers (e.g. tservers) to allow it to resolve the required DNS. +## Networking configuration +Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development. + +The proxy container must be able to access both Accumulo and Zookeeper. + +The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section) + +In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document. + +### Host networking -If you are running an Accumulo instance with more than one tserver you should add each tserver's entry with a new `--add-host "FQDN:IP"` entry. +Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM. +You can test if this will work for you by executing the following: ```commandline -docker run --rm -d -p 42424:42424 --add-host "FQDN:IP" --name accumulo-proxy accumulo-proxy:latest +# Start the accumulo-proxy container and enter it +docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash; + +# Install telnet and verify if you can connect to my.host.com:9995 +apt-get update && apt-get install telnet; +telnet my.host.com 9995 ``` -If you run your Accumulo instance inside a container you can link the containers together using the legacy `--link` approach or place them in the same network ([see official docs](https://docs.docker.com/network/links/)) +If telnet can connect, then your container can resolve `my.host.com` correctly with host networking and therefore you can append `--network="host"` to your docker commands. -Cleanup using: +An example of using host networking: +```commandline +docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest +``` + +Note: You do not need to map your ports (-p) if using host networking, but we include it for clarity. + +For more details see the official docker documentation: [Use host Networking](https://docs.docker.com/network/host) + +### Non-Host networking + +If you run outside of a single node linux installation, e.g. Docker for Mac, Docker for Windows or use a VM to isolate your docker engine then you will likely need to take this path. + +Docker allows you to supply additional addresses to be resolved by the container, and these are automatically added by Docker to the /etc/hosts + +For each host add a `--add-host FQDN:IP` entry to your docker run command, you can add multiple entries if need to, see the official docs covering [network settings](https://docs.docker.com/engine/reference/run/#network-settings). + +An example of using this approach: + +```commandline +docker run --rm -d -p 42424:42424 --add-host "my.host.com:192.168.0.1" --name accumulo-proxy accumulo-proxy:latest +``` + +## Cleanup +Once completed you should stop and remove the container. ```commandline docker stop accumulo-proxy; +docker rm accumulo-proxy; +``` + +## Troubleshooting +It can often be difficult to know where to start with troubleshooting inside containers, if you need to enter the container without starting the proxy we support this: +```commandline +docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash +``` + +The container is very slim so if need be you can add additional tools using `apt`. + +If you wish to manually execute the accumulo-proxy in you can: +```commandline +/opt/accumulo-proxy/bin/accumulo-proxy -p /opt/accumulo-proxy/conf/proxy.properties ``` \ No newline at end of file diff --git a/Dockerfile b/Dockerfile index ff39779c..770e938c 100644 --- a/Dockerfile +++ b/Dockerfile @@ -59,29 +59,20 @@ RUN set -eux; \ download_bin "accumulo.tar.gz" "${ACCUMULO_HASH}" "accumulo/${ACCUMULO_VERSION}/accumulo-${ACCUMULO_VERSION}-bin.tar.gz"; # Install the dependencies into /opt/ -RUN mkdir /opt/hadoop && tar xzf /tmp/hadoop.tar.gz -C /opt/hadoop --strip 1 -RUN mkdir /opt/apache-zookeeper && tar xzf /tmp/apache-zookeeper.tar.gz -C /opt/apache-zookeeper --strip 1 -RUN mkdir /opt/accumulo && tar xzf /tmp/accumulo.tar.gz -C /opt/accumulo --strip 1 +RUN tar xzf /tmp/hadoop.tar.gz -C /opt/ && ln -s /opt/hadoop-${HADOOP_VERSION} /opt/hadoop +RUN tar xzf /tmp/apache-zookeeper.tar.gz -C /opt/ && ln -s /opt/apache-zookeeper-${ZOOKEEPER_VERSION}-bin /opt/apache-zookeeper +RUN tar xzf /tmp/accumulo.tar.gz -C /opt/ && ln -s /opt/accumulo-${ACCUMULO_VERSION} /opt/accumulo && sed -i 's/\${ZOOKEEPER_HOME}\/\*/\${ZOOKEEPER_HOME}\/\*\:\${ZOOKEEPER_HOME}\/lib\/\*/g' /opt/accumulo/conf/accumulo-env.sh ENV HADOOP_HOME /opt/hadoop ENV ZOOKEEPER_HOME /opt/apache-zookeeper ENV ACCUMULO_HOME /opt/accumulo -# Add some useful readme files -COPY README.md DOCKER.md /tmp/ - # Add the proxy binary COPY target/accumulo-proxy-${ACCUMULO_PROXY_VERSION}-bin.tar.gz /tmp/ RUN tar xzf /tmp/accumulo-proxy-${ACCUMULO_PROXY_VERSION}-bin.tar.gz -C /opt/accumulo-proxy --strip 1 ENV ACCUMULO_PROXY_HOME /opt/accumulo-proxy -# Sort out PATH and CLASSPATH configuration +# Ensure Accumulo is on the path. ENV PATH "${PATH}:${ACCUMULO_HOME}/bin" -ENV CLASSPATH=/opt/apache-zookeeper/lib/* - -# TEMPORARY FOR SPEED DURING DEVELOPMENT -#RUN apt-get update && apt-get install vim telnet -y && rm -rf /var/lib/apt/lists/* -#RUN sed -i 's/localhost:2181/host.docker.internal:2181/g' /opt/accumulo-proxy/conf/proxy.properties -#RUN sed -i 's/myinstance/uno/' /opt/accumulo-proxy/conf/proxy.properties CMD ["/opt/accumulo-proxy/bin/accumulo-proxy", "-p", "/opt/accumulo-proxy/conf/proxy.properties"] From 495c341045c4e54fef0db8e90e0f39d4ca7382fe Mon Sep 17 00:00:00 2001 From: Nathaniel Freeman Date: Wed, 29 Apr 2020 11:42:50 -0400 Subject: [PATCH 3/5] Updated the DOCKER.md to include links to the main accumulo project and contact us pages. --- DOCKER.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/DOCKER.md b/DOCKER.md index 57dde552..06077894 100644 --- a/DOCKER.md +++ b/DOCKER.md @@ -107,4 +107,8 @@ The container is very slim so if need be you can add additional tools using `apt If you wish to manually execute the accumulo-proxy in you can: ```commandline /opt/accumulo-proxy/bin/accumulo-proxy -p /opt/accumulo-proxy/conf/proxy.properties -``` \ No newline at end of file +``` + +Some resources for additional help: +* [Main Accumulo Website](https://accumulo.apache.org/) +* [Contact Us page](https://accumulo.apache.org/contact-us/) \ No newline at end of file From 040dfbdfb179bc81a55f44dac2612fba3a0f8560 Mon Sep 17 00:00:00 2001 From: Nathaniel Freeman Date: Wed, 29 Apr 2020 13:50:41 -0400 Subject: [PATCH 4/5] Further docs updates based on the suggestions of Keith, a tidy up of the description and a renaming of ZooKeeper to it's offical uppercase Z uppercase K format. --- DOCKER.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/DOCKER.md b/DOCKER.md index 06077894..dd38009a 100644 --- a/DOCKER.md +++ b/DOCKER.md @@ -16,8 +16,16 @@ limitations under the License. --> # accumulo-proxy-docker +This documentation covers how to stand up [accumulo-proxy](https://github.com/apache/accumulo-proxy/) within a Docker container. + +The guide covers: +* Building the image +* Configuring the `proxy.properties` file +* Selecting an appropriate networking choice +* Starting and stopping the container +* Basic troubleshooting tips -A temporary guide on how to run this up in Docker. +It is not recommended using this guide for a production instance of accumulo-proxy at this time. ## Build the image using Invoke the docker build command to create a container image. @@ -28,7 +36,7 @@ docker build -t accumulo-proxy:latest . ## Default Configuration and Quickstart By default, the container image expects the following to be true: 1. Your Accumulo instance name is "myinstance" -2. Your zookeeper is available (and reachable from the container) at localhost:2181 +2. Your ZooKeeper is available (and reachable from the container) at localhost:2181 You can start the proxy using: ```commandline @@ -44,11 +52,11 @@ docker run --rm -d -p 42424:42424 -v /path/to/proxy.properties:/opt/accumulo-pro ## Networking configuration Container networking can be a very specialised subject therefore we document two common practices that should cover the majority of use cases for development. -The proxy container must be able to access both Accumulo and Zookeeper. +The proxy container must be able to access both Accumulo and ZooKeeper. -The Zookeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section) +The ZooKeeper location can be configured in the `conf/proxy.properties` file, so you can override this to an acceptable value (see "Custom proxy.properties" section) -In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in Zookeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document. +In order to communicate with Accumulo the container will need to be able to resolve the FQDN that the servers have registered in ZooKeeper. If using [fluo-uno](https://github.com/apache/fluo-uno) this is very likely the hostname of your development environment. We'll call this my.host.com and IP 192.168.0.1 for the rest of this document. ### Host networking @@ -104,7 +112,7 @@ docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumu The container is very slim so if need be you can add additional tools using `apt`. -If you wish to manually execute the accumulo-proxy in you can: +If you wish to manually execute the accumulo-proxy in the container you can: ```commandline /opt/accumulo-proxy/bin/accumulo-proxy -p /opt/accumulo-proxy/conf/proxy.properties ``` From d1c2aa607050baeb96afbec9216e72de3b62d8f7 Mon Sep 17 00:00:00 2001 From: Nathaniel Freeman Date: Wed, 29 Apr 2020 15:22:43 -0400 Subject: [PATCH 5/5] Further pull request comments resolved. * Updated all references to "docker" to "Docker" where possible, similiar to ZooKeeper. * Updated the process to verify network connectivity. * Made a reference to the DOCKER.md from the main README.md so that people know it exists. --- DOCKER.md | 38 +++++++++++++++++++++++++------------- README.md | 4 ++++ 2 files changed, 29 insertions(+), 13 deletions(-) diff --git a/DOCKER.md b/DOCKER.md index dd38009a..544a4dd0 100644 --- a/DOCKER.md +++ b/DOCKER.md @@ -27,8 +27,13 @@ The guide covers: It is not recommended using this guide for a production instance of accumulo-proxy at this time. -## Build the image using -Invoke the docker build command to create a container image. +## Build the image +Firstly you will need the tarball of accumulo-proxy, this is documented in the [README.md](README.md) but for simplicity run: +```commandline +mvn clean package -Ptarball +``` + +Once you have the tarball (should be in ./target/ folder) then invoke the Docker build command to create a container image. ```commandline docker build -t accumulo-proxy:latest . ``` @@ -60,19 +65,27 @@ In order to communicate with Accumulo the container will need to be able to reso ### Host networking -Host networking is the simplest mechanism but generally only works for linux hosts where docker has been installed on 'bare metal' e.g. through an RPM. +Host networking is the simplest mechanism but generally only works for linux hosts where Docker has been installed on 'bare metal' e.g. through an RPM. -You can test if this will work for you by executing the following: +You can test if this will work for you by executing the following steps + +Start the accumulo-proxy container and enter it ```commandline -# Start the accumulo-proxy container and enter it docker run -it --rm -p 42424:42424 --network="host" --name accumulo-proxy accumulo-proxy:latest bash; +``` -# Install telnet and verify if you can connect to my.host.com:9995 -apt-get update && apt-get install telnet; -telnet my.host.com 9995 +Once inside the container, execute the curl command to attempt to connect to the monitor webserver: +```commandline +curl my.host.com:9995 ``` -If telnet can connect, then your container can resolve `my.host.com` correctly with host networking and therefore you can append `--network="host"` to your docker commands. +If the terminal returns an error such as: +``` +curl: (7) Failed to connect to my.host.com 9995: Connection refused +``` +then your container cannot see the host, and you will need to look at the next section (Non-Host networking). + +If you receive the HTML for the monitor web page then host networking will work for you and you can add `--network="host"` to each Docker command going forward. An example of using host networking: ```commandline @@ -81,15 +94,14 @@ docker run --rm -d -p 42424:42424 --network="host" --name accumulo-proxy accumul Note: You do not need to map your ports (-p) if using host networking, but we include it for clarity. -For more details see the official docker documentation: [Use host Networking](https://docs.docker.com/network/host) +For more details see the official Docker documentation: [Use host Networking](https://docs.docker.com/network/host) ### Non-Host networking - -If you run outside of a single node linux installation, e.g. Docker for Mac, Docker for Windows or use a VM to isolate your docker engine then you will likely need to take this path. +If you run outside of a single node linux installation, e.g. Docker for Mac, Docker for Windows or use a VM to isolate your Docker engine then you will likely need to take this path. Docker allows you to supply additional addresses to be resolved by the container, and these are automatically added by Docker to the /etc/hosts -For each host add a `--add-host FQDN:IP` entry to your docker run command, you can add multiple entries if need to, see the official docs covering [network settings](https://docs.docker.com/engine/reference/run/#network-settings). +For each host add a `--add-host FQDN:IP` entry to your Docker run command, you can add multiple entries if need to, see the official docs covering [network settings](https://docs.docker.com/engine/reference/run/#network-settings). An example of using this approach: diff --git a/README.md b/README.md index a2a91202..8f56222e 100644 --- a/README.md +++ b/README.md @@ -41,6 +41,10 @@ Thrift language binding). ./bin/accumulo-proxy -p conf/proxy.properties ``` +# Docker Environment + +The Accumulo Proxy can also now be packaged and started in a Docker container, see the [DOCKER.md](DOCKER.md) for full details. + # Build language specific bindings Bindings have been built in `src/main/` for Java, Python, and Ruby.