From 4ea9449e8a846a9e2a8037303cbbad3c3a96af9c Mon Sep 17 00:00:00 2001 From: Nick Dimiduk Date: Mon, 17 May 2021 10:57:51 -0700 Subject: [PATCH] HBASE-25896 Implement a Region Visualization on Master WebUI This is a demonstration of visualization of regions on the cluster. The visualization is a stacked bar chart showing total storefile size per table per region server, with the x-axis being server names, the y-axis being storfile size, and the bars stacked per table. The visualization is generated entirely on the fly from within the browser, implemented using Vega Lite. So far, Vega appears to handle rendering this visualization for a cluster of over 700 region servers with approximately 300,000 regions. Per [0], include an update to the top-level LICENSE.txt. Also update LICENSE files in all binary distributions (i.e., jars), by way of LICENSE.vm. Vega uses a BSD 3-clause variant without advertising clause, and as such is a "Category A" license, per [1]. No changes are made to the NOTICE files, as per the existing example of bundling the minified JQuery, which is also a Category A license. [0]: https://infra.apache.org/licensing-howto.html [1]: https://www.apache.org/legal/resolved.html#category-a Signed-off-by: Andrew Purtell --- LICENSE.txt | 34 +++ hbase-assembly/pom.xml | 1 + hbase-http/pom.xml | 1 + .../src/main/resources/META-INF/LICENSE.vm | 40 ++- hbase-server/pom.xml | 2 + .../hbase/tmpl/master/MasterStatusTmpl.jamon | 4 + .../tmpl/master/RegionVisualizerTmpl.jamon | 119 ++++++++ .../hbase/master/http/RegionVisualizer.java | 256 ++++++++++++++++++ .../resource/ClusterMetricsResource.java | 2 +- .../static/js/vega-embed@6.15.1.min.js | 27 ++ .../static/js/vega-lite@5.0.0.min.js | 2 + .../static/js/vega@5.19.1.min.js | 2 + .../master/http/TestRegionVisualizer.java | 98 +++++++ hbase-shaded/pom.xml | 1 + pom.xml | 4 + 15 files changed, 591 insertions(+), 2 deletions(-) create mode 100644 hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/RegionVisualizerTmpl.jamon create mode 100644 hbase-server/src/main/java/org/apache/hadoop/hbase/master/http/RegionVisualizer.java create mode 100644 hbase-server/src/main/resources/hbase-webapps/static/js/vega-embed@6.15.1.min.js create mode 100644 hbase-server/src/main/resources/hbase-webapps/static/js/vega-lite@5.0.0.min.js create mode 100644 hbase-server/src/main/resources/hbase-webapps/static/js/vega@5.19.1.min.js create mode 100644 hbase-server/src/test/java/org/apache/hadoop/hbase/master/http/TestRegionVisualizer.java diff --git a/LICENSE.txt b/LICENSE.txt index c30fe2b3d32a..d81488489b97 100755 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -639,3 +639,37 @@ under a '2-clause BSD' license. LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +---- +This project bundles a copy of the Vega minified javascript library version +5.19.1, the Vega-Lite minified javascript library version 5.0.0, and the +Vega-Embed minified javascript library version 6.15.1. All three are +available under the following '3-clause BSD' license. + +Copyright (c) 2015-2018, University of Washington Interactive Data Lab +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/hbase-assembly/pom.xml b/hbase-assembly/pom.xml index 795f832d97c6..c808b71d5eb2 100644 --- a/hbase-assembly/pom.xml +++ b/hbase-assembly/pom.xml @@ -53,6 +53,7 @@ ${license.debug.print.included} ${license.bundles.dependencies} ${license.bundles.jquery} + ${license.bundles.vega} ${license.bundles.logo} ${license.bundles.bootstrap} diff --git a/hbase-http/pom.xml b/hbase-http/pom.xml index f42e3244f528..e36989ba83ef 100644 --- a/hbase-http/pom.xml +++ b/hbase-http/pom.xml @@ -65,6 +65,7 @@ ${license.debug.print.included} ${license.bundles.dependencies} ${license.bundles.jquery} + ${license.bundles.vega} ${license.bundles.logo} ${license.bundles.bootstrap} diff --git a/hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm b/hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm index 760208f4b863..afbf0b8842c3 100644 --- a/hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm +++ b/hbase-resource-bundle/src/main/resources/META-INF/LICENSE.vm @@ -222,9 +222,10 @@ under the License. ## Special cases, for e.g. ASL2.0 licensed works that bundle additional third party works #set($bundled-jquery = ${bundled-jquery.equalsIgnoreCase("true")}) +#set($bundled-vega = ${bundled-vega.equalsIgnoreCase("true")}) #set($bundled-logo = ${bundled-logo.equalsIgnoreCase("true")}) #set($bundled-dependencies = ${bundled-dependencies.equalsIgnoreCase("true")}) -#if($bundled-jquery || $bundled-logo || $bundled-dependencies) +#if($bundled-jquery || $bundled-vega || $bundled-logo || $bundled-dependencies) ==== ${project.name} contained works @@ -260,6 +261,40 @@ This project bundles a copy of the JQuery minified javascript library version OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +#end +#macro(vega_license) +This project bundles a copy of the Vega minified javascript library version +5.19.1, the Vega-Lite minified javascript library version 5.0.0, and the +Vega-Embed minified javascript library version 6.15.1. All three are +available under the following '3-clause BSD' license. + +Copyright (c) 2015-2018, University of Washington Interactive Data Lab +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +1. Redistributions of source code must retain the above copyright notice, this +list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright notice, +this list of conditions and the following disclaimer in the documentation +and/or other materials provided with the distribution. + +3. Neither the name of the copyright holder nor the names of its contributors +may be used to endorse or promote products derived from this software +without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. #end #macro(orca_logo_license) ---- @@ -1309,6 +1344,9 @@ You can redistribute it and/or modify it under either the terms of the #if(${bundled-jquery}) #jquery_license() #end +#if(${bundled-vega}) +#vega_license() +#end #if(${bundled-logo}) #orca_logo_license() #end diff --git a/hbase-server/pom.xml b/hbase-server/pom.xml index 619ae146e425..e2f16df0809e 100644 --- a/hbase-server/pom.xml +++ b/hbase-server/pom.xml @@ -34,6 +34,7 @@ true true true + true **/src/main/resources/hbase-webapps/static/*/bootstrap* + + **/hbase-webapps/static/js/vega*.min.js **/*.vm @@ -1726,6 +1728,8 @@ false false + + false ${project.build.finalName}.tar.gz yyyy-MM-dd'T'HH:mm