WIP: GeoSolutions proposed docker implementation by lpasquali · Pull Request #2 · geoserver/docker

lpasquali · 2021-01-15T17:37:15Z

This implementation proposed is still to be perfected but it satisfies these initial requests here:

It supports automated builds from the docker hub if cofigured as per README.md as well custom builds with the provided custom_build.sh script

…ontinuos realease

buehner · 2021-01-29T14:23:52Z

Hi, thank you for proposing this alternative approach. I really like the docker multi-stage approach and also the possibility to build with custom/local WAR files, which is a nice feature that is still missing in #1

I was playing around with your proposal and as you say: This really needs some refinement/fixing. It took me quite a while to get this running in a very truncated version as i got several problems during that process.

As a quickstart i tried

docker build -t geoserver:test --build-arg GEOSERVER_WEBAPP_SRC="http://sourceforge.net/projects/geoserver/files/GeoServer/2.18.2/geoserver-2.18.2-war.zip" .

which is not working. I think the problem is that sourceforge is redirecting such requests and the docker ADD is not able to handle this. But in my eyes an "official" geoserver docker solution should also support the "official" geoserver WAR resource.

One more thought on the ADD approach: I see the advantage of being flexible regarding the source of the war file (use either local archive or http URL), but docker itself does not recommend the usage of ADD due to it's special nature that may lead to confusion and other problems: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#add-or-copy
So i think we should try to find a smarter solution on a script basis that equally satisfies this requirement.

Next, i played around with the custom build script approach. I finally got it running in a truncated version. Some lines made problems and i had to adjust or comment them out. I'll add comments on these lines after this post.

Nonetheless we should definitely provide the possibility to build with custom WARs. So my suggestion for now is that i will pick up the aspect that is still missing in #1 ("burn a custom WAR file to the image") and try to find a smart solution for that in my approach. (Of course this could also happen with other aspects that may be missing in my approach).

In general, I have the impression that we have two interest groups with regard to the Docker image: One group is primarily interested in the possibilities of the Dockerfile as they want to make use of it in future (without the need for an official pre-built docker image). The other group focuses more on the resulting Docker image, but doesn't want to build it themselves if it is provided by a hub. They just want to pull an existing docker image and configure it for their needs (this is probably the main focus of my approach).

It would be great if we could combine both approaches, so that everybody is happy in the end.

custom_build.sh

buehner · 2021-01-29T14:26:49Z

Dockerfile

+
+WORKDIR /output/plugins
+ARG PLUG_IN_URLS=""
+ADD .placeholder ${PLUG_IN_URLS} /output/plugins/


This was failing for me and i had to comment it out

was it failing with custom_build.sh?

Dockerfile

buehner · 2021-01-29T14:31:23Z

custom_build.sh

+		rm -f "${FILE_PATH}"
+	fi
+	if [ ! -f "${FILE_PATH}" ]; then
+		curl -L "${URL}" --output "${FILE_PATH}"


This curl download was not working in my case (for plugin downloads). But i could verify that the generated download URLs for the plugins were correct and working in the browser. So i think this could also be related to the way sourceforge is redirecting here...

I confirm I have similar behaviour where the final war.zip for geoserver is instead an HTMLpage @lpasquali

/output/webapp# head geoserver-2.18.2-war.zip <!doctype html>  <html class="no-js" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no" /> <script> var __gdpr = true; ...

@buehner how was the url this time?

custom_build.sh

buehner · 2021-01-29T14:33:57Z

custom_build.sh

+    clean_up_directory ${PLUGIN_ARTIFACT_DIRECTORY}
+    download_plugin ext monitor
+    download_plugin ext control-flow
+    download_plugin community sec-oauth2-geonode


i had to comment out all plugin downloads to get this running on my machine

they were there just as examples but they should work, I'd need output of:

bash -x ./custom_build.sh <options>

to understand well

custom_build.sh

buehner · 2021-01-29T17:28:00Z

Update: As stated above, i had problems using a sourceforge URL as source for the war file.

I now found out that https://downloads.sourceforge.net/project/geoserver/GeoServer/2.18.2/geoserver-2.18.2-war.zip seems to work with the ADD command (instead of http://sourceforge.net/projects/geoserver/files/GeoServer/2.18.2/geoserver-2.18.2-war.zip)

buehner · 2021-01-29T17:36:38Z

README.md

+
+There are [**build arguments**](https://docs.docker.com/engine/reference/commandline/build/) to customize the image:
+- `GEOSERVER_DATA_DIR_SRC` to add your own custom datadir to the final image. This can be a local zip or directory or remote URL (see [ADD](https://docs.docker.com/engine/reference/builder/#add) instruction Doc)
+- `GEOSERVER_WEBAPP_SRC` to add your own custom web app to the final image. This can be a local zip or directory or remote URL (see [ADD](https://docs.docker.com/engine/reference/builder/#add) instruction Doc)


seems that zip is not extracted automatically by ADD (but tar)

This is handled by https://github.com/geoserver/docker/pull/2/files#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R17

buehner · 2021-01-29T17:48:50Z

In 82e350d i enabled the usage of a custom WAR from local machine by using the ADD magic ;-)

randomorder · 2021-02-01T10:42:40Z

Hi, thank you for proposing this alternative approach. I really like the docker multi-stage approach and also the possibility to build with custom/local WAR files, which is a nice feature that is still missing in #1

I was playing around with your proposal and as you say: This really needs some refinement/fixing. It took me quite a while to get this running in a very truncated version as i got several problems during that process.

As a quickstart i tried

docker build -t geoserver:test --build-arg GEOSERVER_WEBAPP_SRC="http://sourceforge.net/projects/geoserver/files/GeoServer/2.18.2/geoserver-2.18.2-war.zip" .

which is not working. I think the problem is that sourceforge is redirecting such requests and the docker ADD is not able to handle this. But in my eyes an "official" geoserver docker solution should also support the "official" geoserver WAR resource.

One more thought on the ADD approach: I see the advantage of being flexible regarding the source of the war file (use either local archive or http URL), but docker itself does not recommend the usage of ADD due to it's special nature that may lead to confusion and other problems: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#add-or-copy
So i think we should try to find a smarter solution on a script basis that equally satisfies this requirement.

Next, i played around with the custom build script approach. I finally got it running in a truncated version. Some lines made problems and i had to adjust or comment them out. I'll add comments on these lines after this post.

Nonetheless we should definitely provide the possibility to build with custom WARs. So my suggestion for now is that i will pick up the aspect that is still missing in #1 ("burn a custom WAR file to the image") and try to find a smart solution for that in my approach. (Of course this could also happen with other aspects that may be missing in my approach).

In general, I have the impression that we have two interest groups with regard to the Docker image: One group is primarily interested in the possibilities of the Dockerfile as they want to make use of it in future (without the need for an official pre-built docker image). The other group focuses more on the resulting Docker image, but doesn't want to build it themselves if it is provided by a hub. They just want to pull an existing docker image and configure it for their needs (this is probably the main focus of my approach).

It would be great if we could combine both approaches, so that everybody is happy in the end.

Thanks a lot for your review @buehner

It would be great if we could combine both approaches, so that everybody is happy in the end.

I agree. It should be relatively easy to combine both approached and get the best out of the two

lpasquali · 2021-02-01T17:07:50Z

docker build -t geoserver:test --build-arg GEOSERVER_WEBAPP_SRC="http://sourceforge.net/projects/geoserver/files/GeoServer/2.18.2/geoserver-2.18.2-war.zip" .

Hello @buehner many thanks for your comments!

I will start from the first one: unfortunately that url not working for you it is a limitation of sourceforge, taking from the custom_build.sh script, you could use a direct mirror link linke this one "https://netcologne.dl.sourceforge.net/project/geoserver/GeoServer/2.18.1/geoserver-2.18.1-war.zip"

the Dockerfile build would had failed anyway because it missed a file in the context "run_tests.sh" which I just added.
this command should do then the trick:

docker build -t geoserver:test --build-arg GEOSERVER_WEBAPP_SRC="https://netcologne.dl.sourceforge.net/project/geoserver/GeoServer/2.18.1/geoserver-2.18.1-war.zip" .

- fix help function - create ./resources/geoserver-plugins if needed (it was already present but using wrong var) - removed marlin

added RC build in hooks and latest tagging

jodygarnett · 2021-03-10T17:55:34Z

Can we do a 2.19-RC for this? You can release a docker image as early access or something just go get the party started.

randomorder · 2021-03-10T18:22:29Z

Yes, no problem @jodygarnett
We'll make the image available before the end of the week

randomorder · 2021-03-12T15:25:40Z

Hi @jodygarnett @buehner

We pushed the image on GeoSolutions org repo on the docker hub:
docker run -d --rm -p 8888:8080 --name geoserver geosolutionsit/geoserver:2.19-RC_GSIP-192

The image was built according to the instr. in the readme. We used the default datadir that comes with the
docker build -t geosolutionsit/geoserver:2.19-RC_GSIP-192 --build-arg GEOSERVER_WEBAPP_SRC=./resources/geoserver/geoserver.war .

In the process I noticed in the README there are no instr. on how to externalize the datadir. @lpasquali is amending it right now.

lpasquali added 3 commits January 15, 2021 18:19

documentation, deployment diagram

60b25f7

Dockerfile

f4ae855

.placeholder hooks custom_build.sh for custom builds and docker hub c…

6982ec1

…ontinuos realease

lpasquali changed the title ~~WIP: GeoSolutions proposed docker imnplementation~~ WIP: GeoSolutions proposed docker implementation Jan 18, 2021