Skip to content

Conversation

@BewareMyPower
Copy link
Contributor

Motivation

The CI - Pulsar Website Build has been broken for a long time. Here's an example run: https://github.com/apache/pulsar/runs/2635024657?check_suite_focus=true

CMake Error at python/CMakeLists.txt:85 (MESSAGE):
-- Using Boost Python libs: 
  Could not find Boost Python library

It looks like to be the same issue with #9682. #9690 fixed the broken CI for cpp client but the ci-pulsar-website-build.yaml wasn't modified.

Another issue is after I fixed the cpp build for website build, the python-doc-gen.sh still failed because the default pdoc is a Python3 tool

+ pdoc pulsar --html --html-dir /pulsar/generated-site/api/python/2.8.0-SNAPSHOT
Traceback (most recent call last):
  File "/usr/local/bin/pdoc", line 7, in <module>
    from pdoc.__main__ import cli
  File "/usr/local/lib/python3.5/dist-packages/pdoc/__init__.py", line 328
    ) -> str:
    ^

So we need to install a Python2 version pdoc as well.

Modifications

  • Use ubuntu-16.04-py2 tag instead of old ubuntu-16.04 tag in docker-build-site.sh and force CMake to find Python2 in python-doc-gen.sh. The the Pulsar Python client library could be built successfully.
  • Install a Python2 version pdoc in python-doc-gen.sh.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

Copy link
Member

@tuteng tuteng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am testing it locally.
thank you very much !


BUILD_IMAGE_NAME="${BUILD_IMAGE_NAME:-apachepulsar/pulsar-build}"
BUILD_IMAGE_VERSION="${BUILD_IMAGE_VERSION:-ubuntu-16.04}"
BUILD_IMAGE_VERSION="${BUILD_IMAGE_VERSION:-ubuntu-16.04-py2}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where and how do we build this image ?
apachepulsar/pulsar-build:ubuntu-16.04-py2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The image was uploaded manually by @merlimat before. The dockerfile is build/docker/Dockerfile.

@eolivelli
Copy link
Contributor

The change makes sense to me

but if I run the script locally I see this error:

tools/docker-build-site.sh 

.....
+ cp -r /pulsar/target/doxygen/html /pulsar/generated-site/api/cpp/
+ mv /pulsar/target/doxygen/html /pulsar/generated-site/api/cpp/2.8.0-SNAPSHOT
mv: cannot move '/pulsar/target/doxygen/html' to '/pulsar/generated-site/api/cpp/2.8.0-SNAPSHOT/html': File exists

@BewareMyPower
Copy link
Contributor Author

The change makes sense to me

but if I run the script locally I see this error:

tools/docker-build-site.sh 

.....
+ cp -r /pulsar/target/doxygen/html /pulsar/generated-site/api/cpp/
+ mv /pulsar/target/doxygen/html /pulsar/generated-site/api/cpp/2.8.0-SNAPSHOT
mv: cannot move '/pulsar/target/doxygen/html' to '/pulsar/generated-site/api/cpp/2.8.0-SNAPSHOT/html': File exists

I think you have built the website before? I think you need to remove the local directory first.

@BewareMyPower
Copy link
Contributor Author

Because the build script starts a docker container and maps the Pulsar project directory to /pulsar directory in container. So it will also create a generated-site directory.

I think it's a minor bug of the script that it uses mkdir to create directory but not mkdir -p. It's not a problem for CI but it could be annoyed when you test the script in local env.

@BewareMyPower
Copy link
Contributor Author

BewareMyPower commented May 21, 2021

I just built it in my local env. I'm not sure if it's a failure

warning highlight.js@9.18.5: Support has ended for 9.x series. Upgrade to @latest
[2/4] Fetching packages...
info fsevents@1.2.13: The platform "linux" is incompatible with this module.
info "fsevents@1.2.13" is an optional dependency and failed compatibility check. Excluding it from installation.
error @redocly/openapi-core@1.0.0-beta.46: The engine "node" is incompatible with this module. Expected version ">=12.0.0". Got "10.24.0"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Anyway all index.htmls have been generated:

$ ls generated-site/api/
admin            client           cpp              pulsar-broker    pulsar-functions python
$ find generated-site/api -name "index.html"
generated-site/api/pulsar-broker/2.8.0-SNAPSHOT/index.html
generated-site/api/python/index.html
generated-site/api/python/schema/index.html
generated-site/api/python/functions/index.html
generated-site/api/python/2.8.0-SNAPSHOT/index.html
generated-site/api/python/2.8.0-SNAPSHOT/schema/index.html
generated-site/api/python/2.8.0-SNAPSHOT/functions/index.html
generated-site/api/admin/2.8.0-SNAPSHOT/index.html
generated-site/api/cpp/html/index.html
generated-site/api/cpp/2.8.0-SNAPSHOT/index.html
generated-site/api/pulsar-functions/2.8.0-SNAPSHOT/index.html
generated-site/api/client/2.8.0-SNAPSHOT/index.html

@eolivelli
Copy link
Contributor

@BewareMyPower does the website renders correctly on your machine ?
I mean, if you try to see it with a browser

@BewareMyPower
Copy link
Contributor Author

BewareMyPower commented May 21, 2021

@eolivelli

image

image

image

image

image

image

As for 2.8.0-SNAPSHOT URLs, they're the same with those URLs without 2.8.0-SNAPSHOT.

PS. 文件 is Chinese version of file.

@codelipenghui codelipenghui merged commit c9d6b1e into apache:master May 21, 2021
@codelipenghui codelipenghui added this to the 2.8.0 milestone May 21, 2021
@eolivelli
Copy link
Contributor

great

@BewareMyPower BewareMyPower deleted the bewaremypower/fix-website-build-cpp branch May 21, 2021 16:25
yangl pushed a commit to yangl/pulsar that referenced this pull request Jun 23, 2021
### Motivation

The CI - Pulsar Website Build has been broken for a long time. Here's an example run: https://github.com/apache/pulsar/runs/2635024657?check_suite_focus=true

```
CMake Error at python/CMakeLists.txt:85 (MESSAGE):
-- Using Boost Python libs: 
  Could not find Boost Python library
```

It looks like to be the same issue with apache#9682. apache#9690 fixed the broken CI for cpp client but the `ci-pulsar-website-build.yaml` wasn't modified.

Another issue is after I fixed the cpp build for website build, the `python-doc-gen.sh` still failed because the default `pdoc` is a Python3 tool

```
+ pdoc pulsar --html --html-dir /pulsar/generated-site/api/python/2.8.0-SNAPSHOT
Traceback (most recent call last):
  File "/usr/local/bin/pdoc", line 7, in <module>
    from pdoc.__main__ import cli
  File "/usr/local/lib/python3.5/dist-packages/pdoc/__init__.py", line 328
    ) -> str:
    ^
```

So we need to install a Python2 version `pdoc` as well.

### Modifications

- Use `ubuntu-16.04-py2` tag instead of old `ubuntu-16.04` tag in `docker-build-site.sh` and force CMake to find Python2 in `python-doc-gen.sh`. The the Pulsar Python client library could be built successfully.
- Install a Python2 version `pdoc` in `python-doc-gen.sh`.
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
### Motivation

The CI - Pulsar Website Build has been broken for a long time. Here's an example run: https://github.com/apache/pulsar/runs/2635024657?check_suite_focus=true

```
CMake Error at python/CMakeLists.txt:85 (MESSAGE):
-- Using Boost Python libs: 
  Could not find Boost Python library
```

It looks like to be the same issue with apache#9682. apache#9690 fixed the broken CI for cpp client but the `ci-pulsar-website-build.yaml` wasn't modified.

Another issue is after I fixed the cpp build for website build, the `python-doc-gen.sh` still failed because the default `pdoc` is a Python3 tool

```
+ pdoc pulsar --html --html-dir /pulsar/generated-site/api/python/2.8.0-SNAPSHOT
Traceback (most recent call last):
  File "/usr/local/bin/pdoc", line 7, in <module>
    from pdoc.__main__ import cli
  File "/usr/local/lib/python3.5/dist-packages/pdoc/__init__.py", line 328
    ) -> str:
    ^
```

So we need to install a Python2 version `pdoc` as well.

### Modifications

- Use `ubuntu-16.04-py2` tag instead of old `ubuntu-16.04` tag in `docker-build-site.sh` and force CMake to find Python2 in `python-doc-gen.sh`. The the Pulsar Python client library could be built successfully.
- Install a Python2 version `pdoc` in `python-doc-gen.sh`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants