[fix][python-function] Fix the failed CPP test CI caused by python function #15827

zymap · 2022-05-30T01:10:07Z

Motivation

The CPP ci test always failed by the python function tests

ERROR: test_python_instance (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_python_instance
Traceback (most recent call last):
  File "/usr/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/pulsar/pulsar-functions/instance/target/python-instance/tests/test_python_instance.py", line 28, in <module>
    from python_instance import InstanceConfig
  File "/pulsar/pulsar-functions/instance/target/python-instance/python_instance.py", line 37, in <module>
    import Function_pb2
  File "/pulsar/pulsar-functions/instance/target/python-instance/Function_pb2.py", line 51, in <module>
    _descriptor.EnumValueDescriptor(
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py", line 755, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

Modification

Regenerate the proto file for the python instance

…nction --- **Motivation** The CPP ci test always failed by the python function tests ``` ERROR: test_python_instance (unittest.loader._FailedTest) ---------------------------------------------------------------------- ImportError: Failed to import test module: test_python_instance Traceback (most recent call last): File "/usr/lib/python3.8/unittest/loader.py", line 436, in _find_test_path module = self._get_module_from_name(name) File "/usr/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name __import__(name) File "/pulsar/pulsar-functions/instance/target/python-instance/tests/test_python_instance.py", line 28, in <module> from python_instance import InstanceConfig File "/pulsar/pulsar-functions/instance/target/python-instance/python_instance.py", line 37, in <module> import Function_pb2 File "/pulsar/pulsar-functions/instance/target/python-instance/Function_pb2.py", line 51, in <module> _descriptor.EnumValueDescriptor( File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py", line 755, in __new__ _message.Message._CheckCalledFromGeneratedFile() TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates ``` **Modification** Regenerate the proto file for python instance

zymap · 2022-05-30T01:11:45Z

Looks like that because we upgraded the python version in this PR

zymap · 2022-05-30T01:13:51Z

I found the error happens in many PRs.
https://github.com/apache/pulsar/runs/6640572711?check_suite_focus=true#step:11:1760
https://github.com/apache/pulsar/runs/6642677412?check_suite_focus=true#step:11:1809
https://github.com/apache/pulsar/runs/6643634092?check_suite_focus=true#step:11:1766

zymap · 2022-05-30T03:04:59Z

Something wrong with bookkeeper client:

======================================================================
ERROR: test_python_instance (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_python_instance
Traceback (most recent call last):
  File "/usr/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/pulsar/pulsar-functions/instance/target/python-instance/tests/test_python_instance.py", line 28, in <module>
    from python_instance import InstanceConfig
  File "/pulsar/pulsar-functions/instance/target/python-instance/python_instance.py", line 43, in <module>
    import state_context
  File "/pulsar/pulsar-functions/instance/target/python-instance/state_context.py", line 25, in <module>
    from bookkeeper import admin, kv
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/admin/__init__.py", line 15, in <module>
    from bookkeeper.admin.client import Client
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/admin/client.py", line 20, in <module>
    from bookkeeper import types
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/types.py", line 22, in <module>
    from bookkeeper.proto import common_pb2
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/proto/common_pb2.py", line 35, in <module>
    _descriptor.FieldDescriptor(
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

lhotari · 2022-05-30T06:27:04Z

Protobuf was upgraded in #15344 . Is protobuf 3.20.0 compatible with the bookkeeper python client?

zymap · 2022-05-30T07:31:17Z

If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.

@lhotari But we need to upgrade the bookkeeper Protobuf to the 3.19.0+ then regenerate the code?

lhotari · 2022-05-30T18:21:55Z

If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.

@lhotari But we need to upgrade the bookkeeper Protobuf to the 3.19.0+ then regenerate the code?

@zymap I guess so, but that would first require the change in Bookkeeper. I wonder if it would be able to downgrade Protobuf on Pulsar side so that we wouldn't have to depend on a Bookkeeper upgrade.

zymap · 2022-05-31T00:34:52Z

PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

@lhotari I used this workaround, and the python function tests passed 23cd61b

And downgrade Protobuf doesn't work, and I have tested it here zymap#37.

zymap · 2022-05-31T01:10:58Z

CPP flaky test:

FAILED TESTS (3/295):
     637 ms: ./main AuthPluginTest.testAthenz (try #1)
     514 ms: ./main AuthPluginTest.testAuthFactoryAthenz (try #1)
     331 ms: ./main AuthPluginTest.testAuthFactoryAthenz (try #2)

zymap · 2022-05-31T03:33:41Z

The function tests failed by the same reason:

2022-05-31T03:31:15,772+0000 [function-timer-thread-82-1] INFO  org.apache.pulsar.functions.runtime.process.ProcessRuntime - Started process successfully
Traceback (most recent call last):
  File "/pulsar/instances/python-instance/python_instance_main.py", line 41, in <module>
    import python_instance
  File "/pulsar/instances/python-instance/python_instance.py", line 43, in <module>
    import state_context
  File "/pulsar/instances/python-instance/state_context.py", line 25, in <module>
    from bookkeeper import admin, kv
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/admin/__init__.py", line 15, in <module>
    from bookkeeper.admin.client import Client
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/admin/client.py", line 20, in <module>
    from bookkeeper import types
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/types.py", line 22, in <module>
    from bookkeeper.proto import common_pb2
  File "/usr/local/lib/python3.8/dist-packages/bookkeeper/proto/common_pb2.py", line 35, in <module>
    _descriptor.FieldDescriptor(
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py", line 560, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.

(cherry picked from commit 437388a)

lhotari · 2022-05-31T03:55:23Z

...integration/src/test/java/org/apache/pulsar/tests/integration/functions/PulsarStateTest.java

+    // skip this tests and fix it later
+//    @Test(groups = {"python_state", "state", "function", "python_function"})


Is this also required with PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python ?

Yes. The function tests using python runtime will meet the same exception. I added the variable in the docker, and I will revert this change.

lhotari · 2022-05-31T03:59:40Z

And downgrade Protobuf doesn't work, and I have tested it here zymap#37.

@zymap I wonder if downgrading would have required installing a fixed version of the protobuf package with pip?
@cbornet used pip3 install protobuf==3.20.1 --user in #15846

lhotari · 2022-05-31T05:11:31Z

@zymap FYI , #15846 (comment) . The PR will fix the dependencies in the Pulsar's Python client so that Protobuf package version is limited to 3.20.* .

github-actions · 2022-07-01T02:15:00Z

The pr had no activity for 30 days, mark with Stale label.

zymap added area/test area/function doc-not-needed Your PR changes do not impact docs area/ci labels May 30, 2022

zymap self-assigned this May 30, 2022

zymap changed the title ~~[fix][python-function] Fix the failed CPP test CI caused by python fu…~~ [fix][python-function] Fix the failed CPP test CI caused by python function May 30, 2022

zymap requested review from codelipenghui, eolivelli, freeznet, hangc0276, jerrypeng, lhotari, merlimat and wolfstudy May 30, 2022 01:14

Fix the license issue

9c3d596

hangc0276 approved these changes May 30, 2022

View reviewed changes

freeznet approved these changes May 30, 2022

View reviewed changes

Using as a workarounds

23cd61b

Skip the testPythonWordCountFunction to allow the ci passed

e97b877

Fix the connector tests

0d45bb8

(cherry picked from commit 437388a)

lhotari mentioned this pull request May 31, 2022

[fix][python]Fix generated Python protobuf code not compatible with latest protobuf package #15846

Merged

5 tasks

lhotari reviewed May 31, 2022

View reviewed changes

freeznet mentioned this pull request Jun 2, 2022

python runner failed to start the function streamnative/function-mesh#379

Closed

github-actions bot added the Stale label Jul 1, 2022

zymap closed this Jul 1, 2022

		// skip this tests and fix it later
		// @Test(groups = {"python_state", "state", "function", "python_function"})

[fix][python-function] Fix the failed CPP test CI caused by python function #15827

[fix][python-function] Fix the failed CPP test CI caused by python function #15827

Uh oh!

Conversation

zymap commented May 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zymap commented May 30, 2022

Uh oh!

zymap commented May 30, 2022

Uh oh!

zymap commented May 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lhotari commented May 30, 2022

Uh oh!

zymap commented May 30, 2022

Uh oh!

lhotari commented May 30, 2022

Uh oh!

zymap commented May 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zymap commented May 31, 2022

Uh oh!

zymap commented May 31, 2022

Uh oh!

lhotari May 31, 2022

Choose a reason for hiding this comment

Uh oh!

zymap May 31, 2022

Choose a reason for hiding this comment

Uh oh!

lhotari commented May 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lhotari commented May 31, 2022

Uh oh!

github-actions bot commented Jul 1, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zymap commented May 30, 2022 •

edited

Loading

zymap commented May 30, 2022 •

edited

Loading

zymap commented May 31, 2022 •

edited

Loading

lhotari commented May 31, 2022 •

edited

Loading