chore: fix broken make proto

[tmenjo]

Close #234

[tmenjo]

Note that, without change of the sed line, I got a No such file or directory error as follows:

$ make proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sinker -I=pynumaflow/proto/sinker --python_out=pynumaflow/proto/sinker --grpc_python_out=pynumaflow/proto/sinker  pynumaflow/proto/sinker/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/mapper -I=pynumaflow/proto/mapper --python_out=pynumaflow/proto/mapper --grpc_python_out=pynumaflow/proto/mapper  pynumaflow/proto/mapper/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/reducer -I=pynumaflow/proto/reducer --python_out=pynumaflow/proto/reducer --grpc_python_out=pynumaflow/proto/reducer  pynumaflow/proto/reducer/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sourcetransformer -I=pynumaflow/proto/sourcetransformer --python_out=pynumaflow/proto/sourcetransformer --grpc_python_out=pynumaflow/proto/sourcetransformer  pynumaflow/proto/sourcetransformer/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sideinput -I=pynumaflow/proto/sideinput --python_out=pynumaflow/proto/sideinput --grpc_python_out=pynumaflow/proto/sideinput  pynumaflow/proto/sideinput/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sourcer -I=pynumaflow/proto/sourcer --python_out=pynumaflow/proto/sourcer --grpc_python_out=pynumaflow/proto/sourcer  pynumaflow/proto/sourcer/*.proto
sed -i '' 's/^\(import.*_pb2\)/from . \1/' pynumaflow/proto/*/*.py
sed: can't read s/^\(import.*_pb2\)/from . \1/: No such file or directory
make: *** [Makefile:35: proto] Error 2

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.26%. Comparing base (9a390a5) to head (f257264).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #235   +/-   ##
=======================================
  Coverage   94.26%   94.26%           
=======================================
  Files          60       60           
  Lines        2441     2441           
  Branches      124      124           
=======================================
  Hits         2301     2301           
  Misses        101      101           
  Partials       39       39           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tmenjo]

Note that, without change of the sed line, I got a No such file or directory error as follows:

I just noticed that an empty string argument for sed -i option is probably for sed on Mac. I made and checked this PR on Ubuntu 24.04 and GNU sed.

Can anyone here try to run make proto with this PR on Mac?

[tmenjo]

I added a commit that sed in make proto can run both on Linux and Mac: give -i.bak option to sed to preserve current files with .bak extension then rm all of them.

[vigith]

I added a commit that sed in make proto can run both on Linux and Mac: give -i.bak option to sed to preserve current files with .bak extension then rm all of them.

I don't think we need to use the -i option. Any reason you want to use it?

[tmenjo]

I don't think we need to use the -i option. Any reason you want to use it?

Without -i option, the results of sed are output to stdout, and given files (pynumaflow/proto/*/*.py) remain unchanged.

[vigith]

sorry, my bad, why do we need to take the backup? I asked the wrong question

[tmenjo]

why do we need to take the backup?

We don't need backup actually, but I have no choice but to use sed -i.bak (then rm all .bak files) because I want our make proto work on both Linux and Mac.

It's a compatibility issue between GNU sed (Linux) and BSD sed (Mac). Googling with [sed i linux mac], we can find many pages about the issue such as [1].

If I just think about Linux (my dev env using GNU sed), my sed command line can be as follows with no backup. However, it possibly doesn't work properly on Mac (I haven't tried actually since I have no Mac):

sed -i -e 's/^\(import.*_pb2\)/from . \1/' pynumaflow/proto/*/*.py

In contrast to that, the following command line having an empty string for -i option may work on Mac, but doesn't on Linux (I got sed: can't read : No such file or directory):

sed -i '' -e 's/^\(import.*_pb2\)/from . \1/' pynumaflow/proto/*/*.py

[1] https://stackoverflow.com/questions/4247068/sed-command-with-i-option-failing-on-mac-but-works-on-linux

[kohlisid]

(venv) skohli@macos-JQWR9T560R numaflow-python % make proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sinker -I=pynumaflow/proto/sinker --python_out=pynumaflow/proto/sinker --grpc_python_out=pynumaflow/proto/sinker  pynumaflow/proto/sinker/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/mapper -I=pynumaflow/proto/mapper --python_out=pynumaflow/proto/mapper --grpc_python_out=pynumaflow/proto/mapper  pynumaflow/proto/mapper/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/reducer -I=pynumaflow/proto/reducer --python_out=pynumaflow/proto/reducer --grpc_python_out=pynumaflow/proto/reducer  pynumaflow/proto/reducer/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sourcetransformer -I=pynumaflow/proto/sourcetransformer --python_out=pynumaflow/proto/sourcetransformer --grpc_python_out=pynumaflow/proto/sourcetransformer  pynumaflow/proto/sourcetransformer/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sideinput -I=pynumaflow/proto/sideinput --python_out=pynumaflow/proto/sideinput --grpc_python_out=pynumaflow/proto/sideinput  pynumaflow/proto/sideinput/*.proto
poetry run python3 -m grpc_tools.protoc --pyi_out=pynumaflow/proto/sourcer -I=pynumaflow/proto/sourcer --python_out=pynumaflow/proto/sourcer --grpc_python_out=pynumaflow/proto/sourcer  pynumaflow/proto/sourcer/*.proto
sed -i.bak -e 's/^\(import.*_pb2\)/from . \1/' pynumaflow/proto/*/*.py
rm pynumaflow/proto/*/*.py.bak
(venv) skohli@macos-JQWR9T560R numaflow-python % git status

[kohlisid]

Thanks for the changes @tmenjo :D