Implement CPack for Debian/RPM

[AndrewQuijano]

Your checklist for this pull request

• I've documented or updated the documentation of every API function and struct this PR changes.
• I've added tests that prove my fix is effective or that my feature works (if possible)

Detailed description
I created a CPackConfig.txt that can create both Debian and RPM packages. The Debian package matches what we have now. I can look further about making a proper RPM package.

Debian Screenshots


This is identical to #2579

RPM

Here is the generated spec file
libcapstone-dev.spec.txt

I do want to get help with macOS package creation and testing using CPack

I updated how to do this in the Building.md file

...

Test plan
We can probably use something like check_capstone.sh? Although for now, we can merge it as another way to create the Debian Package? My hope is, from here, work can be done to get a Mac OSX package built. I can help with RPM, the work is done, but any last necessary touches I can take care of it.

...

Closing issues

...

[AndrewQuijano]

@Rot127 While I will need help with the Mac stuff, I can try to poke around how capstone made RPM packages before, but your feedback would be helpful to have! I did poke around and find this, it seems the RPM packaging standard is a lot more of a mess than I thought as it seems anyone can package as they see fit.

https://rpmfind.net/linux/rpm2html/search.php?query=libcapstone-devel
https://rpmfind.net/linux/rpm2html/search.php?query=capstone
https://rpmfind.net/linux/rpm2html/search.php?query=capstone-devel

I do want to ask, if the goal is to automate placing Debian packages in APT, this would require creating both a libcapstone package with the shared objects only and another debian with the static library/headers. I could use cmake twice and leverage CPack to make two packages. To avoid the headers being made twice, would I need to delete these lines of code?

https://github.com/capstone-engine/capstone/blob/next/CMakeLists.txt#L799-L801

[Rot127]

Sorry for the late answer. I will review this one after new year. But to answer your questions meanwhile:

I do want to ask, if the goal is to #2537, this would require creating both a libcapstone package with the shared objects only and another debian with the static library/headers.

Yes, this would be nice to have. Though, I can't put any priority on it. And before v6 Beta, it is not necessary. I think for v6 Gold it is though.

try to poke around how capstone made RPM packages before, but your feedback would be helpful to have!

I have no idea. They were made before my time here. You have to contact the package maintainers and/or follow the docs.

[AndrewQuijano]

I poked around a bit more about making the RPM the best I can, so a few things I'll likely need help on,
I will ask based on this
https://github.com/capstone-engine/capstone/blob/next/packages/rpm/capstone.spec

1. It would be nice to have something to check if the package works, is this legit?

ln -s libcapstone.so libcapstone.so.5
make check LD_LIBRARY_PATH="`pwd`"

2. It would be nice to have an install too, would this look accurate to add?

DESTDIR=%{buildroot} LIBDIRARCH=%{_lib} \
INCDIR="%{_includedir}" make install
find %{buildroot} -name '*.la' -exec rm -f {} ';'
find %{buildroot} -name '*.a' -exec rm -f {} ';'

# install python bindings
pushd bindings/python
%{__python2} setup.py install --skip-build --root %{buildroot}
%if 0%{?with_python3}
%{__python3} setup.py install --skip-build --root %{buildroot}
%endif # with_python3
popd

# install java bindings
install -D -p -m 0644 bindings/java/%{name}.jar  %{buildroot}/%{_javadir}/%{name}.jar

3. I'm not sure if a build would be need for the package?

DESTDIR="%{buildroot}" 	V=1 CFLAGS="%{optflags}" \
LIBDIRARCH="%{_lib}" INCDIR="%{_includedir}" make %{?_smp_mflags}

# Fix pkgconfig file
sed -i 's;%{buildroot};;' capstone.pc
grep -v archive capstone.pc > capstone.pc.tmp
mv capstone.pc.tmp capstone.pc

# build python bindings
pushd bindings/python
CFLAGS="%{optflags}" %{__python2} setup.py build
%if 0%{?with_python3}
CFLAGS="%{optflags}" %{__python3} setup.py build
%endif # with_python3
popd

# build java bindings
pushd bindings/java
make CFLAGS="%{optflags}" # %{?_smp_mflags} parallel seems broken
popd

4. Please feel free to check my current generated spec file and poke around what else should be needed to make this as high quality as possible. I'm also realizing there is no contacts to who packages for Fedora and CentOS, etc.

5. No worries, we can wait until next year, but will also need your help on updating CPack to also generate the Mac packages too :)

[Rot127]

Nice! Thanks a lot for getting started with it.

Some general notes:

• Let's exclude OSX for now, please. It gets too much otherwise and they can be part of another PR.
• The packages/rpm/capstone.spec file shouldn't be necessary. I think it can be removed. CPack should generate it.
• Once we have tested the package build locally, we can add the process to the release workflow.

Regarding #2590 (comment)

This looks a little hacky. Let's not worry about the bindings for now. Because they need some refactoring before (see: #2291).
And maybe there are way better ways to create such packages. In the case of the Python bindings, they are already uploaded to pypi.

[AndrewQuijano]

@Rot127 Just updated the CPack file per your suggestions (Note both Debian and RPM package content looks identical now), three questions remain:

1. Is there a way to test the package after the installation to confirm it works?
2. Have you had a minute to check if the RPM package works as intended?
3. Once this is tested, should I delete all the docker stuff from the deb folder too? Should I also replicate this on v5 branch as I do still really hope to get a v5.0.4 package ready for faster PANDA installation relatively soon

https://github.com/AndrewQuijano/capstone/releases/tag/6.0.2-Alpha3

[Rot127]

Is there a way to test the package after the installation to confirm it works?

I just ran a cstool command the last time to check. Because cstool uses the dynamic library.
For headers we can just check, if we can include them as system header in a C source file.

Have you had a minute to check if the RPM package works as intended?

Not yet. I waited for you to add the workflow job, so I don't have to set up to much. Will do review this one again tomorrow.

Once this is tested, should I delete all the docker stuff from the deb folder too?

Yeah, sure. Less duplicates are always better.

Should I also replicate this on v5 branch as I do still really hope to get a v5.0.4 package ready for faster PANDA installation relatively soon

This would be awesome!

@kabeor Do you know if we could get the release before lunar new year?
If you have enough time, @AndrewQuijano can open the PR now and we can prioritize it. So it is merged quickly.

[AndrewQuijano]

@Rot127 Just updated the PR, and just curious would you look into how to update labeler to v5? Also, figure I should ask, you don't mind I use your ISSUE_TEMPLATE folder and labeler stuff for any other open-source work right?

[Rot127]

Looks good!

But the build fails for me: https://github.com/Rot127/capstone/actions/runs/12734529842/job/35492198289

...
 Enabling CAPSTONE_WASM_SUPPORT
Enabling CAPSTONE_BPF_SUPPORT
Enabling CAPSTONE_RISCV_SUPPORT
Enabling CAPSTONE_SH_SUPPORT
Enabling CAPSTONE_TRICORE_SUPPORT
Enabling CAPSTONE_ALPHA_SUPPORT
Enabling CAPSTONE_HPPA_SUPPORT
Enabling CAPSTONE_LOONGARCH_SUPPORT
Enabling CAPSTONE_XTENSA_SUPPORT
CMake Error at /usr/local/share/cmake-3.31/Modules/WriteBasicConfigVersionFile.cmake:43 (message):
  No VERSION specified for WRITE_BASIC_CONFIG_VERSION_FILE()
Call Stack (most recent call first):
  /usr/local/share/cmake-3.31/Modules/CMakePackageConfigHelpers.cmake:402 (write_basic_config_version_file)
  CMakeLists.txt:897 (write_basic_package_version_file)


-- Configuring incomplete, errors occurred!

[Rot127]

Just updated the PR, and just curious would you look into how to update labeler to v5?

It isn't a priority currently for me. Too much other stuff to do unfortunately. If you want to work on it feel free to do so! If you are faster then @kabeor reviewing, we can replace my downgrading PR with yours.

Also, figure I should ask, you don't mind I use your ISSUE_TEMPLATE folder and labeler stuff for any other open-source work right?

Not at all. It is open-source in the end :D

[Rot127]

Debian works fine.
But the rpm package fails to install:

nothing proivdes libc6 >= 2.2.5

The package name for libc is apparently glibc (tested on Fedora). Also see https://packages.fedoraproject.org/pkgs/capstone/capstone/fedora-rawhide.html.

[AndrewQuijano]

@Rot127 just updated CPack, hopefully this should fix the issue

[Rot127]

Very nice! Thanks a lot!

[AndrewQuijano]

@kabeor Please merge and create a 5.0.4 on the v5 Branch

[kabeor]

@kabeor Please merge and create a 5.0.4 on the v5 Branch

Done.

[AndrewQuijano]

@kabeor Thank you for the new release! I'm wondering, would it be possible to re-release 5.0.4? It seems the code to dynamically update the version got deleted here.

The released packages are missing the version in the file name (I aimed to follow standard RPM/Debian package practices), since the code to dynamically pick up the version from the tag got deleted.

https://github.com/capstone-engine/capstone/releases/tag/5.0.4

Also, could you please merge #2590? I should note, this PR also has the same technique to dynamically update the version of Capstone based on input from tag name. This should save time from having to manually bump the version number for every release, which seemed to be ok with @Rot127

I see based on the v5 release, I think a better solution is to use a version.h.in file, with some modifications from CMake, we can dynamically get this version data, automating this work on every release. Should this just be another PR?

#ifndef VERSION_H
#define VERSION_H

#define PROJECT_VERSION_MAJOR @PROJECT_VERSION_MAJOR@
#define PROJECT_VERSION_MINOR @PROJECT_VERSION_MINOR@
#define PROJECT_VERSION_PATCH @PROJECT_VERSION_PATCH@

#endif // VERSION_H

[AndrewQuijano]

Two things

1- I did a bit more digging around for pkgconfig.mk, it is used by Makefile and Python, I propose having CMake be able to dynamically overwrite pkgconfig.mk with the version data if it is provided. I'm not sure where I'd write the updated version of pkgconfig for use? I'd also be curious what the values in bindings/python/capstone/__init__.py are used for?

2- I can add a version.h file to be created and updated dynamically in include/capstone folder. Let me know what to do next @Rot127 @kabeor and if it should be included in this PR (and perhaps in v5 as well too?)

[Rot127]

It seems the code to dynamically update the version got deleted here.

Just pinged him. Was an accident while resolving conflicts. Thanks for pointing it out!
@kabeor will do a 5.0.5.

I see based on the v5 release, I think a better solution is to use a version.h.in file, with some modifications from CMake, we can dynamically get this version data, automating this work on every release. Should this just be another PR?

Good idea. At least on the next branch we should have it.
Would you mind opening an issue about it.

Regarding the whole pkgconfig changes:
Let's do them in the next release. Not now, in the v5.0.5 one. Otherwise you have to wait even longer for it :D And we might break it while rushing.
But we can discuss details in the issue. TO have them contained at one place.

[Rot127]

@AndrewQuijano Please review #2606

[Rot127]

Ah, also. Please rebase this one on the latest next. The labeler is fixed again.

[AndrewQuijano]

@Rot127 Rebased complete!

@kabeor Thanks for v5.0.5 release! Will be updating PANDA to be using the new Debian package shortly panda-re/panda#1562

[kabeor]

Thank you, merged.