Add an ability to install OFRAK from source

[ANogin]

• I have reviewed the OFRAK contributor guide and attest that this pull request is in accordance with it.

One sentence summary of this PR (This should go in the CHANGELOG!)
Add an ability to install OFRAK from source

Link to Related Issue(s)
N/A

Please describe the changes in your request.
This is a source tree equivalent of pip install for OFRAK - works similar to build_image.py approach, but does not require docker.

• Run make install_core/make install_tutorial/make install_develop to pip-install OFRAK from the current source tree. The core and tutorial versions do regular install (make install for packages listed in the corresponding .yml), while the install_develop does an "editable mode" install (make develop for packages).
• Pass OFRAK_INSTALL_DEPS=brew/OFRAK_INSTALL_DEPS=apt to make to have the dependencies (including the npm/rollout prerequisites) automatically installed.
• Pass OFRAK_INSTALL_PYTHON=python3.x (with or without the full path) to make to have OFRAK installed for the particular instance of python on your system.

(This also adds handling of binja as a potentially-missing dependency, as make install_develop easily triggers a situation where the ofrak binja modules are there, but binja is not - let me know if you'd rather have that as a separate PR).

Anyone you think should look at this, specifically?
@Edward-Larson probably? Maybe @rbs-jacob?

[rbs-jacob]

I've made several suggestions for minor changes.

I wonder if we should also incorporate this such that it gets run inside of the Docker images during build? That would improve the consistency between Docker and non-Docker OFRAK setups. I could see it also making the code more maintainable. @whyitfor and @EdwardLarson will probably want to weigh in on that.

[rbs-jacob]

Haven't yet had a chance to test, but I wonder if making Binary Ninja an "external tool" creates the possibility of a confusing situation for a user: they explicitly set the Binary Ninja back end by either discovering it or setting a command-line argument, they set exclude_components_missing_dependencies=True (also settable by a command-line argument), and there is an issue with their Binary Ninja installation. In this case, when they go to analyze their file, it won't unpack code regions with Binary Ninja, but also won't give the user any feedback as to why.

This may not be an issue, whether because this situation is sufficiently unlikely, because we don't deem this to be a problem, or because this issue doesn't actually manifest this way, but it's worth considering this situation.

[rbs-jacob]

One other concern I have about making Binary Ninja an external tool like this is that it's not consistent with the other analysis back ends. We only show information about when Binary Ninja is missing, but don't show any of the respective information about which components won't be loaded when we don't have Ghidra installed, , for example.

[ANogin]

Note that this is here because the binja backend is the one that actually prevents OFRAK from working at all if binja is not installed. Other backends do not have the issue.

[rbs-jacob]

I don't think npm is included in the results of ofrak deps, so the error message of this function is incorrect when npm is not found.

[ANogin]

@jacob, the npm is checked before ofrak is installed. The error message (when dependency_mechanism is neither apt nor brew) is:

                print(f"** {executable} not found, please install manually,")
                print('** or use "--install_deps" / "OFRAK_INSTALL_DEPS"')
                print("** to have it be installed automatically for you:")
                print(f"**    apt:  sudo apt install -y {executable}")
                print(f"**    brew: brew install {executable}")

Do not see anything wrong with it - when --install_deps is apt or brew, then npm will be installed - --install_deps is not only for ofrak deps dependencies, but also for build dependencies as well.

[whyitfor]

@ANogin, if I understand the intent behind this PR, you are looking for a way to install OFRAK from source natively.

Did you consider building a Makefile target to do this? What I'm envisioning is something like:

OFRAK_CONFIG=ofrak-dev.yml PYTHON_PATH=python3 make develop

The Makefile target could then:

1. Parse the yaml to get list of packages to install
2. Call make develop/install in the respective directory
3. There could be an optional argument at the end to run apt/brew commands

I also wonder if we need an automated script to install from source -- this is going to increase the amount of things taht need to be tested and maintained.

[ANogin]

Did you consider building a Makefile target to do this? What I'm envisioning is something like:

OFRAK_CONFIG=ofrak-dev.yml PYTHON_PATH=python3 make develop

@whyitfor , I can add OFRAK_CONFIG, but then would probably want to do the same for the build_image targets? Right now we have a few hardcoded targets with specific configs, rather than OFRAK_CONFIG, so I kept the same for consistency.

1. Parse the yaml to get list of packages to install
2. Call make develop/install in the respective directory

At least the above steps would presumably still require a python script?

3. There could be an optional argument at the end to run apt/brew commands

Is there any benefit to handling the apt vs brew vs just print the suggestion logic in Makefile rather than in Python?

I also wonder if we need an automated script to install from source -- this is going to increase the amount of things taht need to be tested and maintained.

[whyitfor]

@ANogin, with 3.3.0, there are clearer instructions on installing OFRAK python from source, https://ofrak.com/docs/install/source.html, as well as some Makefile commands to help with this process.

Given the age of this issue, and the number of unrelated changes in this diff, I think it is best to close this. If there are some quality of life things outstanding that you think would help with working with OFRAK from source, can you please make an issue so we can track and align on implementation. Thanks!