Trying to add a binary file as observable from analyzer returns error

[dadokkio]

Trying to add a binary file as observable returns error:

Traceback (most recent call last): File "/opt/cortex/analyzers/VirusTotal/virustotal.py", line 273, in <module> VirusTotalAnalyzer().run() File "/opt/cortex/analyzers/VirusTotal/virustotal.py", line 262, in run self.report(results) File "/usr/local/lib/python3.7/dist-packages/cortexutils/analyzer.py", line 104, in report 'artifacts': self.artifacts(full_report), File "/opt/cortex/analyzers/VirusTotal/virustotal.py", line 114, in artifacts artifacts.append(self.build_artifact("file", self.obs_path)) File "/usr/local/lib/python3.7/dist-packages/cortexutils/analyzer.py", line 81, in build_artifact copyfileobj(src, os.fdopen(dst, 'w')) File "/usr/lib/python3.7/shutil.py", line 79, in copyfileobj buf = fsrc.read(length) File "/usr/lib/python3.7/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final)UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: invalid continuation byte

Probably due to python3 migration.
At the moment there are no analyzer that are adding file as observable so I could make a pull with binary support but I'm not sure if str support must be kept for any reason.

    def build_artifact(self, data_type, data, **kwargs):
        if data_type == 'file':
            if os.path.isfile(data):
                (dst, filename) = tempfile.mkstemp(dir=os.path.join(self.job_directory, "output"))
                with open(data, 'rb') as src:
                    copyfileobj(src, os.fdopen(dst, 'wb'))
                    kwargs.update({'dataType': data_type, 'file': ntpath.basename(filename),
                                   'filename': ntpath.basename(data)})
                    return kwargs
        else:
            kwargs.update({'dataType': data_type, 'data': data})
            return kwargs

[dadokkio]

Converting to binary I see the observable in the gui

but when I'm going to import the observable I have an attributecheckingerror

message: "[Invalid format for attachment: FSeq(List(FObject(Map(name -> FString(efdc94e9b627fa760c88d61ce6ec8ba0388224938725c37a33b5533f33a599fb), size -> FNumber(56320.0), id -> FString(efdc94e9b627fa760c88d61ce6ec8ba0388224938725c37a33b5533f33a599fb), contentType -> FString(application/octet-stream), hashes -> FSeq(List(FString(efdc94e9b627fa760c88d61ce6ec8ba0388224938725c37a33b5533f33a599fb), FString(5054c2793f027a8b3c3a7eb5d21f7ae1b14035f4), FString(df100c2a6d13f5d413eed98345681491))))))), expected file (file)]"

[nadouani]

Hey @dadokkio what do you mean by but I'm not sure if str support must be kept for any reason ?

[dadokkio]

That I'm not sure if this code is used somewhere else. So I'm not sure if I can change it directly or it needs a try except for both

[dadokkio]

Proposed pull, not sure if I need to open an issue in thehive to manage the import error.

[nadouani]

There is in fact an issue in TheHive 4 that doesn't allow importing the file. I'll check in TheHive 3

[nadouani]

So this is a bug in TheHive 4. The workflow works fine in TheHive 3

[dadokkio]

I've opened the issue in thehive to track update, closing this one.
Thanks for the feedback :)