How to override bot to only output `Error` or `Abort` logs

[vladdoster]

What is your question?

I am having trouble overriding the bot logger. Ideally, I would only want to output ERROR messages unless specified otherwise.

I think I chased it down to here:

singularity-cli/spython/main/base/__init__.py

Line 27 in e325c62

class Client:

I tried sub-classing client. Subclass client and then override the __init__(): method to call set_verbositywith my desiredLOG_LEVEL? I also tried setting env variables to SINGULARITY_MESSAGELEVEL&MESSAGELEVEL` but didnt seem to get anywhere. I am not too sure how to proceed. Any ideas on how to approach this? I am all ears!

Also I don't quite understand how the Client.build(stream=True) allows it to error or else it will just proceed. How does returning the generator stream allow it to error? Is it not run in bg vs fg when streamed thus allows to error in same python process?

• I found the documentation at https://singularityhub.github.io/singularity-cli/
• The documentation does not answer my question

If the documentation doesn't answer your question, where do you think it would be appropriate to add (i.e., where did you go looking for it?)

[vsoch]

Can you give me some basic commands to reproduce your issue (from the start of opening a Python interactive console, and include any dependency files)? Then I can use that to (try) and show you how to set it.

[vladdoster]

So a user submits arguments via argparse, and it instantiates a subclass which inherits from BaseHandler. Ideally the --verbose flag (True/False) would allow me to swap between no output logged and being able to change between whatever we desire (probably the default logging level). It then uses Client.build() to try and build said URI or file path the user provides.

"""
MIT License

Copyright (c) 2020 Vlad Doster

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
import contextlib
import json
import os
import pprint
import re
import shutil
import sys
from argparse import ArgumentParser
from argparse import RawTextHelpFormatter
from logging import basicConfig as LoggingConfig, getLogger, DEBUG
from time import sleep
from contextlib import contextmanager

from spython.logger import bot
from spython.logger.message import SingularityMessage, ABORT, ERROR
from spython.main import Client
from spython.main.parse.parsers import get_parser
from spython.main.parse.writers import get_writer

from csubmit.version import __version__

LoggingConfig(stream=sys.stderr, level=DEBUG)
log = getLogger()


def csubmitter_parser():
    parser = ArgumentParser(
        description="Submit a registry URI to get the container approved to run on O2.",
        formatter_class=RawTextHelpFormatter,
        prog="csubmit",
    )

    distribution_choices = ["o2", "hmsrc-registry"]
    parser.add_argument(
        "-d",
        "--distribution",
        choices=distribution_choices,
        default=distribution_choices[0],
        dest="distribution",
        help="Where to save container if it passes scan",
        required=False,
    )

    parser.add_argument(
        "-f",
        "--force",
        action="store_true",
        dest="force",
        help="Push to distribution regardless of scan results (only available to admins)",
    )

    parser.add_argument(
        "-e",
        "--email-alert",
        dest="email",
        help="Send email of status of submitted URI",
        metavar="EMAIL_ADDRESS",
        nargs=1,
    )

    parser.add_argument(
        "--image-path",
        dest="img_fpath",
        help="Supported Types:\n"
             "0. path to a Singularity .def, .simg, or .sif             (More info ar https://sylabs.io/guides/3.0/user-guide/definition_files.html)\n"
             "1. path to a Dockerfile (dockerfile should be in name)    (More info at https://docs.docker.com/engine/reference/builder/)\n",
        metavar="FILE PATH",
        nargs=1,
    )

    parser.add_argument(
        "--image-uri",
        dest="img_uri",
        help="Supported Types:\n"
             "0. docker://user/image:tag   (Find URIs at https://hub.docker.com/search?type=image)\n"
             "1. shub://user/image:tag     (Find URIs at https://singularity-hub.org/apps)\n",
        metavar="URI",
        nargs=1,
        required=False,
        type=str,
    )

    parser.add_argument(
        "--status",
        dest="status",
        help="Supply a submission id to get status",
        metavar="ID",
        nargs=1,
        required=False,
    )

    parser.add_argument(
        "-v",
        "--verbose",
        action="store_true",
        dest="verbose",
        help="Verbose details about scan",
    )

    parser.add_argument(
        "-V",
        "--version",
        action="version",
        help="show version and exit.",
        version="%(prog)s (" + __version__ + ")",
    )

    return parser


def main():
    parser = csubmitter_parser()


    def help(retval=0):
        """
        Print help, including the software version and active client and exit with return code.
        """
        parser.print_help()
        sys.exit(retval)

    # Show help if no arguments
    if len(sys.argv) == 1:
        help()
    try:
        args = parser.parse_args()
    except Exception:
        sys.exit(0)

    try:
        user = os.getenv("USER")
        if args.status:
            sys.exit("--status is not implemented yet.")
        if args.img_fpath and args.img_uri:
            sys.exit("Can not use --image-path and --image-uri at the same time")
        if args.img_fpath:
            print(FileHandler(
                distribution=args.distribution,
                force=args.force,
                user=user,
                verbose=args.verbose,
                fpath=args.img_fpath[0],
            ).process())
        elif args.img_uri:
            UriHandler(
                distribution=args.distribution,
                force=args.force,
                user=user,
                verbose=args.verbose,
                uri=args.img_uri[0],
            ).process()
        else:
            sys.exit("Need to supply either --image-path or --image-uri")
    except KeyboardInterrupt:
        print("interrupt received, cancelling submission…")
    finally:
        pass


# class ModifiedSingularityMessage(SingularityMessage):
#     super().__init__(MESSAGELEVEL=ABORT)
#


class BaseHandler(object):
    def __init__(self, distribution, force, user, verbose, fpath=None, uri=None):
        # self.bot = ModifiedSingularityMessage()
        self.distribution = distribution
        self.force = force
        self.fpath = fpath
        self.uri = uri
        self.user = user
        self.verbose = verbose
        self.def_dir = os.path.join(os.path.curdir, "image_definitions")
        self.builds_dir = os.path.join(os.path.curdir, "container_builds")
        # process
        self.verify()
        self.gen_details()

    def __str__(self):
        pp = pprint.PrettyPrinter(indent=4)
        pp.pprint(self.__dict__)

    def gen_details(self):
        raise NotImplementedError

    def verify(self):
        raise NotImplementedError

    def process(self):
        raise NotImplementedError


class FileHandler(BaseHandler):
    def __init__(self, distribution, force, user, verbose, fpath):
        self.dockerfile = False
        super().__init__(distribution, force, user, verbose, fpath)

    def verify(self):
        if not os.path.exists(self.fpath):
            sys.exit("File path does not point to a valid file")
        return True

    def gen_details(self):
        self.fname, self.fext = os.path.splitext(os.path.split(self.fpath)[1])
        self.img_def, self.img_sif = (
            self.user + "_" + self.fname.lower() + ".def",
            self.user + "_" + self.fname.lower(),
        )
        if "dockerfile" in self.img_def:
            self.dockerfile = True
        elif self.fext not in [".def", ".simg", ".sif"]:
            sys.exit("File needs to end in .def, .simg, or .sif")
        return True

    def process(self):
        try:
            if self.dockerfile:
                DockerParser = get_parser("docker")
                parser = DockerParser(self.fpath)
                SingularityWriter = get_writer("singularity")
                writer = SingularityWriter(parser.recipe)
                with open(
                        file=os.path.join(self.def_dir, self.img_def), mode="w+"
                ) as file:
                    file.write(writer.convert())
            else:
                shutil.copy(self.fpath, os.path.join(self.def_dir, self.img_def))
            _, builder = Client.build(
                build_folder=self.builds_dir,
                recipe=os.path.join(self.def_dir, self.img_def),
                image=self.img_sif,
                force=True,
                ext=".sif",
                sudo=True,
                stream=True,
                sandbox=False,
                quiet=True,
            )
            for line in builder:
                print(line, end='')
            return True
        except Exception:
            sys.exit("Check that %s syntax is correct" % self.fname)


class UriHandler(BaseHandler):
    def __init__(self, distribution, force, user, verbose, uri):
        super().__init__(distribution, force, user, verbose, uri)
        self.__str__()

    def verify(self):
        if not re.search("^[a-zA-Z]+://", self.uri):
            sys.exit("URI does not seem to be valid")

    def gen_details(self):
        self.image = self.uri[0]
        self.fname = os.path.split(self.image)[1]
        self.img_name = self.user + "_" + self.fname.lower() + ".sif"

    def process(self):
        try:
            Client.pull(
                image=self.uri,
                name=self.img_name,
                force=True,
                stream=False,
                pull_folder=self.builds_dir,
                quiet=self.verbose,
            )
            fpath, elapsed_sec, time_to_error = (
                os.path.join(self.builds_dir, self.img_name),
                0,
                5,
            )
            while not os.path.exists(fpath):
                sleep(1)
                elapsed_sec += 1
                if elapsed_sec > time_to_error:
                    raise Exception
            return True
        except Exception:
            sys.exit("Error: Image passed is invalid")


@contextmanager
def silence_stdout():
    new_target = open(os.devnull, "w")
    old_target = sys.stdout
    sys.stdout = new_target
    try:
        yield new_target
    finally:
        sys.stdout = old_target


if __name__ == "__main__":
    main()

[vsoch]

I think the issue is that you are running spython from inside python, and then executing to the command line to run singularity. The environment variables (to be honored by the client) need to be exported to the caller first. So do this before you run anything:

export MESSAGELEVEL=ERROR

If you are using it programatically like this, I suggest just setting quiet to True: https://github.com/singularityhub/singularity-cli/blob/master/spython/main/build.py#L28. Have you tried that?

[vsoch]

You can do a print here https://github.com/singularityhub/singularity-cli/blob/master/spython/logger/message.py#L286 to see if it's discovered.

[vladdoster]

It does seem to be getting caught. But I still get INFO

[vladdoster]

Even when I assign level="ERROR" and quiet=True I still get WARNING and INFO logs? Maybe I am mistaking these logs as singularity logs?

[vsoch]

Ohh thank you for showing me that! So that is an output message from Singularity, not Singularity python. You would need to pass in a verbosity level to the Singularity client via build options. Let me know if that doesn’t make sense.

[vladdoster]

ahh okay. I will do that then! Thank you so much!!!

[vladdoster]

Do you happen to know which flags .mconfig or is modifying a .go file?

[vsoch]

Oh sorry, so I mean input arguments to Client.build(), not the Singularity build proper. So if you do singularity --help you can see options for verbosity:

Linux container platform optimized for High Performance Computing (HPC) and
Enterprise Performance Computing (EPC)

Usage:
  singularity [global options...]

Description:
  Singularity containers provide an application virtualization layer enabling
  mobility of compute via both application and environment portability. With
  Singularity one is capable of building a root file system that runs on any 
  other Linux system where Singularity is installed.

Options:
  -c, --config string   specify a configuration file (for root or
                        unprivileged installation only) (default
                        "/usr/local/etc/singularity/singularity.conf")
  -d, --debug           print debugging information (highest verbosity)
  -h, --help            help for singularity
      --nocolor         print without color output (default False)
  -q, --quiet           suppress normal output
  -s, --silent          only print errors
  -v, --verbose         print additional information
      --version         version for singularity

And you would want to pass this to Client.build(). But actually, the options that go into there are passed to build (and not to singularity)

https://github.com/singularityhub/singularity-cli/blob/master/spython/main/build.py#L14

I can open a PR for you to add a singularity_options parameter that you could add something like:

Client.build(..., singularity_options=["--silent"])

Would that work? What I'd suggest you do is try the different options that the singularity (base client) can provide, and if you can find how to control the logging level, please share here with me. If you can't, then open an issue on sylabs/singularity and ask how they should go about doing this. We will expose something here that makes it possible. Sound good?

But what I'm referring to is the command line use,

[vladdoster]

Ah, thank you! That makes sense. I can definitely do that. I will report back!

[vladdoster]

Oh yes, the --silent option is ideal! So nice!

[vsoch]

okay, let me get in a PR asap so you can test.

[vladdoster]

So it seems that from most logging to least logging is:
--debug - Special DEBUG messages with what look to be memory addresses + WARNING + INFO

--verbose - Special VERBOSE messages which give some insight, In a build, I got 1 VERBOSE message + WARNING + INFO

--quiet - Only WARNING

--silent - No output unless error

[vsoch]

I think --debug and --quiet are passed on from singularity python to singularity, but not the others. Opening up a PR for you to test soon.