Discuss adding --clean-obo option to convert

[beckyjackson]

Some ontologies contain axioms that will not properly convert to OBO, and the OBO writer sometimes fails with no good information on how to fix the issue - see OBOFoundry/COB#168

We should consider adding a hack to convert that removes these bad axioms (e.g., inverse in subClass axioms). My main concern with this is how to handle is multiple annotations (label, definition, comment). From above discussion:

Do we drop all of them, or do we just pick a random one to drop? That could result in some faulty outputs and spurious diffs.

cc @matentzn

[matentzn]

Whatever is necessary, I would vote for lexical sort order, delete everything but the first. This helps with transparency. -vvv will also INFO: what has been dropped exactly!

Right now, I am adding robot remove --axioms annotation commands in many pipelines just so I can create an OBO file. --check false is really not all that useful, if it creates illegal OBO on the other side..

I think --clean-obo extension to convert would be sweet and straight forward to implement.

[jamesaoverton]

I agree that this is a good feature to add. It needs to be deterministic, and @matentzn's suggestion of first in lexical order, and INFO for dropped annotations, sounds good to me.

Are you volunteering to implement, or find someone to implement this @matentzn?

[matentzn]

I will send my sensors out. :)

[gouttegd]

Should that --clean-obo option also direct the removal of the “untranslated axioms” (which normally end up in the owl-axioms header tag)?

I would argue that it should, given the number of pipelines in the wild that contain variations of:

robot convert -i input.owl -f obo -o tmp.obo && grep -v ^owl-axioms tmp.obo > output.obo

Those untranslated axioms are clearly not wanted by a lot of people! And it’s a pretty low hanging fruit, since the OWLAPI already has an option to prevent them from being included in the output, all ROBOT has to do is to use that option.

[matentzn]

Yes! The logger should probably Report which axioms are dropped by such an option, either with info or warning.

[gouttegd]

OK, then should it also drop GCI axioms? This is something that has been requested for CL, on the rationale that many OBO parsers are too simplistic, do not expect the presence of GCIs, and therefore do not interpret them correctly.

I am inclined to say it should (or at least, that it should allow to do it). If we add a “clean OBO“ output option, it should be suitable for existing use cases.

[matentzn]

Formally ideally, there should be sepearte options for all subproblems

• "drop GCIs" fits best under --axioms gci
• --drop-duplicate-labels
• --drop-duplicate-comments
• --axioms obo-format-incompatible

And whatever else needs to go there, and then have a super option --clean-obo that sets the truth values of all of the above to true?

[gouttegd]

I don’t really like the idea of cluttering the convert command with a bunch of OBO-specific options. It would be better then to have a dedicated obo-export command, which could take as many options as needed.

Rather, I was imaging that the single --clean-obo option could accept, in addition to a single true or false parameter, a comma-separated list of keywords that dictate the exact behaviour(s) to implement:

• strip-untranslated-axioms
• strip-gci-axioms
• fix-supernumerary-annotations

Examples:

• --clean-obo false: produce “normal” OBO output (default behaviour);
• --clean-obo strip-untranslated-axioms,strip-gci-axioms: remove all untranslated axioms as well as GCI axioms;
• --clean-obo true: clean everything we can (equivalent to --clean-obo strip-untranslated-axioms,strip-gci-axioms,fix-supernumerary-annotations).

[matentzn]

A sorry, I was talking about extending remove, not convert;

If you think extending convert is better, your suggestion is fine, bit you should use --the "usual way of grouping lists" in ROBOT rather than using , I think!

[gouttegd]

remove could probably be extended to add at least a --axioms gci option, but I am not keen on a --axioms obo-format-incompatible option.

The easiest way to drop OBO incompatible axioms is to rely on the built-in feature from the OWLAPI:

OWLAPIOwl2Obo oboConverter = new OWLAPIOwl2Obo(ontologyManager);
oboConverter.setDiscardUntranslatable(true);
OBODoc oboDoc = oboConverter.convert(ontology);
// Send the oboDoc to a OBOFormatWriter for actual writing

But this can only be used in the context of a writing operation, just before we actually write the output.

Trying to remove OBO incompatible axioms outside of such of context (e.g. during a remove operation) would require ROBOT to implement its own logic to figure out which axioms are “OBO incompatible” and must be removed. This is I believe a needless complication, and one that carries the risk that the OWLAPI and ROBOT could have a different idea of which axioms are “untranslatable”.

[gouttegd]

Regarding the treatment of “supernumerary annotations”, I propose the following behaviour:

For labels

• If one label is language-agnostic and the others have a language tag, we keep only the language-agnostic one.
• Otherwise, we do as already proposed earlier: sort the labels lexicographically and keep whichever label comes out first.

For definitions

Same treatment as the labels.

For comments

• If one comment is language-agnostic and the others have a language tag, we keep only the language-agnostic one.
• Otherwise, we sort the comments lexicographically and merge them into a single comment (adding a space between them).

Rationale for merging the comments: (1) Because that is what has been requested for CL. (2) It avoids a loss of potentially useful information. (3) Merging comments should be relatively harmless, contrary to merging labels (which would be a very bad idea, as it is unlikely that a label formed by merging several labels together could end up being meaningful; likewise for definitions).