Skip to content

Select query failing if miliseconds used as time for indexing#6937

Merged
jon-wei merged 6 commits intoapache:masterfrom
mirkojotic:master
Feb 25, 2019
Merged

Select query failing if miliseconds used as time for indexing#6937
jon-wei merged 6 commits intoapache:masterfrom
mirkojotic:master

Conversation

@mirkojotic
Copy link
Copy Markdown
Contributor

If milliseconds were passed to DateTime#of as string joda would throw an Illegal Argument Exception because it expects milliseconds to be in form of a Long type. I've added a couple of tests to make sure it is working for seconds and milliseconds.

Let me know if I need to change anything.

Solves #1332

@fjy fjy added the Bug label Jan 29, 2019
@fjy fjy added this to the 0.14.0 milestone Jan 29, 2019
return new DateTime(instant, ISOChronology.getInstanceUTC());
}
catch (IllegalArgumentException ex) {
return new DateTime(Long.valueOf(instant), ISOChronology.getInstanceUTC());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will obscure the original exception, by reporting something like "java.lang.NumberFormatException: For input string" rather than the original DateTime String parsing exception. How about doing this instead:

    try {
      return new DateTime(instant, ISOChronology.getInstanceUTC());
    }
    catch (IllegalArgumentException ex) {
      try {
        return new DateTime(Long.valueOf(instant), ISOChronology.getInstanceUTC());
      } catch (IllegalArgumentException ex2) {
        // Throw the original exception, since it is more likely to be meaningful to the user.
        throw ex;
      }
    }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An oversight on my part. I'll make the change. Thanks

Copy link
Copy Markdown
Contributor

@gianm gianm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Production code LGTM, but please adjust the test.

{
Long milis = System.currentTimeMillis();
DateTime dt1 = DateTimes.of(milis.toString());
Assert.assertTrue(DateTimes.COMMON_DATE_TIME_PATTERN.matcher(dt1.toString()).matches());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please edit this test to check that the parsed values are actually what we expect (not just that it matches COMMON_DATE_TIME_PATTERN). It would probably help to use a constant instead of System.currentTimeMillis().

@gianm
Copy link
Copy Markdown
Contributor

gianm commented Feb 19, 2019

@mirkojotic - are you able to adjust the test? If so we can get this into Druid 0.14.0.

@mirkojotic
Copy link
Copy Markdown
Contributor Author

@gianm Yes. Will be working on it tomorrow.

- Try converting to Integer and then multiply by 1000L to achieve milis.
- If not successful try converting to Long or rethrow original
exception.
- in addition to seconds and milisecs, this method currently supports
even a date string.
{
return new DateTime(instant, ISOChronology.getInstanceUTC());
try {
Integer seconds = Integer.valueOf(instant);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code won't work for timestamps that are near the epoch. I think it's a bit too clever and we should stick to only working on milliseconds timestamps and ISO8601 strings.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I wrote it last night I realized it's doing too much.

I'll work on it today.

@mirkojotic
Copy link
Copy Markdown
Contributor Author

@gianm let me know if it needs more work

Copy link
Copy Markdown
Contributor

@jon-wei jon-wei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jon-wei jon-wei merged commit f6a8e03 into apache:master Feb 25, 2019
jon-wei pushed a commit to jon-wei/druid that referenced this pull request Feb 25, 2019
…#6937)

* [apache#1332] Fix - select failing if milis used for idx.

* Formating correction.

* Address comment: throw original exception.

* Using constant values in tests

- Try converting to Integer and then multiply by 1000L to achieve milis.
- If not successful try converting to Long or rethrow original
exception.

* DateTime#of has to support "2011-01-01T00:00:00"

- in addition to seconds and milisecs, this method currently supports
even a date string.

* Handle only milisec timestamps and ISO8601 strings
fjy pushed a commit that referenced this pull request Feb 26, 2019
…#7140)

* [#1332] Fix - select failing if milis used for idx.

* Formating correction.

* Address comment: throw original exception.

* Using constant values in tests

- Try converting to Integer and then multiply by 1000L to achieve milis.
- If not successful try converting to Long or rethrow original
exception.

* DateTime#of has to support "2011-01-01T00:00:00"

- in addition to seconds and milisecs, this method currently supports
even a date string.

* Handle only milisec timestamps and ISO8601 strings
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants