Auditing should support Java 8 Date & Time types [DATACMNS-411]

[spring-projects-issues]

Nick Williams opened DATACMNS-411 and commented

This may or may not be in the right project. I think it is, but it may need to be moved to Spring Data Commons.

Currently, fields annotated @CreatedDate and @LastModifiedDate can be Joda Time's DateTime or (I think) java.util.Date, java.util.Calendar, or java.sql.Timestamp. If it doesn't support these last three types, it should.

Auditing should also support the Java 8 Date & Time API. Fields modified with one of these two annotations should be able to be java.time.Instant, java.time.LocalDateTime, java.time.OffsetDateTime, and java.time.ZonedDateTime

---

Affects: 1.7 M1 (Codd), 1.6.3 (Babbage SR2)

Issue Links:

• SPR-11259 Add Converter implementations that convert legacy Date instances into JDK 8 date/time types
  ("depends on")

• DATAJPA-592 Auditing annotations not working when applied to methods

• DATACMNS-307 Remove JodaTime dependency

Referenced from: commits 51b128f

[spring-projects-issues]

Nick Williams commented

It's been almost five months since I recorded this issue, and I haven't even received a comment about it. I'd like to get some clarification on:

1. Which auditing date types does Spring Data support other than Joda's DateTime? I think java.util.Date, java.util.Calendar, and java.sql.Timestamp, but I can't find any confirmation of this.
2. I was hoping support for java.time.Instant, java.time.LocalDateTime, java.time.OffsetDateTime, and java.time.ZonedDateTime would make it into Commons 1.6/JPA 1.4, but it appears that didn't happen. Can I get some feedback on whether this can be scheduled for Commons 1.7/JPA 1.5?
3. Can I get some feedback on whether this issue is even in the right project?

[spring-projects-issues]

Oliver Drotbohm commented

The currently supported types can be found in AnnotationAuditingMetadata. The reasons the support for JDK 8 date time types not being of primary focus right now are manifold:

1. There's not a lot of people (read: none :) ) asking for that support besides you. We generally prioritize based on feedback and votes.
2. The point above is probably caused by Java 8 being far from production ready and thus not being used intesively right now
3. Adding support for that stuff in Spring Data Commons would require us to compile against Java 8. Currently there doesn't seem to be support for a compilation scenario of Java 8 sources down to Java 6 compatible bytecode with Maven.

The third point is the most blocking reason right now. I've briefly thought about making the ConversionService to be used when setting the date values configurable but eventually refrained from that as we have to apply the registration of additional converters which might cause unwanted side effects with the registered service instance.

I've filed SPR-11259 as Spring Framework already builds against JDK 8 (using Gradle) and if it added the necessary converters there's only little changes we needed to apply in Spring Data Commons. Plus, we generally support Java 8 on Spring 4 anyway, so it'd be a nice match and we didn't have to rule out incompatible scenarios (e.g. Spring 3.2.x on Java 8) ourselves

[spring-projects-issues]

Oliver Drotbohm commented

The tip regarding compilation against JDK 8 types on Twitter was really helpful. Thanks a lot for that. However our tests do not run on Java 8, mostly because of the ASM version inlined in Spring 3.2.x is not capable of handling Java 8 classes and thus a lot of container internals start to fail.

The workaround would be to run our tests against Spring 4 which effectively won't work without a hard dependency upgrade to Spring 4 (again, due to Maven restrictions). I'll see Juergen Hoeller early January so we might be able to roll the fix for the Spring Framework ticket into Spring 4.0.1, so that we can transparently benefit from using Spring Data alongside Spring 4. If that happens, we even include the necessary changes from our side into a bugfix release of Spring Data Commons

[spring-projects-issues]

Nick Williams commented

1. Thanks for moving the issue to DATACMNS for me.
2. You definitely won't need Gradle to use JDK 8 while compiling for Java 6. Here's how: ```xml

<project>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<fork>true</fork><!-- this must be enabled for below to work -->
<executable>${JDK_8}/bin/javac</executable>
</configuration>
</plugin>
</plugins>
</build>
...
</project>```
This ensures that the Java 8 compiler is used but that only Java 6 language features are used. Alternatively you can just document the requirement for a Java 8 compiler, remove the \<fork> and `<executable>`, and the project will simply fail to compile if they use Java 7 or earlier.

1. Okay, I see that we're supporting Joda's DateTime, java.util.Date, Long, and long. IMO, anything officially supported for timestamps in JPA should be officially supported for auditing in Spring Data, so I think we should also support java.util.Calendar and java.sql.Timestamp. Can that, at least, make it into Commons 1.7?
2. Java 8 will be released to developers on January 24 and to the general public on March 14. Its API is frozen, its specification is approved. That's not "far from production ready" :-). My interest is that I'm writing a book covering Spring Framework 4.0 / Java EE 7 / Java SE 8 / JPA / Spring Data / Spring Security. I go to the printers on January 25. I'd really love to say Spring Data auditing supports the Java 8 Date and Time types.
3. I'm not sure what the point of SPR-11259 and converters is. Wouldn't it be more performant to instantiate the java.time types directly? Right now you're always creating a Joda DateTime and then converting it to a Date or Long if needed. This is not ideal. It sounds like for java.time you're going to create a Joda DateTime and then convert it to a Date and then convert that to a java.time type. That's even less ideal. If developers aren't using Joda Time, they're not going to want Joda Time on their classpath. If they're using java.time, they're REALLY not going to want Joda Time on their classpath. Personally, I'll just not use auditing at all if it means I have to have Joda Time on my classpath. Just instantiate each type on its own: ```java
   new Date()
   new Date().getTime()
   new Timestamp(new Date().getTime())
   Calendar.getInstance()
   Instant.now()
   LocalDateTime.now()
   OffsetDateTime.now()
   ZonedDateTime.now()

**Here's the deal:** I've contributed code to Spring Data before and I'm willing to do it again. However, my time is very limited right now. _*IF*_ a pull request from me in the next week that gets support for all of these types will be approved for Commons 1.7, I can set aside the time to make it happen. However, if it will be deferred to 1.8 no matter how quickly I get the pull request in, I'll have to wait until the book's out the door to find the time for it. Thoughts?

[spring-projects-issues]

Oliver Drotbohm commented

My general sentiment is that I don't want to start to spread "if Java 8 do that, else do that" code inside our codebase. Spring Framework already has this kind of stuff in it, so if it can take over the burden, even better. Especially as we support Java 8 on Spring 4 only. So we'd derail from this path by adding Java 8 specific code to a Spring 3.2.x codebase and then guarding it with "if (on Java 8 && on Spring 4) { …".

Regarding 3. To be honest: types like Calendar and Timestamp are definitely ones I'd like to avoid to leak into my domain model as much as I could. So I'd like to wait for further feedback if it's really necessary to add this support.

Regarding 4. I wasn't arguing from a stability perspective of Java 8. I was arguing from a distribution perspective amongst Java developers and even more important: real-world Java projects. We've taken care of Spring Data working without issues on Java 8 in the latest release and will continue to improve the support going forward. I just don't want to rush a half-baked solution into the library quickly, if there's no urgent need for it, especially if the alternatives are clearly obvious and just need a bit of time to be discussed. Going forward, there's definitely an option to arrange suiting release dates alongside external deadlines. This just not going to work 3 weeks in advance.

Regarding 5. I think it probably would. But I don't think the gained performance benefit is worth the complexity added to the codebase. The different supported types are effectively different representations of the same thing, exactly the kind of thing the ConversionService is made for

[spring-projects-issues]

Nick Williams commented

You misunderstand. I'm not suggesting we have even one if(Java 8) or if(Spring 4) test. That would be poor design. I'm talking about something that has good design, performs well, and doesn't have hard-coded dependencies on Joda, Spring 4, or Java 8. For example (almost pseudocode):

public interface AuditingTimestampGenerator<T> {
    T generateTimestamp();
}

public class DateAuditingTimestampGenerator implements AuditingTimestampGenerator<Date> {
    Date generateTimestamp() {
        return new Date();
    }
}

public class CalendarAuditingTimestampGenerator implements AuditingTimestampGenerator<Calendar> {
    Calendar generateTimestamp() {
        return Calendar.getInstance();
    }
}

public class TimestampAuditingTimestampGenerator implements AuditingTimestampGenerator<Timestamp> {
    Timestamp generateTimestamp() {
        return new Timestame(new Date().getTime());
    }
}

public class DateTimeAuditingTimestampGenerator implements AuditingTimestampGenerator<DateTime> {
    DateTime generateTimestamp() {
        return new DateTime();
    }
}

public class InstantAuditingTimestampGenerator implements AuditingTimestampGenerator<Instant> {
    Instant generateTimestamp() {
        return Instant.now();
    }
}

public class LocalDateTimeAuditingTimestampGenerator implements AuditingTimestampGenerator<LocalDateTime> {
    LocalDateTime generateTimestamp() {
        return LocalDateTime.now();
    }
}

public class OffsetDateTimeAuditingTimestampGenerator implements AuditingTimestampGenerator<OffsetDateTime> {
    OffsetDateTime generateTimestamp() {
        return OffsetDateTime.now();
    }
}

public class ZonedDateTimeAuditingTimestampGenerator implements AuditingTimestampGenerator<ZonedDateTime> {
    ZonedDateTime generateTimestamp() {
        return ZonedDateTime.now();
    }
}

Then, AnnotationAuditingMetadata would pick the appropriate generator like so:

...
        if (Date.class == returnType) {
            return new DateAuditingTimestampGenerator();
        }
        else if (Calendar.class == returnType) {
            return new CalendarAuditingTimestampGenerator();
        }
        else if (Timestamp.class == returnType) {
            return new TimestampAuditingTimestampGenerator();
        }
        else if ("org.joda.time.DateTime".equals(returnType.getName())) {
            return new DateTimeAuditingTimestampGenerator();
        }
        else if ("java.time.Instant".equals(returnType.getName())) {
            return new InstantAuditingTimestampGenerator();
        }
        else if ("java.time.LocalDateTime".equals(returnType.getName())) {
            return new LocalDateTimeAuditingTimestampGenerator();
        }
        else if ("java.time.OffsetDateTime".equals(returnType.getName())) {
            return new OffsetDateTimeAuditingTimestampGenerator();
        }
        else if ("java.time.ZonedDateTime".equals(returnType.getName())) {
            return new ZonedDateTimeAuditingTimestampGenerator();
        }
...

This is nearly identical to how Spring's ServletRequestMethodArgumentResolver resolves arguments, except Spring Data's series of if statements would only execute once because they're part of metadata generation:

...
                else if (Locale.class.equals(paramType)) {
                        return RequestContextUtils.getLocale(request);
                }
                else if (TimeZone.class.equals(paramType)) {
                        TimeZone timeZone = RequestContextUtils.getTimeZone(request);
                        return (timeZone != null ? timeZone : TimeZone.getDefault());
                }
                else if ("java.time.ZoneId".equals(paramType.getName())) {
                        return ZoneIdResolver.resolveZoneId(request);
                }
...

My primary concern now is the hard-coded dependency on Joda Time, even if the entity is using Date or Long

[spring-projects-issues]

Oliver Drotbohm commented

Feels like we've got different understanding of clean code. A huge if-cascade clearly is not what I was imagining. We can avoid all this stuff and simply add a single

if (conversionService.canConvert(Date.class, targetType) {
  conversionService.convert(value.toDate(), targetType);
}

and be done with it. No need for a plethora of one-method-of-one-line types per supported type, let alone the additional SPI interface etc.

I don't quite get the hassle with JodaTime. It just works, and makes things easier. Yes we could make its usage optional and only require it if someone actually uses Auditable, but if so, I'd like to track that in a separate ticket

[spring-projects-issues]

Nick Williams commented

Well that's a fair point (the if-cascade complaint) that can be solved by using a Hashtable of class names to generators. But intentionally constructing one type just to convert it to another type when you can construct the other type more easily and efficiently is not clean, either.

...
        // in AnnotationAuditingMetadata
        generators.add(Long.class.getName(), new LongAuditingTimestampGenerator());
        generators.add(Date.class.getName(), new DateAuditingTimestampGenerator());
        generators.add(Calendar.class.getName(), new CalendarAuditingTimestampGenerator());
        generators.add(Timestamp.class.getName(), new TimestampAuditingTimestampGenerator());
        if(ReflectionUtils.classExists("org.joda.time.DateTime")) {
            generators.add("org.joda.time.DateTime", new DateTimeAuditingTimestampGenerator());
        }
        if(ReflectionUtils.classExists("java.time.Instant")) {
            generators.add("java.time.Instant", new InstantAuditingTimestampGenerator());
            generators.add("java.time.LocalDateTime", new LocalDateTimeAuditingTimestampGenerator());
            generators.add("java.time.OffsetDateTime", new OffsetDateTimeAuditingTimestampGenerator());
            generators.add("java.time.ZonedDateTime", new ZonedDateTimeAuditingTimestampGenerator());
        }
...
        AuditingTimestampGenerator<?> generator = generators.get(targetType.getName());
...
        // in wherever AnnotationAuditingMetadata is used
        setDateCreatedMethod.invoke(entity, setDateCreatedGenerator.generateTimestamp());
...

That's really clean.

The "hassle" is a lot of developers don't want classes on their classpath that they aren't using. Every additional library in a Java EE web application increases the startup time because that's another library whose classes must be scanned. Prior to Java 8, I would've been using Joda Time, because like you said, it's easier and it "just works." But now that Joda Time has been ported to the Java SE platform as java.time in Java 8, I don't want the Joda Time library on my classpath. It's completely redundant. I would manually set all of my auditing properties in my services for my dozens of entities before I would use Spring Data auditing in its current state

[spring-projects-issues]

Oliver Drotbohm commented

Just wanted to give you some feedback as we made reasonable progress, which I'll polish up in the upcoming days but haven't finished yet. We'll be able to support JDK 8 types on Spring 4 transparently with the changes introduced in SPR-11259.

We'll move to a Calendar based API to achieve that which will also make the JodaTime dependency completely optional (only required if you're using Auditable or want to use JodaTime types in annotation based auditing). The fix will be shipped in Codd M1 (currently scheduled for end of January) so that you can safely bet on that stuff working for your book.

The problematic part of the support is now really moved to the underlying store as currently there's no JPA persistence provider supporting these types natively. The Jadira Usertypes project for Hibernate also only works for the backport. I haven't tested the MongoDB driver yet, but we could potentially register appropriate converters ourselves. For the JPA module we might even be able to ship some JPA 2.1 converter but the devil is in the details here again

[spring-projects-issues]

Nick Williams commented

Oliver Drotbohm,

I wouldn't worry about persisting the types to the underlying datastore. IF the user choses to create an entity whose auditing dates are the Java 8 Date & Time types, then they already have a plan for how to convert them. If they didn't have a plan, they'd choose some other auditing date type. Plus, JPA 2.2 will have native support for the Java 8 Date & Time types. I just don't think we have to put any effort into providing converters or other capabilities for converting these types to their persistence store.

Now, with all that said, I'm very glad to hear this will make its way into Commons 1.7. I see RC1 was released a few days ago. Hurray! I took a look at the code and I have a couple of questions/comments:

• I see that I can have long, Long, java.util.Date, and java.time auditing fields in my entities. That's obvious from the code. What's not clear is whether or not I can have Calendar auditing fields in my entities. Calendar was not previously supported, but now that we have a Calendar-based API it seems like we should naturally support Calendar auditing fields. Can you confirm that Calendar is supported for auditing fields? It's not in the List of supported types with Date, Long, etc. That's why I'm confused.
• I think supporting all types that start with java.time is a mistake. There are a lot of types that we wouldn't want to support in java.time. For example, you don't want to support any of the classes in java.time.format (DateTimeFormatter, DecimalStyle, etc.) do you? What about java.time.DayOfWeek? Most of the types in java.time make no sense as auditing properties. The thing that long, Date, Calendar, DateTime, and LocalDateTime all have in common is that, somehow, they all store the date and the time. Thus, I recommend we limit the support to java.time.Instant, java.time.LocalDateTime, java.time.OffsetDateTime, and java.time.ZonedDateTime. Those are the only java.time classes that store the date and the time.

Thanks again!

[spring-projects-issues]

Oliver Drotbohm commented

@ Java 8 Date / Time types and JPA - Well, there's other aspects to add here that might not be so obvious from a very "my special way of using it" perspective. Some of these aspects were reasons I wasn't to eager to get the support in so quickly but let me outline this:

Us providing the support for Java 8 Date / Time types will make people assume they just work as other date / time types, which is what the Java 8 ones clearly do not for now. It might not be a surprise to you, but it will be to others.

Not sure, where you found the information about the JPA 2.2 feature set (I am on the expert group and don't have any so far) but even if we assume that JPA 2.2 will require persistence providers to support these types, when exactly will be actually seeing implementations for that? By 2016? Seems like we're at least going to see support for Hibernate in that regard pretty soon.

@ Calendar support - yes, that's there. We'll have the documentation on that improved for GA.

@ java.time types support - we do not support all date time types. We support the ones that Spring 4 can convert a Calendar to. Our sanity check for package actually is just that: a guard to generally shield against any other unsupported types. I don't want to repeat the exact checks, Spring 4 does as this would require Spring Data Commons to be compiled against JDK 8 and Spring 4, which is definitely not worth the effort for now

[spring-projects-issues]

Nick Williams commented

Oliver Drotbohm, thanks for clearing this up for me. Makes a lot of sense.

I didn't "find" information about the JPA 2.2 feature set. Java EE 8 will naturally require Java SE 8, which requires support for the java.time types in JDBC, so naturally I imagine JPA will require support for the java.time types. Also (some time ago) I created JPA_SPEC-63 to address this requirement. I did not realize you were on the expert group—that's good to know. I'm in the process of joining the JCP and am interested in joining the JPA expert group.

I understand the motivation for including converters in Spring Data and/or Spring Framework. However, if you do, please be sure that autoApply in @Converter is set to false. Otherwise, they may interfere with converters the user supplies

[spring-projects-issues]

Holger Stolzenberg commented

Hey guys,

I am currently setting up a new project with spring-4, spring-boot-1.0.0.RC4, Hibernate.4.3, mysql-5.6 and JDK-8. I´d also like to use the new java.time API with JPA auditing. Everything is fine so far, but I ended up using Jadira, otherwise the bean properties for auditing are persisted as tinyblob or varbinary, depending on the RDBMS used. Here is my bean mapping:

@MappedSuperclass
@EntityListeners(AuditingEntityListener.class)
public class AbstractEntity implements Serializable {
  private static final long serialVersionUID = -7004803893934683334L;

  @Id
  @GeneratedValue(strategy = GenerationType.AUTO)
  private Long id;

  @Column(name = "created_by")
  @CreatedBy
  private String createdBy;

  @Column(name = "created_when")
  @CreatedDate
  @Type(type = "org.jadira.usertype.dateandtime.threeten.PersistentZonedDateTime")
  @DateTimeFormat(iso = DateTimeFormat.ISO.DATE_TIME)
  private ZonedDateTime createdWhen;

  @Column(name = "modified_by")
  @LastModifiedBy
  private String modifiedBy;

  @Column(name = "modified_when")
  @LastModifiedDate
  @Type(type = "org.jadira.usertype.dateandtime.threeten.PersistentZonedDateTime")
  @DateTimeFormat(iso = DateTimeFormat.ISO.DATE_TIME)
  private ZonedDateTime modifiedWhen;

  @Column(name = "revision")
  @Version
  private Long revision;

  AbstractEntity() {
  }

  …
}

I have defined my database columns for created_when and modified_when as MySQL Timestamps. The Hibernate Validator is instructed to only validate the DB schema. Hibernate then complains:

Caused by: org.hibernate.HibernateException: Wrong column type in dashboard.agent for column created_when. Found: timestamp, expected: tiny blob

Having tinyblob for time columns is no option, as it can´t be queried easily via SQL. I tried to annotate the fields with @Temporal but then another mapping exception is thrown.

I also tried following example project, but the problem occurs there two. Am I missing something or is this the desired behaviour? I ended up using Jadira, but this makes explicit @Type declarations necessary.

I am not sure if this is an spring-boot, spring-data-jpa or Hibernate issue, so please forgive if this is not the right location for the report.

In the end, is it possible to map ZonedDateTime to MySQL timestamp columns without the use of Jadira?

[spring-projects-issues]

Nick Williams commented

If you're using Hibernate directly, you need to create a custom UserType to handle the ZonedDateTime and then annotate your entity properties with Hibernate's user type association annotation so that Hibernate picks up the custom UserType.

If you're using JPA, you must use the latest version (JPA 2.1, part of Java EE 7), create a custom Converter, then annotate your entity properties with the Convert annotation so that Hibernate picks up the custom Converter

[spring-projects-issues]

Holger Stolzenberg commented

Thanks for the quick reply. So in the end I will stick with Jadira for now, as I see no advantage in writing my own Converter