Added hot configuration reloading for Spring

[iliuta]

Hi,

This is a proposal for hot configuration reloading for Spring.
It is based on a thread created by EmbeddedJmxTransFactory, which scans every X seconds (configurable) the last modification date of every configuration json file.

If the last update date is greater than the last saved value, then:

1. the JmxTrans instance is stopped
2. the queries and output writers are cleaned
3. all configuration files are merged into jmxtrans
4. the JmxTrans instance is started again

A new attribute has been defined in jmxtrans-1.0.xsd, configuration-scan-period.
It corresponds to the new configurationScanPeriod attribute of EmbeddedJmxTransFactory. The new attribute is parsed and populated into EmbeddedJmxTransFactory by EmbeddedJmxTransBeanDefinitionParser.
If configuration-scan-period is 0 (default), then the hot configuration reloading is disabled (thread not created and started).

Regards,
Adrian

[buildhive]

jmxtrans project » embedded-jmxtrans #96 SUCCESS
This pull request looks good
(what's this?)

[buildhive]

jmxtrans project » embedded-jmxtrans #97 SUCCESS
This pull request looks good
(what's this?)

[buildhive]

jmxtrans project » embedded-jmxtrans #98 SUCCESS
This pull request looks good
(what's this?)

[cyrille-leclerc]

Thanks @iliuta, I'll merge it this week.

Cyrille

[iliuta]

Thank you Cyrille. Everything is all right with my pull request? Do you need further information?

[cyrille-leclerc]

Hello @iliuta, can you please explain me how to you integrate this daemon thread with the shutdown procedure?

I can't see how you stop this thread.

Cyrille

[iliuta]

Hi Cyrille, just to understand your question, do we really need to integrate this thread in a shutdown procedure? Since it is a daemon thread, it will not prevent the JVM to exit and it will be terminated anyway with no harm at the end. Am I missing some point here?

Anyway, I could easily interrupt this thread in the destroy method of EmbeddedJmxTransFactory to allow it to stop but it will make the code a little bit more complicated (not too much however). Do you want me to do this?

[cyrille-leclerc]

Hello @iliuta, yes we need to stop this thread during org.jmxtrans.embedded.EmbeddedJmxTrans#stop(). For example, it is important for webapp that can be stopped or redeployed.

Using java.util.concurrent.Executors#newScheduledThreadPool(int) will return a ScheduledThreadPool that is very simple to stop.

Cyrille

[iliuta]

Hello Cyrille,
Got it, I didn't think about application hot deployment and restart, since I never do it.
I changed to a ScheduledExecutorService with a shutdown call in destroy method.
I tested it into a Tomcat container with a jvisualvm connected to it. All threads created by embedded-jmxtrans, including the new one are properly stopped when the containing war is stopped or undeployed.

[buildhive]

jmxtrans project » embedded-jmxtrans #99 SUCCESS
This pull request looks good
(what's this?)

[cyrille-leclerc]

Hi @iliuta. I'm waiting for the Pull Request jmxtrans/jmxtrans.github.io#2 to be integrated before moving forward.

We lost the XSD files on www.jmxtrans.org.

[cyrille-leclerc]

I slightly changed the contribution

• renaming the xml config attribute configuration-scan-period into configuration-scan-period-in-seconds to prevent misunderstandings (checking the config last modified date every 30ms would have damaged applications)
• introducing a new jmxtrans XSD jmxtrans-1.1.xsd to introduce this new attribute (jmxtrans-1.0.xsd should remain immutable)

I cut the release 1.0.14, I should soon be available in Maven Central Repo.

https://github.com/jmxtrans/embedded-jmxtrans/releases/tag/embedded-jmxtrans-1.0.14

[iliuta]

Great! Now I can push my stuff into production.
Thank you, Cyrille.

[cyrille-leclerc]

Thanks Adrian,

Can you share with us the kind of OutputWriter you use? Graphite?

Cyrille

[iliuta]

I'm using GraphiteWriter to export general metrics (jvm and Tomcat) with Jmxtrans embedded into a distinct war deployed in every Tomcat on the platform. Application maintainers can provide extra jmxtrans.json files to export their application specific metrics in addition to the provided configuration file.

I had some problems with GraphiteWriter and wildcard expressions. If the subset of metrics covered by the wildcard is too large, some of them will not be sent to Graphite. It is however quite hard to reproduce that and I did not investigate further. The workaround was to provide some finer queries.

Of course, a StatsdWriter would have been nicer but GraphiteWriter is okay for now. Maybe one day I will write one from the model of main Jmxtrans project ...