runtime.properties not propagated to indexing job on external Hadoop leading to Javascript not being enabled in indexing task

[hochbergg]

I've set 'druid.javascript.enabled' to true in common.runtime.properties, however in the JavaScriptConfig in the mapred job on the EMR cluster "enabled" is "false".

I verified it isn't a config problem as javascript is enablewhen I run the task using the local mapred executor in the middle managers it works.

I think this has something to do with config files not being pushed to the Hadoop mapred tasks, and my reasoning is that the peon says:
2017-04-17T21:14:12,232 INFO [main] io.druid.cli.CliPeon - * druid.javascript.enabled: true

In the druid log, but the map tasks have:
2017-04-17T21:14:55,224 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.js.JavaScriptConfig] from props[druid.javascript.] as [JavaScriptConfig{enabled=false}]

I'm using 0.10.0-rc2 with EMR 2.7.3

Has anyone encountered this? Does anyone know how to work around this?

Thanks!
Gal

[gianm]

See also https://groups.google.com/d/msg/druid-user/V6CqunkE5Qs/SSC8-dc6DQAJ.

[hochbergg]

In the Google Group @gianm correctly said I can add
"jobProperties" : {
"mapreduce.map.java.opts" : "-server -Xms512m -Xmx512m -Ddruid.javascript.enabled=true -Duser.timezone=UTC -Dfile.encoding=UTF-8",
"mapreduce.reduce.java.opts" : "-server -Xms2g -Xmx2g -Ddruid.javascript.enabled=true -Duser.timezone=UTC -Dfile.encoding=UTF-8"
}

to the jobProperties to set the system properties on the job, making it enable javascript.

Its probably worth either documenting this or adding code to have the Indexing Service pass this down to the Hadoop job.

[b-slim]

Fixed in #4116