Right now if a supervisor crashes on startup it will appear normal from outside but will not start any tasks. If you dig through the relevant logs you will find the error (the error in my specific case is attached below) it would be great to be able to have an API to see all errors on the supervisor.
2018-10-24T21:43:31,434 ERROR [KinesisSupervisor-flow0] org.apache.druid.indexing.kinesis.supervisor.KinesisSupervisor - KinesisSupervisor[flow0] faile
d to handle notice: {class=org.apache.druid.indexing.kinesis.supervisor.KinesisSupervisor, exceptionType=class java.lang.NoSuchMethodError, exceptionMe
ssage=com.amazonaws.http.JsonResponseHandler.<init>(Lcom/amazonaws/transform/Unmarshaller;)V, noticeClass=RunNotice}
java.lang.NoSuchMethodError: com.amazonaws.http.JsonResponseHandler.<init>(Lcom/amazonaws/transform/Unmarshaller;)V
at com.amazonaws.services.kinesis.AmazonKinesisClient.describeStream(AmazonKinesisClient.java:644) ~[aws-java-sdk-kinesis-1.10.61.jar:?]
at com.amazonaws.services.kinesis.clientlibrary.proxies.KinesisProxy.getStreamInfo(KinesisProxy.java:169) ~[amazon-kinesis-client-1.6.3.jar:?]
at com.amazonaws.services.kinesis.clientlibrary.proxies.KinesisProxy.getShardList(KinesisProxy.java:223) ~[amazon-kinesis-client-1.6.3.jar:?]
at com.amazonaws.services.kinesis.clientlibrary.proxies.KinesisProxy.getAllShardIds(KinesisProxy.java:246) ~[amazon-kinesis-client-1.6.3.jar:?]
at org.apache.druid.indexing.kinesis.KinesisRecordSupplier.getPartitionIds(KinesisRecordSupplier.java:379) ~[imply-kinesis-lib-0.13.0_0.jar:0.1
3.0_0]
at org.apache.druid.indexing.kinesis.supervisor.KinesisSupervisor.updatePartitionDataFromKinesis(KinesisSupervisor.java:809) ~[kinesis-indexing
-service-0.13.0_0.jar:0.13.0_0]
at org.apache.druid.indexing.kinesis.supervisor.KinesisSupervisor.runInternal(KinesisSupervisor.java:722) ~[kinesis-indexing-service-0.13.0_0.j
ar:0.13.0_0]
at org.apache.druid.indexing.kinesis.supervisor.KinesisSupervisor$RunNotice.handle(KinesisSupervisor.java:556) ~[kinesis-indexing-service-0.13.
0_0.jar:0.13.0_0]
at org.apache.druid.indexing.kinesis.supervisor.KinesisSupervisor$2.run(KinesisSupervisor.java:353) [kinesis-indexing-service-0.13.0_0.jar:0.13
.0_0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_181]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
(I had some incompatible libs loaded)
Right now if a supervisor crashes on startup it will appear normal from outside but will not start any tasks. If you dig through the relevant logs you will find the error (the error in my specific case is attached below) it would be great to be able to have an API to see all errors on the supervisor.
(I had some incompatible libs loaded)