Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
8cad71d
SAMZA-2397: Updating gauge-val function on newGauge on same metric name
rmatharu Nov 22, 2019
f64d6cf
Adding unit test for metrics
rmatharu Nov 22, 2019
7498855
Adding separate method on MetricsHelper to get new gauge with overwrite
rmatharu Nov 22, 2019
245eefd
Adding separate method on MetricsHelper to get new gauge with overwrite
rmatharu Nov 22, 2019
4764ade
Empty commit to trigger build
rmatharu Dec 2, 2019
959b3d8
Addressing review comments
rmatharu Dec 12, 2019
16256fb
Minor naming changes
rmatharu Dec 13, 2019
7e558bf
Trigger build
rmatharu Jan 18, 2020
dfb4cee
SAMZA-2333: [AM isolation] Use cytodynamics classloader to launch job…
cameronlee314 Nov 26, 2019
d2e0375
update release instruction to include integration tests (#1224)
lhaiesp Nov 26, 2019
069a8c0
SAMZA-2403: fix standalone integration failure (#1225)
lhaiesp Nov 27, 2019
20eb345
SAMZA-2388: [Scala cleanup] Convert Util to Java (#1221)
cameronlee314 Dec 2, 2019
5e9244a
update gradle properties to mark 1.4.0 as the next version (#1230)
lhaiesp Dec 5, 2019
6d206d5
[DOCS] Update kafka consumer command to use bootstrap-server (#1232)
lakshmi-manasa-g Dec 6, 2019
c8417b7
doc and blog related update on master branch for 1.3.0 release (#1231)
lhaiesp Dec 9, 2019
c55e558
Fix AvroRelConverter to only consider cached schema while populating …
atoomula Dec 9, 2019
d181ed8
SAMZA-2305: Stream processor should ensure previous container is stop…
mynameborat Dec 10, 2019
1854e3a
Adding null-check before incrementing metrics for bytesSerialized (#1…
rmatharu-zz Dec 10, 2019
b53cb14
SAMZA-2418: Integration tests are failing and do not properly pass pa…
cameronlee314 Dec 16, 2019
404a0fc
SAMZA-2407: Add job.config.loader.* configs in JobConfig & correspond…
Dec 18, 2019
78b6150
Update javadoc
Jan 6, 2020
fdee896
Add getConfigLoaderFactory() for easy access to config loader factory.
Jan 6, 2020
3e72812
SAMZA-2423: Heartbeat failure causes incorrect container shutdown (#1…
Dec 20, 2019
f1e0a35
SAMZA-2425: Add support for sub-query in joins
atoomula Jan 2, 2020
77499b7
Revert indentions change
Jan 8, 2020
bfb4d43
Fix typo and switch to Optional for getConfigLoaderFactory
Jan 8, 2020
abbcf86
SAMZA-2406: Add ConfigLoader, ConfigLoaderFactory interface and defau…
Dec 18, 2019
6d841d2
Add license info
Dec 18, 2019
30875ac
Update javadoc
Jan 6, 2020
e24df2b
Remove unused import
Jan 6, 2020
730dbcb
Update JavaDocs
Jan 7, 2020
bd0c537
Add javadoc for PropertiesConfigLoader
Jan 7, 2020
677f7f5
Update javadoc
Jan 9, 2020
c491a13
Move CONFIG_LOADER_PROPERTIES_PREFIX from JobConfig to ConfigLoaderFa…
Jan 9, 2020
bda8859
Refactor LocalApplicationRunner (#1249)
kw2542 Jan 10, 2020
6cf114a
Add "strict" parameter to all the log4j.xml files (#1244)
sborya Jan 13, 2020
03583a8
SAMZA-2428: Clean up unused org.apache.samza.sql.util.ConfigUtil (#1245)
kw2542 Jan 14, 2020
c2c73dd
Clean up unused org.apache.samza.autoscaling module (#1250)
kw2542 Jan 14, 2020
afcb11c
Fix the coordinator stream creation workflow.
shanthoosh Jan 15, 2020
a35de71
SAMZA-2431: Fix the checkpoint and changelog topic auto-creation. (#1…
shanthoosh Jan 15, 2020
405d8a6
Disable Flaky Tests
Jan 15, 2020
c867cd3
SAMZA-2421: Add SystemProducer for Azure Blob Storage (#1239)
lakshmi-manasa-g Jan 16, 2020
09ac78c
Fix the RocksDB TTL type conversion in change log properties generati…
shanthoosh Jan 17, 2020
c780c1a
SAMZA-2409: Update YarnJob to construct job submission env variable (…
kw2542 Jan 17, 2020
cfcec2d
SAMZA-2408: Update RemoteApplicationRunner to submit job only (#1247)
kw2542 Jan 17, 2020
d7ed48f
SAMZA-2410: Update ClusterBasedJobCoordinator config retrieval logic …
kw2542 Jan 17, 2020
dec12ff
Removing additional method
rmatharu Jan 19, 2020
dfba824
Merge branch 'master' into test-rocksdb-metric-fix
rmatharu Jan 21, 2020
e9bd445
Undoing MetricGroup change
rmatharu Jan 21, 2020
1f50662
Minor
rmatharu Jan 21, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,10 @@ class MetricsRegistryMap(val name: String) extends ReadableMetricsRegistry with
}

def newGauge[T](group: String, gauge: Gauge[T]) = {
debug("Adding new gauge %s %s %s." format (group, gauge.getName, gauge))
putAndGetGroup(group).putIfAbsent(gauge.getName, gauge)
if (putAndGetGroup(group).containsKey(gauge.getName)) {
debug("Updating existing gauge %s %s %s" format (group, gauge.getName, gauge))
}
putAndGetGroup(group).put(gauge.getName, gauge)
val realGauge = metrics.get(group).get(gauge.getName).asInstanceOf[Gauge[T]]
listeners.foreach(_.onGauge(group, realGauge))
realGauge
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import java.util

import org.apache.samza.SamzaException
import org.apache.samza.config.MapConfig
import org.apache.samza.metrics.{Gauge, MetricsRegistryMap}
import org.apache.samza.metrics.{Counter, Gauge, MetricsRegistryMap, MetricsVisitor, Timer}
import org.apache.samza.util.ExponentialSleepStrategy
import org.junit.{Assert, Test}
import org.rocksdb.{FlushOptions, Options, RocksDB, RocksIterator}
Expand Down Expand Up @@ -210,4 +210,27 @@ class TestRocksDbKeyValueStore

rocksDB.close()
}

@Test
def testRocksDBMetricsWithBulkLoadRWRecreate(): Unit = {
val registry = new MetricsRegistryMap("registrymap")
val metrics = new KeyValueStoreMetrics("dbstore", registry)

// Sample metric values for estimate-num-keys metrics
val bulkloadStoreMetricValue = "100"
val readWriteStoreMetricValue = "10"

// Metric during bulk-load/bootstrap
metrics.newGauge("estimate-num-keys", () => bulkloadStoreMetricValue)

assert(registry.getGroup("org.apache.samza.storage.kv.KeyValueStoreMetrics").
get("dbstore-estimate-num-keys").asInstanceOf[Gauge[String]].getValue.eq("100"))

// Bulk-load complete, new store in read-write mode
metrics.newGauge("estimate-num-keys", () => readWriteStoreMetricValue.toString)

assert(registry.getGroup("org.apache.samza.storage.kv.KeyValueStoreMetrics").
get("dbstore-estimate-num-keys").asInstanceOf[Gauge[String]].getValue.eq("10"))
}

}