Skip to content

Add mutex locks to sample indexing strategies#4075

Merged
achamayou merged 17 commits intomicrosoft:mainfrom
eddyashton:indexing_thread_safety
Jul 31, 2022
Merged

Add mutex locks to sample indexing strategies#4075
achamayou merged 17 commits intomicrosoft:mainfrom
eddyashton:indexing_thread_safety

Conversation

@eddyashton
Copy link
Member

@eddyashton eddyashton commented Jul 27, 2022

Resolves #4038.

The mutexes are small and simple. The unit tests are large, complex, largely copied and pasted setup from other instances, and not worth reading.

@eddyashton eddyashton requested a review from a team July 27, 2022 14:11
@ghost
Copy link

ghost commented Jul 27, 2022 •
edited by ghost
Loading

indexing_thread_safety@48212 aka 20220731.2 vs main ewma over 20 builds from 47897 to 48201

Click to see table

main

build_id build_number tpcc_sgx_cft^ tpcc_sgx_cft_mem ls_sgx_cft^ ls_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_v8_sgx_cft^ ls_v8_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_full_v8_sgx_cft^ ls_full_v8_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem hist_sgx_cft^ RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
47897 20220720.28 6377.58 8.3497e+07 20748.1 1.69124e+07 5738.23 1.56017e+07 2569.08 1.03588e+07 1637.76 1.66335e+08 2128.14 1.21939e+07 1440.04 9.81771e+07 1934.55 9.04813e+06 20951.1 883890 1.36386e+06 9.17958e+06 3.54933e+07
47905 20220721.2 6708.02 8.32349e+07 20841.9 1.58639e+07 5644.58 1.56017e+07 2562.93 1.00967e+07 1621.37 1.66597e+08 2123.05 9.31027e+06 1427.49 9.81771e+07 1978.68 8.78598e+06 23318.4 898512 1.35198e+06 9.30901e+06 3.55556e+07
47909 20220721.4 6659.57 8.3497e+07 20657 1.58639e+07 5693.2 1.53396e+07 2558.29 1.00967e+07 1632.89 1.66859e+08 2128.53 9.31027e+06 1443.32 9.87014e+07 1918.81 9.04813e+06 22526.7 854860 1.36351e+06 9.24179e+06 3.58669e+07
47923 20220721.10 6395.09 8.3497e+07 20406.3 1.58639e+07 5573.28 1.53396e+07 2520.68 1.2456e+07 1609.11 1.66859e+08 2114.86 9.31027e+06 1421.32 9.84392e+07 1875.17 9.04813e+06 22713.6 904947 1.37182e+06 9.26688e+06 3.58663e+07
47926 20220722.1 6559.88 8.32349e+07 19974.9 1.6126e+07 5745.67 1.50774e+07 2554.17 1.00967e+07 1666.79 1.65024e+08 2132.5 9.31027e+06 1439.08 9.84392e+07 1940.42 8.78598e+06 18130.5 889049 1.35819e+06 9.27121e+06 3.58042e+07
47939 20220722.7 5985.98 8.37592e+07 19798.5 1.58639e+07 5651.98 1.53396e+07 2551.84 1.00967e+07 1635.45 1.67121e+08 2125.5 9.31027e+06 1420.71 9.81771e+07 1976.48 8.78598e+06 18489.4 888720 1.37669e+06 9.36002e+06 3.58036e+07
47944 20220722.9 6418.19 8.37592e+07 20099.7 1.6126e+07 5601.76 1.56017e+07 2549.59 1.00967e+07 1589.25 1.66859e+08 2126.97 9.31027e+06 1416.11 9.81771e+07 1934.91 9.04813e+06 18668.2 905590 1.36016e+06 9.31328e+06 3.59292e+07
47950 20220722.11 6346.72 8.42835e+07 20498 1.66503e+07 5696.14 1.48153e+07 2547.54 1.00967e+07 1593.79 1.66597e+08 2087.82 9.31027e+06 1417.11 9.76528e+07 1929.17 8.78598e+06 20113.6 900215 1.37311e+06 9.21273e+06 3.56794e+07
47964 20220725.3 6445.58 8.45456e+07 20264.1 1.6126e+07 5715.87 1.53396e+07 2543.74 1.00967e+07 1612.58 1.66597e+08 2108.59 9.31027e+06 1435.99 9.76528e+07 1937.49 1.0621e+07 18993.5 902517 1.33865e+06 9.39876e+06 3.48893e+07
47991 20220726.3 6424.54 8.29727e+07 20484.1 1.58639e+07 5660.34 1.53396e+07 2552.74 1.00967e+07 1590.25 1.66597e+08 2122.4 9.04813e+06 1429.67 9.81771e+07 1926.78 8.78598e+06 20464.4 890933 1.38154e+06 9.48144e+06 3.62478e+07
47999 20220726.6 6463.07 8.3497e+07 20411.6 1.58639e+07 5679.05 1.56017e+07 2544.17 1.00967e+07 1621.25 1.66859e+08 2164.94 9.31027e+06 1426.71 9.84392e+07 2024.37 9.04813e+06 17785.5 908399 1.36251e+06 9.39872e+06 3.65714e+07
48019 20220726.14 6543.72 8.42835e+07 20091.2 1.56017e+07 5601.14 1.50774e+07 2499.24 9.83456e+06 1600.03 1.645e+08 2122.92 9.31027e+06 1420.35 9.79149e+07 2014.48 9.04813e+06 19942.2 903392 1.39861e+06 9.02994e+06 3.59298e+07
48039 20220726.21 6604.8 8.45456e+07 20061.9 1.63882e+07 5657.52 1.53396e+07 2462.46 1.29803e+07 1599.33 1.67121e+08 2133.23 9.04813e+06 1444.67 9.79149e+07 1932.82 9.04813e+06 20529.4 898398 1.37154e+06 9.20445e+06 3.59292e+07
48056 20220727.3 7376.83 8.48077e+07 20455.4 1.58639e+07 5645.94 1.53396e+07 2553.28 1.00967e+07 1612.2 1.66597e+08 2117.56 9.31027e+06 1424.76 9.81771e+07 1932.75 8.78598e+06 18180.7 905868 1.38004e+06 9.22514e+06 3.58669e+07
48076 20220728.1 6377.5 8.45456e+07 20423.6 1.6126e+07 5695.92 1.53396e+07 2536.06 1.03588e+07 1631.52 1.66859e+08 2129.7 9.31027e+06 1436.56 9.81771e+07 1937.91 9.04813e+06 19899.1 861326 1.3754e+06 9.20855e+06 3.55549e+07
48083 20220728.4 6537.16 8.32349e+07 20590.5 1.6126e+07 5720.58 1.56017e+07 2631.03 9.83456e+06 1599.78 1.64762e+08 2126.15 9.31027e+06 1454.7 9.81771e+07 1946.79 9.04813e+06 22317.3 890155 1.36259e+06 9.20855e+06 3.58669e+07
48101 20220728.10 6449.49 8.42835e+07 20328.8 1.56017e+07 5590.59 1.50774e+07 2545.39 1.00967e+07 1608.68 1.66597e+08 2124.69 9.31027e+06 1415.34 9.87014e+07 1928.96 9.04813e+06 19989.8 872335 1.34153e+06 9.21684e+06 3.58663e+07
48116 20220729.2 6419.03 8.40213e+07 20316.1 1.6126e+07 5998.81 1.53396e+07 2559.25 1.00967e+07 1611.63 1.66335e+08 2119.38 9.04813e+06 1399.69 9.63421e+07 1926.59 9.04813e+06 20842.4 905830 1.37219e+06 9.22934e+06 3.61199e+07
48182 20220729.22 7435.73 8.48077e+07 20197.3 1.69124e+07 5628.12 1.56017e+07 2549.84 1.00967e+07 1619.5 1.67121e+08 2151.72 9.57242e+06 1420.17 9.81771e+07 1929.97 9.04813e+06 20385.6 906430 1.3676e+06 9.27108e+06 3.56168e+07
48201 20220729.31 6490.45 8.37592e+07 20065.7 1.6126e+07 5634.34 1.53396e+07 2558.72 1.00967e+07 1588.91 1.66859e+08 2120.95 9.31027e+06 1413.06 9.79149e+07 1935.36 9.04813e+06 21085.2 906511 1.3756e+06 9.31752e+06 3.65708e+07

indexing_thread_safety

build_id build_number tpcc_sgx_cft^ tpcc_sgx_cft_mem ls_sgx_cft^ ls_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_v8_sgx_cft^ ls_v8_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_full_v8_sgx_cft^ ls_full_v8_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem hist_sgx_cft^ RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
48065 20220727.6 6220.41 8.42835e+07 20351.2 1.58639e+07 5652.83 1.53396e+07 2545.5 1.00967e+07 1582.98 1.66859e+08 2078.55 9.31027e+06 1409.17 9.81771e+07 1924.14 9.04813e+06 20913.6 904388 1.38023e+06 9.2879e+06 3.58663e+07
48087 20220728.5 7608.09 8.32349e+07 20644.2 1.6126e+07 5655.24 1.53396e+07 2540.54 1.03588e+07 1626.34 1.66859e+08 2128.34 9.57242e+06 1428.32 9.81771e+07 1932.68 9.04813e+06 18205.2 886459 1.36341e+06 9.29637e+06 3.56794e+07
48178 20220729.20 6475.19 8.3497e+07 20459.7 1.58639e+07 5626.95 1.53396e+07 2601.9 1.00967e+07 1599.43 1.645e+08 2114.58 9.31027e+06 1416.48 9.84392e+07 1927.45 9.04813e+06 20434.3 902640 1.39795e+06 9.22523e+06 2.63578e+07
48212 20220731.2 6494.16 8.3497e+07 20368 1.63882e+07 5700.6 1.53396e+07 2477.4 1.00967e+07 1623.5 1.66597e+08 2130.09 9.04813e+06 1426.41 9.79149e+07 1942.31 9.04813e+06 20533.4 901998 1.36332e+06 9.27952e+06 3.57411e+07

images

@achamayou achamayou added 2.x-todo PRs which should be backported to 2.x auto-backport Automatically backport this PR to LTS branch labels Jul 28, 2022
@achamayou achamayou enabled auto-merge (squash) July 29, 2022 14:01
@achamayou achamayou merged commit 6b04017 into microsoft:main Jul 31, 2022
github-actions bot pushed a commit that referenced this pull request Jul 31, 2022
@eddyashton
Add mutex locks to sample indexing strategies (#4075)
ab6a50a 
(cherry picked from commit 6b04017)
@github-actions github-actions bot mentioned this pull request Jul 31, 2022
Merged
@github-actions github-actions bot added the backported This PR was successfully backported to LTS branch label Jul 31, 2022
This was referenced Aug 4, 2022
Merged
Closed
achamayou pushed a commit to achamayou/CCF that referenced this pull request Aug 16, 2022
@github-actions
[release/2.x] Cherry pick: Add mutex locks to sample indexing strateg…
4a52012 
…ies (microsoft#4075) (microsoft#4086)
@achamayou achamayou mentioned this pull request Jan 6, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@achamayou achamayou achamayou approved these changes

Assignees

No one assigned

Labels

2.x-todo PRs which should be backported to 2.x auto-backport Automatically backport this PR to LTS branch backported This PR was successfully backported to LTS branch

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Built-in indexing strategies are not thread-safe

2 participants

@eddyashton @achamayou