Skip to content

Backport 1.5: fix deadlock on core state lock (#10456)#11250

Merged
mladlow merged 2 commits into
release/1.5.xfrom
backport-pr-10456-1.5
Apr 12, 2021
Merged

Backport 1.5: fix deadlock on core state lock (#10456)#11250
mladlow merged 2 commits into
release/1.5.xfrom
backport-pr-10456-1.5

Conversation

@ncabatoff
Copy link
Copy Markdown
Collaborator

  • fix race that can cause deadlock on core state lock

The bug is in the grabLockOrStop function. For specific concurrent
executions the grabLockOrStop function can return stopped=true when
the lock is still held. A comment in grabLockOrStop indicates that the
function is only used when the stateLock is held, but grabLockOrStop is
being used to acquire the stateLock. If there are concurrent goroutines
using grabLockOrStop then some concurrent executions result in
stopped=true being returned when the lock is acquired.

The fix is to add a lock and some state around which the parent and
child goroutine in the grabLockOrStop function can coordinate so that
the different concurrent executions can be handled.

This change includes a non-deterministic unit test which reliably
reproduces the problem before the fix.

  • use rand instead of time for random test stopCh close

Using time.Now().UnixNano()%2 ends up being system dependent because
different operating systems and hardware have different clock
resolution. A lower resolution will return the same unix time for a
longer period of time.

It is better to avoid this issue by using a random number generator.
This change uses the rand package default random number generator. It's
generally good to avoid using the default random number generator,
because it creates extra lock contention. For a test it should be fine.

* fix race that can cause deadlock on core state lock

The bug is in the grabLockOrStop function. For specific concurrent
executions the grabLockOrStop function can return stopped=true when
the lock is still held. A comment in grabLockOrStop indicates that the
function is only used when the stateLock is held, but grabLockOrStop is
being used to acquire the stateLock. If there are concurrent goroutines
using grabLockOrStop then some concurrent executions result in
stopped=true being returned when the lock is acquired.

The fix is to add a lock and some state around which the parent and
child goroutine in the grabLockOrStop function can coordinate so that
the different concurrent executions can be handled.

This change includes a non-deterministic unit test which reliably
reproduces the problem before the fix.

* use rand instead of time for random test stopCh close

Using time.Now().UnixNano()%2 ends up being system dependent because
different operating systems and hardware have different clock
resolution. A lower resolution will return the same unix time for a
longer period of time.

It is better to avoid this issue by using a random number generator.
This change uses the rand package default random number generator. It's
generally good to avoid using the default random number generator,
because it creates extra lock contention. For a test it should be fine.
@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 31, 2021

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployments, click below or on the icon next to each commit.

vault – ./website

🔍 Inspect: https://vercel.com/hashicorp/vault/8zZJnJwS6u112CtsRPPiy2eLEHHW
✅ Preview: Canceled

[Deployment for 45d6e29 failed]

vault-storybook – ./ui

🔍 Inspect: https://vercel.com/hashicorp/vault-storybook/Ek82zAXPznCT7X51Pizu5Pfv17q8
✅ Preview: Canceled

[Deployment for 45d6e29 canceled]

@ncabatoff ncabatoff added this to the 1.5.8 milestone Mar 31, 2021
@vercel vercel Bot temporarily deployed to Preview – vault-storybook April 12, 2021 17:05 Inactive
@mladlow mladlow merged commit 12bda48 into release/1.5.x Apr 12, 2021
pull Bot pushed a commit to mayocream/vault that referenced this pull request Dec 15, 2025
* replace Hds::Reveal with Hds::Accordion

* adjust spacing to render in Hds::Form component

* fix spacing in policy-example

* cleanup form-section class usage

* implement visual builder in create policy form

* hide visual editor in search select modal

* use general selectors, alphabetize form/field selectors

* update test coverage to check for visual policy editor

* reorganzie tests by module

* add saving functionality for visual editor

* refactor event handling methods

* refactor component so parent manages stanzas

* move snippets to automation-snippets tab component

* polish up policy diff modal

* refactor arg to be isCompact

* update test coverage and export new component

* rearrange methods to make diff easier

* small cleanup, abc vars and remove unneeded change

* add lanuage and update test coverage

* update comment

* fix form hierarchy

* fix modal spacing;

Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants