Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make client a start/stop service #272

Merged
merged 1 commit into from
Mar 17, 2024
Merged

Conversation

brandur
Copy link
Contributor

@brandur brandur commented Mar 16, 2024

Here, continue service refactoring to its ultimate conclusion, and make
the client itself a start/stop service, having a couple advantages:

  • (IMO) considerably simplifies the start and stop code, putting it all
    in one place, and largely even one function. Fewer helpers to follow
    around to understand what's going on.

  • Makes the client behave gracefully on double starts/stops.

  • Allows the client to easily handle start/stop under duress. Any number
    of goroutines can be starting or stopping it simultaneously and it's
    guaranteed free from races.

Because the client's stop functions are different from the signature of
other start/stop services (taking a context and returning an error), I
had to modify the base start/stop service somewhat and allow for an
additional StopInit helper (like StartInit) that lets stop behavior
be customized while still providing race-free operations.

@brandur brandur force-pushed the brandur-client-start-stop-2 branch from fb3c6c6 to 7696b6e Compare March 16, 2024 23:09
@brandur
Copy link
Contributor Author

brandur commented Mar 16, 2024

This one a lot more readable without whitespace changes:

https://github.com/riverqueue/river/pull/272/files?w=1

err := client.Stop(ctx)
require.Error(t, err)
require.Equal(t, "client not started", err.Error())
})
Copy link
Contributor Author

@brandur brandur Mar 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally left this error in, but to guarantee non-raciness, I would've had to an extra mutex to protect it, and it didn't seem worth it since the error's not particularly valuable. (The error wasn't race protected before either, but now we're actually testing it so the problem was revealed.)

@brandur brandur requested a review from bgentry March 17, 2024 00:05
Comment on lines +678 to +679
go func() {
defer close(stopped)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be more readable and easily debuggable if some of these closures were split into internal client methods. Thinking in particular of the potential for a trace to have anonymous goroutines rather than named methods.

Not a huge concern but it might be worth trying to shuffle some of this around to improve the readability of Start()? Particularly here and in the large startup closure above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a look at this, and not against it, but I think I'm going to punt for the time being. I kind of like having all the logic Start (and the majority of the stop) logic in one place because you can scan it all at once without having to jump around. Also lets us reduce the number of struct variables and helpers in favor of definitions that can't leak into other contexts, and are defined more visibly, closer to home.

client_test.go Outdated
Comment on lines 2577 to 2578
// ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
// t.Cleanup(cancel)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mistakenly commented?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops yep. Actually we should talk about this, but the reason I had it commented is that in case your program is actually stuck, the canceled context supersedes go test's -timeout option, so instead of -timeout stopping the test of a full stack trace of every goroutine that was still running, you end up with a canceled test with no debugging information.

So a long way of saying that tighter -timeout parameters are a better option than context timeouts when it comes to use in tests, but in the interest of not getting into that now, I just uncommented these lines.

}

// StopInit provides a way to build a more customized Stop implementation. It
// should be avoided unless there'a an exception reason not to because Stop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// should be avoided unless there'a an exception reason not to because Stop
// should be avoided unless there's an exceptional reason not to because Stop

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks.

@brandur brandur force-pushed the brandur-fast-completer branch 3 times, most recently from e5d4268 to 71abf47 Compare March 17, 2024 20:22
Base automatically changed from brandur-fast-completer to master March 17, 2024 20:25
@brandur brandur force-pushed the brandur-client-start-stop-2 branch 3 times, most recently from 4091846 to 54fed4c Compare March 17, 2024 20:34
Here, continue service refactoring to its ultimate conclusion, and make
the client itself a start/stop service, having a couple advantages:

* (IMO) considerably simplifies the start and stop code, putting it all
  in one place, and largely even one function. Fewer helpers to follow
  around to understand what's going on.

* Makes the client behave gracefully on double starts/stops.

* Allows the client to easily handle start/stop under duress. Any number
  of goroutines can be starting or stopping it simultaneously and it's
  guaranteed free from races.

Because the client's stop functions are different from the signature of
other start/stop services (taking a context and returning an error), I
had to modify the base start/stop service somewhat and allow for an
additional `StopInit` helper (like `StartInit`) that lets stop behavior
be customized while still providing race-free operations.
@brandur brandur force-pushed the brandur-client-start-stop-2 branch from 54fed4c to bed599c Compare March 17, 2024 20:37
@brandur
Copy link
Contributor Author

brandur commented Mar 17, 2024

ty.

@brandur brandur merged commit a939d34 into master Mar 17, 2024
10 checks passed
@brandur brandur deleted the brandur-client-start-stop-2 branch March 17, 2024 20:44
bgentry added a commit that referenced this pull request May 30, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request May 30, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request May 30, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request May 31, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 3, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 4, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 9, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 9, 2024
* fix StopAndTest after ongoing Stop

This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.

* refactor StopAndCancel tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants