support the context package

[kevinburke]

It would be nice if go-github supported the context package, so you could coordinate cancelation across many different goroutines.

It's now available in the standard library as https://golang.org/pkg/context/.

[dmitshur]

In theory, you can do it, but not directly. You'd have to use Client.NewRequest to create a new *http.Request, then set the context of that HTTP request via Request.WithContext, and finally call github.Client.Do.

However, it's so roundabout that it's probably not even worth considering.

To resolve this fully, we'd have to break the API and add ctx context.Context as first parameter to all methods.

---

To get an idea of what it'd look like, see github.com/tambet/go-asana/asana, a similar package that is the client for Asana API, where context support was added (in tambet/go-asana#4).

---

I'd be in favor of doing that, since it's only way to move forward. People who vendor this library can update on their own time. We would bump libraryVersion.

However, this decision is largely in the hands of @willnorris and @gmlewis.

[dmitshur]

I'd like to work on this, if we can get approval.

Some planning thoughts on how it would be done.

The change would be largely mechanical, once we decide on how to make the change. It would affect all endpoint methods by adding ctx context.Context as the first parameter. For example:

// Get a single issue.
//
// GitHub API docs: http://developer.github.com/v3/issues/#get-a-single-issue
func (s *IssuesService) Get(owner string, repo string, number int) (*Issue, *Response, error) { ... }

Would become:

// Get a single issue.
//
// GitHub API docs: http://developer.github.com/v3/issues/#get-a-single-issue
func (s *IssuesService) Get(ctx context.Context, owner string, repo string, number int) (*Issue, *Response, error) { ... }

The inside of a typical method looks like this:

func (s *IssuesService) Get(owner string, repo string, number int) (*Issue, *Response, error) {
	u := fmt.Sprintf("repos/%v/%v/issues/%d", owner, repo, number)
	req, err := s.client.NewRequest("GET", u, nil)
	if err != nil {
		return nil, nil, err
	}

	// TODO: remove custom Accept header when this API fully launches.
	req.Header.Set("Accept", mediaTypeReactionsPreview)

	issue := new(Issue)
	resp, err := s.client.Do(req, issue)
	if err != nil {
		return nil, resp, err
	}

	return issue, resp, err
}

There are 2 places where we could set the context of the request. Either inside NewRequest, or inside Do.

After thinking about it, I think it should be done inside Do. It's more appropriate. A request can be created in advance of it being "done", and context is meant to be request scoped. So it's better to set it later rather than earlier. It's the approach that the golang.org/x/net/context/ctxhttp does and I think it's a good example. See here specifically, note that Get creates the *http.Request but doesn't set a context on it, it passes ctx to Do and Do sets the context on the request. This is the conclusion I came to after thinking a bit about it, but I welcome feedback.

If that's what we do, it would look like this:

func (s *IssuesService) Get(ctx context.Context, owner string, repo string, number int) (*Issue, *Response, error) {
	u := fmt.Sprintf("repos/%v/%v/issues/%d", owner, repo, number)
	req, err := s.client.NewRequest("GET", u, nil)
	if err != nil {
		return nil, nil, err
	}

	// TODO: remove custom Accept header when this API fully launches.
	req.Header.Set("Accept", mediaTypeReactionsPreview)

	issue := new(Issue)
	resp, err := s.client.Do(ctx, req, issue)
	if err != nil {
		return nil, resp, err
	}

	return issue, resp, err
}

// Do sends an API request and returns the API response.  The API response is
// JSON decoded and stored in the value pointed to by v, or returned as an
// error if an API error has occurred.  If v implements the io.Writer
// interface, the raw response body will be written to v, without attempting to
// first decode it.  If rate limit is exceeded and reset time is in the future,
// Do returns *RateLimitError immediately without making a network API call.
func (c *Client) Do(ctx context.Context, req *http.Request, v interface{}) (*Response, error) {
	rateLimitCategory := category(req.URL.Path)

	// If we've hit rate limit, don't make further requests before Reset time.
	if err := c.checkRateLimitBeforeDo(req, rateLimitCategory); err != nil {
		return nil, err
	}

	resp, err := c.client.Do(req.WithContext(ctx))
	if err != nil {
		// handle err
	}

	...

One potentially big problem is dealing with older versions of Go. 1.7 has context in standard library, but 1.6 and older would need to use golang.org/x/net/context package. That means we couldn't have the same context.Context in signature of each method without duplicating every file, once for 1.7+ importing context, and once for older importing golang.org/x/net/context.

We could use golang.org/x/net/context everywhere for now, since that package would be compatible with older Go versions.

For 1.6 and older, we'd either have to do what golang.org/x/net/context/ctxhttp does as far as using ctx on the HTTP request (since net/http of Go 1.6 doesn't have built-in support), or ignore ctx altogether. Best solution might be to simply use golang.org/x/net/context/ctxhttp package.

Anyway, that's enough planning for now until we hear back from @willnorris and @gmlewis.

[kevinburke]

I'm not sure 1.6 can handle it at all with the golang.org/x/ import or without because the http.Request object has no WithContext method in 1.6.

FWIW, here's how I handle this in my twilio-go library - I have an underlying rest client library that has NewRequest(method, path, body), and Do(request, v interface{}) where Do parses the response as JSON and decodes into the interface.

The twilio-go library has a function that calls rest.NewRequest, adds a context, then calls Do.

https://github.com/kevinburke/twilio-go/blob/master/http.go#L268-L291
https://github.com/kevinburke/twilio-go/blob/master/http_noctx.go
https://github.com/kevinburke/rest/blob/master/client.go#L81-L155

[dmitshur]

I'm not sure 1.6 can handle it at all with the golang.org/x/ import or without because the http.Request object has no WithContext method in 1.6.

It can be done by using http.Request.Cancel channel (which has existed for a while), it's just a lot of code. See how golang.org/x/net/context/ctxhttp package does it.

FWIW, here's how I handle this in my twilio-go library

Got it, thanks for a data point. 👍

[willnorris]

I'm not opposed to this change.

I'm also not opposed to just making the library require go1.7, though I'd probably want to wait until go1.8 is released so then we support "current version, plus one back". We'd document that if you need support for <1.7, then use X previous version.

Let's maybe start this work in a separate "context" branch?

[dmitshur]

I was thinking about that option too, and I really like it. Let's wait for 1.8 to come out (a matter of weeks), then we can update to use context package and require 1.7+ (which will be current plus previous).

We can make a branch or a tag with the last version that's compatible with Go 1.6 and older, but anyone who needs latest GitHub API features is likely to be using a modern version of Go as well.

Let's maybe start this work in a separate "context" branch?

Since this is a largely mechanical change, after deciding on the strategy, it seems unhelpful to start doing everything now since there are likely going to be changes before then (so it'd be more work in total). I'd be fine with making the decisions now, but starting implementation on a branch once 1.8 comes out.

We can make the change for one endpoint just to discuss how it looks. I'll make that PR.

[dmitshur]

Going with 1.7+ only gives us another option I haven't considered before.

We can either modify Do to accept ctx and set it internally, as I mentioned before:

@@ -396,7 +397,7 @@ func (c *Client) Rate() Rate {
 // interface, the raw response body will be written to v, without attempting to
 // first decode it.  If rate limit is exceeded and reset time is in the future,
 // Do returns *RateLimitError immediately without making a network API call.
-func (c *Client) Do(req *http.Request, v interface{}) (*Response, error) {
+func (c *Client) Do(ctx context.Context, req *http.Request, v interface{}) (*Response, error) {
        rateLimitCategory := category(req.URL.Path)
 
        // If we've hit rate limit, don't make further requests before Reset time.
@@ -404,7 +405,7 @@ func (c *Client) Do(req *http.Request, v interface{}) (*Response, error) {
                return nil, err
        }
 
-       resp, err := c.client.Do(req)
+       resp, err := c.client.Do(req.WithContext(ctx))
        if err != nil {
                if e, ok := err.(*url.Error); ok {
                        if url, err := url.Parse(e.URL); err == nil {

And call it like this inside endpoint methods:

@@ -220,7 +221,7 @@ func (s *IssuesService) ListByRepo(owner string, repo string, opt *IssueListByRe
 // Get a single issue.
 //
 // GitHub API docs: http://developer.github.com/v3/issues/#get-a-single-issue
-func (s *IssuesService) Get(owner string, repo string, number int) (*Issue, *Response, error) {
+func (s *IssuesService) Get(ctx context.Context, owner string, repo string, number int) (*Issue, *Response, error) {
        u := fmt.Sprintf("repos/%v/%v/issues/%d", owner, repo, number)
        req, err := s.client.NewRequest("GET", u, nil)
        if err != nil {
@@ -231,7 +232,7 @@ func (s *IssuesService) Get(owner string, repo string, number int) (*Issue, *Res
        req.Header.Set("Accept", mediaTypeReactionsPreview)
 
        issue := new(Issue)
-       resp, err := s.client.Do(req, issue)
+       resp, err := s.client.Do(ctx, req, issue)
        if err != nil {
                return nil, resp, err
        }

Or, we can keep Do unmodified and instead add the context to the request inside each endpoint method. E.g.:

@@ -220,7 +221,7 @@ func (s *IssuesService) ListByRepo(owner string, repo string, opt *IssueListByRe
 // Get a single issue.
 //
 // GitHub API docs: http://developer.github.com/v3/issues/#get-a-single-issue
-func (s *IssuesService) Get(owner string, repo string, number int) (*Issue, *Response, error) {
+func (s *IssuesService) Get(ctx context.Context, owner string, repo string, number int) (*Issue, *Response, error) {
        u := fmt.Sprintf("repos/%v/%v/issues/%d", owner, repo, number)
        req, err := s.client.NewRequest("GET", u, nil)
        if err != nil {
@@ -231,7 +232,7 @@ func (s *IssuesService) Get(owner string, repo string, number int) (*Issue, *Res
        req.Header.Set("Accept", mediaTypeReactionsPreview)
 
        issue := new(Issue)
-       resp, err := s.client.Do(req, issue)
+       resp, err := s.client.Do(req.WithContext(ctx), issue)
        if err != nil {
                return nil, resp, err
        }

I honestly don't know which is better. Any preferences?

I'm leaning towards modifying Do to accept ctx parameter.

[dmitshur]

Made #529, feel free to look and review the approach.