Skip to content

Conversation

@mshields822
Copy link
Contributor

@mshields822 mshields822 commented Apr 19, 2016

Continuing from #120

  • Move PubsubClient and friends out of sdk.io and into sdk.util.
  • Add PubsubApiaryClient since gRPC has onerous boot class path requirements which I don't wish to inflict upon other runners.
  • Add PubsubTestClient in preparation for unit testing PubsubUnbounded{Source,Sink}.
  • Unit tests for all of above.

@mshields822
Copy link
Contributor Author

R: @dhalperi

swegner pushed a commit to swegner/beam that referenced this pull request Apr 22, 2016
Wrap Exceptions thrown in StartBundle in the InProcessPipelineRunner
@mshields822
Copy link
Contributor Author

I'm guessing this one fell off your radar @dhalperi.
I've since replaced the TransportType enum with a factory and added a PubsubTestClient.
Do you want to see that here or a future pr?

LOG.error("Failed to delete subscription: ", e);
}
if (finallyBlockException != null) {
Throwables.propagate(finallyBlockException);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kennknowles fyi

I think the current recommendation is simply:

throw new RuntimeException("Some message", finallyBlockException);

Even if you intended to use Throwables.propagate, you should use throw Throwables.propagate... to work around some IDEs/cases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, see #70.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@dhalperi
Copy link
Contributor

May as well dump it here.

To bootstrap review, can you please update the PR message with everything that's in the diff, assuming I didn't look at your original #120 and discussion with Kenn?

options.getGcpCredential(),
// Do not log 404. It clutters the output and is possibly even required by the caller.
new RetryHttpRequestInitializer(ImmutableList.of(404))))
.setRootUrl(options.getPubsubRootUrl())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you need to do any pubsub-specific failure handling here? 410, 403 rate limited, any of the crap we see in other APIs?

Copy link
Contributor Author

@mshields822 mshields822 May 2, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is straight from the impl we already have in Transport. It has been load tested, used both from vms and desktop, and seems to do the business.

Note that I'd like to remove Transport.newPubsubClient but it is used from various examples so leaving it in place for a future refactor. I at least marked ti deprecated.

@mshields822 mshields822 changed the title [BEAM-53] Pubsub Apiary client [BEAM-53] PubsubApiaryClient, PubsubTestClient May 2, 2016
@mshields822
Copy link
Contributor Author

PTAL @dhalperi
Not much to copy over from #120 discussion. Mostly a few glitches, double check jar deps, and move out of io into util.
However we've since split io into various sub-packages, so let me know if there's a better home.

@mshields822
Copy link
Contributor Author

(off for root canal. forgive any grumpiness later today...)


public String getV1Beta1Path() {
String[] splits = path.split("/");
Preconditions.checkState(splits.length == 4);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, would strongly prefer to have better error messages at all of these validation checks. For example,

checkState(splits.length == 4, "Expected 4 path components in Pubsub path %s", path);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@dhalperi
Copy link
Contributor

dhalperi commented May 3, 2016

LGTM modulo a pending rebase and few minor (optional) comments.

// Otherwise we'll fall back to the topic's project.
// Note that they don't need to be the same.
String project = c.getPipelineOptions().as(PubsubOptions.class).getProject();
if (project == null || project.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isNullOrEmpty?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@mshields822
Copy link
Contributor Author

PTAL
I think this covers everything.
I'll be occupied most of Wed so if there's only a small wibble holding this back please feel free to do it in the merge.
Could do with a good squash too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants