Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# SDK Developer Documentation

This file documents developing the SDK itself. If you simply wish to use the SDK or run examples, see [README.md](./README.md)

## Setup

- Install JDK 17
- Recommended to use SDK Man: https://sdkman.io/ and `sdk use java 17.0.16-tem`
- Ensure you can run all tests and checks: `./gradlew check build`
- IDE Setup
- Intellij Community
- Ubuntu: `sudo snap install intellij-idea-community`
- Other: https://www.jetbrains.com/idea/download/
- (Optional) Install pre-commit hooks: `./gradlew installGitHooks`
- These hooks automatically run common checks for you but CI also runs the same checks before merging to the main branch is allowed
- NOTE: this will overwrite existing hooks. Take backups before running

## Running a local OpenTelemetry collector

OpenTelemetry provides a local collector with a debug exporter which logs all traces, logs, and metrics to stdout.

To run a local collector:

```
# Assumes you're in the repo root
docker run --rm -p 4318:4318 -v "$PWD/localcollector/collector.yaml:/etc/otelcol/config.yaml" otel/opentelemetry-collector:0.136.0 # latest release will probably also work
```

To send Braintrust otel data to the local collector:

```
# assumes you have BRAINTRUST_API_KEY and OPENAI_API_KEY exported
export BRAINTRUST_API_URL="http://localhost:4318" ; export BRAINTRUST_TRACES_PATH="/v1/traces"; export BRAINTRUST_LOGS_PATH="/v1/logs" ; ./gradlew :examples:runOpenAIInstrumentation
```
35 changes: 0 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,38 +51,3 @@ The SDK uses a standard slf4j logger and will use the default log level (or not
All Braintrust loggers will log into the `dev.braintrust` namespace. To adjust the log level, consult your logger documentation.

For example, to enable debug logging for slf4j-simple you would set the system property `org.slf4j.simpleLogger.log.dev.braintrust=DEBUG`

# SDK Developer Docs

The remaining sections document developing the SDK itself. Nothing below is required if you simply wish to use the SDK or run examples.

## Setup

- Install JDK 17
- Recommended to use SDK Man: https://sdkman.io/ and `sdk use java 17.0.16-tem`
- Ensure you can run all tests and checks: `./gradlew check build`
- IDE Setup
- Intellij Community
- Ubuntu: `sudo snap install intellij-idea-community`
- Other: https://www.jetbrains.com/idea/download/
- (Optional) Install pre-commit hooks: `./gradlew installGitHooks`
- These hooks automatically run common checks for you but CI also runs the same checks before merging to the main branch is allowed
- NOTE: this will overwrite existing hooks. Take backups before running

## Running a local OpenTelemetry collector

OpenTelemetry provides a local collector with a debug exporter which logs all traces, logs, and metrics to stdout.

To run a local collector:

```
# Assumes you're in the repo root
docker run --rm -p 4318:4318 -v "$PWD/localcollector/collector.yaml:/etc/otelcol/config.yaml" otel/opentelemetry-collector:0.136.0 # latest release will probably also work
```

To send Braintrust otel data to the local collector:

```
# assumes you have BRAINTRUST_API_KEY and OPENAI_API_KEY exported
export BRAINTRUST_API_URL="http://localhost:4318" ; export BRAINTRUST_TRACES_PATH="/v1/traces"; export BRAINTRUST_LOGS_PATH="/v1/logs" ; ./gradlew :examples:runOpenAIInstrumentation
```
3 changes: 1 addition & 2 deletions src/main/java/dev/braintrust/eval/Eval.java
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import com.fasterxml.jackson.databind.ObjectMapper;
import dev.braintrust.api.BraintrustApiClient;
import dev.braintrust.config.BraintrustConfig;
import dev.braintrust.spec.SdkSpec;
import dev.braintrust.trace.BraintrustContext;
import dev.braintrust.trace.BraintrustTracing;
import io.opentelemetry.api.common.AttributeKey;
Expand All @@ -26,7 +25,7 @@
*/
public final class Eval<INPUT, OUTPUT> {
private static final AttributeKey<String> PARENT =
AttributeKey.stringKey(SdkSpec.Attributes.PARENT);
AttributeKey.stringKey(BraintrustTracing.PARENT_KEY);
private static final ObjectMapper JSON_MAPPER =
new com.fasterxml.jackson.databind.ObjectMapper();
private final @Nonnull String experimentName;
Expand Down
12 changes: 0 additions & 12 deletions src/main/java/dev/braintrust/spec/SdkSpec.java

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package dev.braintrust.trace;

import dev.braintrust.config.BraintrustConfig;
import dev.braintrust.spec.SdkSpec;
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.context.Context;
import io.opentelemetry.sdk.common.CompletableResultCode;
Expand All @@ -21,10 +20,9 @@
*/
@Slf4j
class BraintrustSpanProcessor implements SpanProcessor {

// Braintrust-specific attributes
public static final AttributeKey<String> PARENT =
AttributeKey.stringKey(SdkSpec.Attributes.PARENT);
AttributeKey.stringKey(BraintrustTracing.PARENT_KEY);

private final BraintrustConfig config;
private final SpanProcessor delegate;
Expand Down
1 change: 1 addition & 0 deletions src/main/java/dev/braintrust/trace/BraintrustTracing.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
*/
@Slf4j
public final class BraintrustTracing {
public static final String PARENT_KEY = "braintrust.parent";
static final String OTEL_SERVICE_NAME = "braintrust-app";
static final String INSTRUMENTATION_NAME = "braintrust-java";
static final String INSTRUMENTATION_VERSION = loadVersionFromProperties();
Expand Down
3 changes: 1 addition & 2 deletions src/test/java/dev/braintrust/eval/EvalTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@

import dev.braintrust.api.BraintrustApiClient;
import dev.braintrust.config.BraintrustConfig;
import dev.braintrust.spec.SdkSpec;
import dev.braintrust.trace.BraintrustTracing;
import dev.braintrust.trace.BraintrustTracingTest;
import io.opentelemetry.api.GlobalOpenTelemetry;
Expand Down Expand Up @@ -74,7 +73,7 @@ public void evalOtelTraceWithProperAttributes() {
span -> {
var parent =
span.getAttributes()
.get(AttributeKey.stringKey(SdkSpec.Attributes.PARENT));
.get(AttributeKey.stringKey(BraintrustTracing.PARENT_KEY));
assertEquals(
"experiment_id:" + experiment.id(),
parent,
Expand Down