Skip to content

Commit 909261e

Browse files
watsonwatson
authored
chore: add add-new-instrumentation agent skill (#7564)
chore: add add-new-instrumentation agent skill Adds a shared skill guide for creating new dd-trace instrumentations and plugins. The skill lives under `.agents/skills/` and is symlinked into both `.claude/skills/` and `.cursor/skills/` so it's available to all AI agents. Updates `.gitignore` to track the skill files while continuing to exclude other Claude/Cursor settings. Address review comments Co-authored-by: thomas.watson <thomas.watson@datadoghq.com>
1 parent 4ae024c commit 909261e

4 files changed

Lines changed: 373 additions & 2 deletions

File tree

‎.agents/skills/add-new-instrumentation/SKILL.md‎

Lines changed: 369 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,369 @@
1+
---
2+
name: add-new-instrumentation
3+
description: Guide for adding new instrumentation and plugins to dd-trace. Use when creating a new plugin, adding instrumentation for a third-party library, or when the user asks about adding new instrumentations or plugins.
4+
---
5+
6+
# Adding New Instrumentation
7+
8+
## Architecture Overview
9+
10+
The instrumentation system has two layers that communicate via Node.js diagnostic channels:
11+
12+
1. **Instrumentation** (`packages/datadog-instrumentations/src/<name>.js`) — hooks into third-party library internals using `addHook()` and `shimmer`, then publishes events to named diagnostic channels.
13+
2. **Plugin** (`packages/datadog-plugin-<name>/src/index.js`) — subscribes to those channels to implement APM tracing logic (spans, metadata, errors).
14+
15+
This separation means you almost always need to create **both** files.
16+
17+
## Step 1: Create the Instrumentation File
18+
19+
Create `packages/datadog-instrumentations/src/<name>.js`. The following is a starting-point template — adapt the wrapped method(s), context fields, and channel operations to match the actual library's API. Read 1-2 existing instrumentations for the library type you're adding (e.g. `kafkajs.js` for messaging, `redis.js` for caching) before writing yours.
20+
21+
```javascript
22+
'use strict'
23+
24+
const { channel, addHook } = require('./helpers/instrument')
25+
const shimmer = require('../../../datadog-shimmer')
26+
27+
// Channel naming convention: apm:<name>:<operation>:<event>
28+
// Events: start, finish, error, async-start, async-finish
29+
const startCh = channel('apm:<name>:<operation>:start')
30+
const finishCh = channel('apm:<name>:<operation>:finish')
31+
const errorCh = channel('apm:<name>:<operation>:error')
32+
33+
addHook({ name: '<module-name>', versions: ['>=1.0'] }, (moduleExports) => {
34+
shimmer.wrap(moduleExports, 'methodToWrap', function (original) {
35+
return function wrappedMethod (...args) {
36+
if (!startCh.hasSubscribers) {
37+
return original.apply(this, args)
38+
}
39+
40+
const ctx = { /* relevant context */ }
41+
return startCh.runStores(ctx, () => {
42+
try {
43+
const result = original.apply(this, args)
44+
finishCh.publish(ctx)
45+
return result
46+
} catch (err) {
47+
ctx.error = err
48+
errorCh.publish(ctx)
49+
throw err
50+
}
51+
})
52+
}
53+
})
54+
return moduleExports
55+
})
56+
```
57+
58+
**Key patterns:**
59+
- Always guard with `if (!startCh.hasSubscribers)` for performance — skip instrumentation if no plugin is listening
60+
- Use `startCh.runStores(ctx, () => {...})` to propagate async context
61+
- Use `shimmer.wrap()` to patch methods non-destructively
62+
- The `versions` array is a semver range; check existing instrumentations for precedents
63+
- For multiple files in a package: use `file: 'path/within/package.js'` in `addHook`
64+
- For multiple module names mapping to the same hooks: call `addHook` multiple times
65+
66+
## Step 2: Create the Plugin Directory and File
67+
68+
```bash
69+
mkdir -p packages/datadog-plugin-<name>/{src,test}
70+
```
71+
72+
### Choosing the Right Base Class
73+
74+
| Scenario | Base Class | Import Path |
75+
|---|---|---|
76+
| Creating trace spans for a single operation type | `TracingPlugin` | `../../dd-trace/src/plugins/tracing` |
77+
| Wrapping an outbound client call (HTTP, gRPC, DB) | `OutboundPlugin` extends `TracingPlugin` | `../../dd-trace/src/plugins/outbound` |
78+
| Wrapping an inbound server/consumer call | `InboundPlugin` extends `TracingPlugin` | `../../dd-trace/src/plugins/inbound` |
79+
| Key-value cache client (Redis, Memcached) | `CachePlugin` extends `TracingPlugin` | `../../dd-trace/src/plugins/cache` |
80+
| Multiple sub-concerns (producer + consumer, or tracing + code-origin) | `CompositePlugin` | `../../dd-trace/src/plugins/composite` |
81+
| Non-tracing feature only | `Plugin` | `../../dd-trace/src/plugins/plugin` |
82+
83+
### Template: Simple TracingPlugin
84+
85+
```javascript
86+
'use strict'
87+
88+
const TracingPlugin = require('../../dd-trace/src/plugins/tracing')
89+
90+
class MyPlugin extends TracingPlugin {
91+
static id = '<name>' // must match module name
92+
static operation = '<operation>' // e.g., 'query', 'send', 'request'
93+
static system = '<system>' // e.g., 'redis', 'kafka' (used for peer.service)
94+
95+
bindStart (ctx) {
96+
const { relevantField } = ctx
97+
98+
this.startSpan({
99+
resource: relevantField,
100+
service: this.serviceName(),
101+
meta: {
102+
'some.tag': relevantField
103+
}
104+
}, ctx)
105+
}
106+
107+
bindFinish (ctx) {
108+
this.finish()
109+
}
110+
111+
bindError (ctx) {
112+
this.finish(ctx.error)
113+
}
114+
}
115+
116+
module.exports = MyPlugin
117+
```
118+
119+
### Template: CompositePlugin
120+
121+
```javascript
122+
'use strict'
123+
124+
const CompositePlugin = require('../../dd-trace/src/plugins/composite')
125+
const ProducerPlugin = require('./producer')
126+
const ConsumerPlugin = require('./consumer')
127+
128+
class MyPlugin extends CompositePlugin {
129+
static id = '<name>'
130+
131+
static get plugins () {
132+
return {
133+
producer: ProducerPlugin,
134+
consumer: ConsumerPlugin
135+
}
136+
}
137+
}
138+
139+
module.exports = MyPlugin
140+
```
141+
142+
For composite plugins, create separate files in `src/` for each sub-plugin (e.g., `src/producer.js`, `src/consumer.js`).
143+
144+
## Step 3: Register the Plugin
145+
146+
Add an entry to `packages/dd-trace/src/plugins/index.js`:
147+
148+
```javascript
149+
// Inside the plugins object:
150+
get '<module-name>' () { return require('../../../datadog-plugin-<name>/src') },
151+
```
152+
153+
If multiple npm package names map to the same plugin (e.g., `redis` and `@redis/client`), add one getter per name.
154+
155+
## Step 4: Add TypeScript Definitions
156+
157+
In `index.d.ts`, add to the `plugins` namespace:
158+
159+
```typescript
160+
// In the Plugins interface:
161+
'<name>': plugins.<name>;
162+
163+
// Add a plugin interface (in alphabetical order with other plugin interfaces):
164+
interface <name> extends Instrumentation {}
165+
// Or with config options:
166+
interface <name> extends Instrumentation {
167+
optionName?: string | boolean;
168+
}
169+
```
170+
171+
## Step 5: Update docs/test.ts
172+
173+
Add a type-check call in `docs/test.ts`:
174+
175+
```typescript
176+
tracer.use('<name>');
177+
// Or with options:
178+
tracer.use('<name>', { optionName: 'value' });
179+
```
180+
181+
## Step 6: Document in docs/API.md
182+
183+
Add a section in `docs/API.md` (alphabetically ordered):
184+
185+
```markdown
186+
<h5 id="<name>"><h5>
187+
188+
This plugin automatically patches the [<LibraryName>](<url>) module.
189+
190+
| Option | Default | Description |
191+
|--------|---------|-------------|
192+
| `service` | | Service name override. |
193+
```
194+
195+
## Step 7: Add to CI Workflow
196+
197+
Add a job to `.github/workflows/apm-integrations.yml`:
198+
199+
```yaml
200+
<name>:
201+
runs-on: ubuntu-latest
202+
env:
203+
PLUGINS: <name>
204+
# SERVICES: <docker-service> # if external services needed
205+
steps:
206+
- uses: actions/checkout@v4
207+
- uses: ./.github/actions/testagent/start
208+
- uses: ./.github/actions/node
209+
with:
210+
version: ${{ matrix.node-version }}
211+
- uses: ./.github/actions/install
212+
- run: yarn test:plugins:ci
213+
strategy:
214+
matrix:
215+
node-version: [18, 22]
216+
```
217+
218+
Check `.github/workflows/apm-integrations.yml` for the exact current step format used by other plugins.
219+
220+
## Step 8: Write Tests
221+
222+
### Unit Tests
223+
224+
Create `packages/datadog-plugin-<name>/test/index.spec.js`:
225+
226+
```javascript
227+
'use strict'
228+
229+
const agent = require('../../dd-trace/test/plugins/agent')
230+
const { withVersions } = require('../../dd-trace/test/setup/mocha')
231+
232+
describe('Plugin', () => {
233+
describe('<name>', () => {
234+
withVersions('<name>', '<module-name>', (version) => {
235+
let myLib
236+
237+
beforeEach(() => {
238+
return agent.load('<name>')
239+
})
240+
241+
beforeEach(() => {
242+
myLib = require(`../../../versions/<module-name>@${version}`)
243+
})
244+
245+
afterEach(() => {
246+
return agent.close({ ritmReset: false })
247+
})
248+
249+
it('should create a span', (done) => {
250+
agent.use(traces => {
251+
const span = traces[0][0]
252+
expect(span.name).to.equal('<name>.<operation>')
253+
expect(span.service).to.equal('test-<name>')
254+
}).then(done, done)
255+
256+
// trigger the instrumented operation
257+
})
258+
})
259+
})
260+
})
261+
```
262+
263+
**Key test helpers:**
264+
- `withVersions(pluginName, moduleName, cb)` — runs tests across installed versions
265+
- `agent.load(pluginName)` — starts a test agent and loads the plugin
266+
- `agent.close({ ritmReset: false })` — tears down (use `ritmReset: false` to preserve require cache)
267+
- `agent.use(traces => { ... })` — asserts on captured traces
268+
- `withNamingSchema(agent, ...)` — tests naming schema conventions
269+
- `withPeerService(agent, ...)` — tests peer service tag
270+
271+
### ESM Integration Tests
272+
273+
ESM tests verify the plugin works with native ES module imports. They live in `packages/datadog-plugin-<name>/test/integration-test/` and use a `FakeAgent` to assert on captured spans.
274+
275+
Create `packages/datadog-plugin-<name>/test/integration-test/server.mjs` — a minimal ESM script that initialises the tracer and triggers the instrumented operation:
276+
277+
```javascript
278+
import 'dd-trace/init.js'
279+
import myLib from '<module-name>'
280+
281+
// trigger the instrumented operation
282+
await myLib.someOperation()
283+
```
284+
285+
Create `packages/datadog-plugin-<name>/test/integration-test/client.spec.js` — the test that spawns the ESM server and asserts spans arrive:
286+
287+
```javascript
288+
'use strict'
289+
290+
const assert = require('node:assert/strict')
291+
292+
const {
293+
FakeAgent,
294+
sandboxCwd,
295+
useSandbox,
296+
checkSpansForServiceName,
297+
spawnPluginIntegrationTestProcAndExpectExit,
298+
varySandbox,
299+
} = require('../../../../integration-tests/helpers')
300+
const { withVersions } = require('../../../dd-trace/test/setup/mocha')
301+
302+
describe('esm', () => {
303+
let agent
304+
let proc
305+
let variants
306+
307+
withVersions('<name>', '<module-name>', version => {
308+
useSandbox([`'<module-name>@${version}'`], false, [
309+
'./packages/datadog-plugin-<name>/test/integration-test/*'])
310+
311+
beforeEach(async () => {
312+
agent = await new FakeAgent().start()
313+
})
314+
315+
before(async function () {
316+
variants = varySandbox('server.mjs', '<module-name>', '<namedExport>')
317+
})
318+
319+
afterEach(async () => {
320+
proc && proc.kill()
321+
await agent.stop()
322+
})
323+
324+
for (const variant of varySandbox.VARIANTS) {
325+
it(`is instrumented ${variant}`, async () => {
326+
const res = agent.assertMessageReceived(({ headers, payload }) => {
327+
assert.strictEqual(headers.host, `127.0.0.1:${agent.port}`)
328+
assert.ok(Array.isArray(payload))
329+
assert.strictEqual(checkSpansForServiceName(payload, '<name>.<operation>'), true)
330+
})
331+
332+
proc = await spawnPluginIntegrationTestProcAndExpectExit(sandboxCwd(), variants[variant], agent.port)
333+
334+
await res
335+
}).timeout(20000)
336+
}
337+
})
338+
})
339+
```
340+
341+
**Key points for ESM tests:**
342+
- `varySandbox('server.mjs', bindingName, namedExport)` generates three import-style variants (`default`, `star`, `destructure`) from `server.mjs` so the instrumentation is verified under all ESM import patterns.
343+
- `varySandbox.VARIANTS` is `['default', 'star', 'destructure']`.
344+
- Pass `byPassDefault: true` as the fifth argument to `varySandbox` when the module has no default export (named-only packages).
345+
- `useSandbox` installs the package versions into a temp sandbox dir; the second argument controls whether it runs `yarn install` inside the sandbox.
346+
- `spawnPluginIntegrationTestProcAndExpectExit` spawns `node <script>` with `DD_TRACE_AGENT_PORT` set to the `FakeAgent` port.
347+
- Each `it` must have a generous timeout (e.g. `20000`) because sandbox setup and process spawning take time.
348+
349+
## Running Tests
350+
351+
```bash
352+
# Run the unit plugin test
353+
./node_modules/.bin/mocha packages/datadog-plugin-<name>/test/index.spec.js
354+
355+
# Or via the test:plugins script (unit tests only)
356+
PLUGINS="<name>" npm run test:plugins
357+
358+
# Run the ESM integration tests
359+
PLUGINS="<name>" npm run test:integration:plugins
360+
```
361+
362+
## Reference Files
363+
364+
- Instrumentation helpers: `packages/datadog-instrumentations/src/helpers/instrument.js`
365+
- Plugin registration: `packages/dd-trace/src/plugins/index.js`
366+
- Example simple plugin: `packages/datadog-plugin-redis/src/`
367+
- Example composite plugin: `packages/datadog-plugin-kafkajs/src/`
368+
- Example instrumentation: `packages/datadog-instrumentations/src/kafkajs.js`
369+
- Example instrumentation: `packages/datadog-instrumentations/src/redis.js`

‎.claude/skills/add-new-instrumentation‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../.agents/skills/add-new-instrumentation

‎.cursor/skills/add-new-instrumentation‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../.agents/skills/add-new-instrumentation

‎.gitignore‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,6 @@ typings/
9898
!.vscode/launch.json
9999
!.vscode/extensions.json
100100
.history
101-
.cursor/skills
102101

103102
# IntelliJ
104103
.idea
@@ -142,7 +141,8 @@ __queuestorage__/AzuriteConfig
142141
.analysis/
143142

144143
# ignore claude settings
145-
.claude/
144+
.claude/*
145+
!.claude/skills/
146146

147147
# Husky generates a helper dir under .husky/_ (including husky.sh). Don't commit it.
148148
.husky/_/

0 commit comments

Comments
 (0)