n8n/\agentuity startup scraper workflow example #24
Conversation
WalkthroughThis change introduces a new AI agent project, "Startup_News_Scraper," including its configuration, documentation, and source code. The project fetches startup-related news from multiple RSS feeds, summarizes articles using OpenAI's API, and provides structured summaries. The update includes TypeScript types, agent logic, configuration files, and development tooling setup. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Agent (Startup_News_Scraper)
participant fetchFeeds.ts
participant summarize.ts
participant OpenAI API
User->>Agent (Startup_News_Scraper): POST request (optional perFeed)
Agent (Startup_News_Scraper)->>fetchFeeds.ts: getStartupLinks(perFeed)
fetchFeeds.ts->>RSS Feeds: Fetch and parse feeds
fetchFeeds.ts-->>Agent (Startup_News_Scraper): Filtered startup links
Agent (Startup_News_Scraper)->>summarize.ts: summarizeStartups(links)
summarize.ts->>OpenAI API: Summarize each article (batch)
OpenAI API-->>summarize.ts: JSON summaries
summarize.ts-->>Agent (Startup_News_Scraper): Array of StartupSummary
Agent (Startup_News_Scraper)-->>User: JSON response with summaries
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
✨ Finishing Touches
🧪 Generate Unit Tests
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Nitpick comments (15)
agents/Startup_News_Scraper/biome.json (1)
17-23: Add TypeScript formatting rules.
You currently only configure JavaScript formatting. Consider adding atypescriptsection to enforce consistent quotes, trailing commas, and semicolons in.tsfiles as well.agents/Startup_News_Scraper/.cursor/rules/agentuity.mdc (1)
3-4: Broaden glob pattern for matching.
Theglobs: "agentuity.yaml"entry may not match files in nested directories. Use a glob like**/agentuity.yamlto ensure it applies project-wide.agents/Startup_News_Scraper/.cursor/rules/agent.mdc (2)
9-10: Clarify terminology for importing types.
Instead of "loading types from the node modules package", use "importing types from@agentuity/sdk" for clearer intent.
28-29: Fix grammatical error.
Change "helper methods and public variables which can be used for working with data has been passed"
→ "helper methods and public variables for working with data that has been passed".agents/Startup_News_Scraper/tsconfig.json (2)
6-7:jsx: "react-jsx"is superfluous for a non-React agentNothing in the project tree indicates JSX/React usage.
Keeping the option increases the compiled lib size and allows JSX accidentally slipping in.
Remove the setting unless you really plan to embed React components.
15-16: DisablingnoUnused*hides dead-code & typo mistakes
"noUnusedLocals": falseand"noUnusedParameters": falsesilence the compiler on two of the most common bugs (unused vars / wrong param names).
Enable them and fix the inevitable low-hanging warnings – you’ll catch mistakes earlier with zero runtime cost.- "noUnusedLocals": false, - "noUnusedParameters": false, + "noUnusedLocals": true, + "noUnusedParameters": true,agents/Startup_News_Scraper/package.json (1)
21-25: Declare the Bun runtime inenginesfor clearer install constraints"private": true, + "engines": { + "bun": ">=1.2.4" + },Helps CI and external consumers fail fast on wrong runtimes.
agents/Startup_News_Scraper/README.md (2)
2-7: Image missing alt-text (MD045)Accessibility & markdown-linter complain.
-<img src="https://raw.githubusercontent.com/agentuity/cli/refs/heads/main/.github/Agentuity.png" alt="Agentuity" width="100"/> <br/> +<img src="https://raw.githubusercontent.com/agentuity/cli/refs/heads/main/.github/Agentuity.png" alt="Agentuity logo" width="100"/> <br/>
65-70: Code block lacks language hint (MD040)-``` +```text ├── agents/ ...agents/Startup_News_Scraper/src/agents/startup_scraper/fetchFeeds.ts (1)
21-45: Consider deduplicating & sorting across feedsCurrent implementation:
- Returns items in feed-order; not necessarily “newest”.
- Same article appearing in multiple feeds will be duplicated.
Not blocking, but worth a TODO for better UX.
agents/Startup_News_Scraper/src/agents/startup_scraper/types.ts (1)
8-16: Prefer camelCase & align optional/nullable semanticsThe interface mixes camelCase (
funding_source) and snake_case (why_it_matters) with camel (industry). Type-Script conventions generally lean on camelCase.
In addition,summaryis required but can benull, while the other nullable properties are markedoptional. Consider making all optional+nullable for consistency, or all required with| null.- why_it_matters?: string | null; - funding_source?: string | null; + whyItMatters?: string | null; + fundingSource?: string | null;Down-stream code must be updated accordingly.
agents/Startup_News_Scraper/index.ts (1)
3-9: Extendingprocesspollutes global typesAugmenting the built-in
Processinterface directly can leak into other dependencies and cause type clashes. Prefer declaring a local helper:const isBun = (process as any).isBun === true;and use
isBuninstead of attaching to the global namespace.agents/Startup_News_Scraper/src/agents/startup_scraper/index.ts (1)
34-36: Surface error details for easier debuggingReturning a generic
"Agent failed"string gives no hints when running outside Agentuity. Attach the error message in non-production environments, or at least logerr.stack.agents/Startup_News_Scraper/src/agents/startup_scraper/summarize.ts (2)
20-24: Silent loss of rejected summariesRejected promises are discarded, so callers never know some articles failed to summarise. At minimum, log
res.reason; ideally, return afailedlist or retry.- for (const res of settled) { - if (res.status === 'fulfilled') summaries.push(res.value); - } + for (const res of settled) { + if (res.status === 'fulfilled') { + summaries.push(res.value); + } else { + console.warn('OpenAI summarisation failed:', res.reason); + } + }
44-52: Hard-coded model & prompt size – make configurable
gpt-4o-mini,max_tokens: 400, andtemperature: 0.4are project choices but should live in env vars / constants to avoid code edits for tuning or quota issues.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
agents/Startup_News_Scraper/bun.lockis excluded by!**/*.lock
📒 Files selected for processing (15)
agents/Startup_News_Scraper/.cursor/rules/agent.mdc(1 hunks)agents/Startup_News_Scraper/.cursor/rules/agentuity.mdc(1 hunks)agents/Startup_News_Scraper/.cursor/rules/sdk.mdc(1 hunks)agents/Startup_News_Scraper/.editorconfig(1 hunks)agents/Startup_News_Scraper/.gitignore(1 hunks)agents/Startup_News_Scraper/README.md(1 hunks)agents/Startup_News_Scraper/agentuity.yaml(1 hunks)agents/Startup_News_Scraper/biome.json(1 hunks)agents/Startup_News_Scraper/index.ts(1 hunks)agents/Startup_News_Scraper/package.json(1 hunks)agents/Startup_News_Scraper/src/agents/startup_scraper/fetchFeeds.ts(1 hunks)agents/Startup_News_Scraper/src/agents/startup_scraper/index.ts(1 hunks)agents/Startup_News_Scraper/src/agents/startup_scraper/summarize.ts(1 hunks)agents/Startup_News_Scraper/src/agents/startup_scraper/types.ts(1 hunks)agents/Startup_News_Scraper/tsconfig.json(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (2)
agents/Startup_News_Scraper/src/agents/startup_scraper/fetchFeeds.ts (1)
agents/Startup_News_Scraper/src/agents/startup_scraper/types.ts (1)
StartupLink(3-6)
agents/Startup_News_Scraper/src/agents/startup_scraper/summarize.ts (1)
agents/Startup_News_Scraper/src/agents/startup_scraper/types.ts (2)
StartupLink(3-6)StartupSummary(8-16)
🪛 markdownlint-cli2 (0.17.2)
agents/Startup_News_Scraper/README.md
6-6: Images should have alternate text (alt text)
null
(MD045, no-alt-text)
65-65: Fenced code blocks should have a language specified
null
(MD040, fenced-code-language)
🔇 Additional comments (2)
agents/Startup_News_Scraper/package.json (1)
4-5: ```shell
#!/bin/bashVerify presence of "module" entry and existence of index.js/index.ts in the package folder
echo "Searching for a 'module' field in package.json:"
grep -R '"module"' -n agents/Startup_News_Scraper/package.json || echo "→ No 'module' field found"echo
echo "Locating index.js in agents/Startup_News_Scraper:"
find agents/Startup_News_Scraper -type f -name index.js || echo "→ index.js not found"echo
echo "Locating index.ts in agents/Startup_News_Scraper:"
find agents/Startup_News_Scraper -type f -name index.ts || echo "→ index.ts not found"</details> <details> <summary>agents/Startup_News_Scraper/.cursor/rules/sdk.mdc (1)</summary> `1-98`: **Docs look good – no blocking issues** No actionable code concerns here. </details> </blockquote></details> </details> <!-- This is an auto-generated comment by CodeRabbit for review status -->
| indent_style = tab | ||
| indent_size = 2 |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Resolve indent style mismatch.
.editorconfig enforces tab indentation but biome.json uses space indentation of 2 spaces. This conflict can lead to inconsistent formatting across editors and CI tooling. Please align the indent style across both configurations.
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/.editorconfig at lines 7 to 8, the indent_style
is set to tab while biome.json uses 2 spaces for indentation, causing
inconsistency. Update the .editorconfig to set indent_style to space and
indent_size to 2 to match biome.json, ensuring consistent indentation across all
tools and editors.
| # logs | ||
| logs | ||
| _.log | ||
| report.[0-9]_.[0-9]_.[0-9]_.[0-9]_.json |
There was a problem hiding this comment.
Fix glob pattern for log files.
The entry _.log only matches files starting with an underscore. It should be *.log to ignore all log files.
Additionally, review the report.[0-9]_.[0-9]_.[0-9]_.[0-9]_.json pattern for correctness or simplification.
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/.gitignore around lines 13 to 16, the pattern
`_.log` only matches files starting with an underscore and ending with .log, but
it should be changed to `*.log` to ignore all log files. Also, review the
`report.[0-9]_.[0-9]_.[0-9]_.[0-9]_.json` pattern for correctness or
simplification, possibly replacing it with a simpler or more accurate glob
pattern that matches the intended report files.
| "enabled": true, | ||
| "indentStyle": "space", | ||
| "indentWidth": 2 | ||
| }, |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Align indentation settings with .editorconfig.
biome.json specifies space-based indentation (indentStyle: "space") whereas .editorconfig enforces tabs. Please unify the indent style to avoid editor and formatter discrepancies.
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/biome.json around lines 13 to 16, the indentation
settings specify spaces ("indentStyle": "space") which conflicts with the
tab-based indentation defined in the .editorconfig file. Update the
"indentStyle" value from "space" to "tab" to align with the .editorconfig and
ensure consistent formatting across editors and tools.
agents/Startup_News_Scraper/src/agents/startup_scraper/fetchFeeds.ts
Outdated
Show resolved
Hide resolved
| enabled: true | ||
| identifier: bunjs | ||
| language: javascript | ||
| runtime: bunjs | ||
| agents: | ||
| dir: src/agents | ||
| ignore: | ||
| - node_modules/** | ||
| - dist/** | ||
| - src/** |
There was a problem hiding this comment.
Bundler ignores the entire src/** tree – agent will bundle nothing
bundler.ignore lists src/**, which also matches src/agents/** – the only place your code lives.
When agentuity bundle runs, the resulting artifact will be empty and startup will fail.
Remove the pattern or narrow it:
- ignore:
- - node_modules/**
- - dist/**
- - src/**
+ ignore:
+ - node_modules/**
+ - dist/**Verify a fresh agentuity bundle afterwards – the generated .agentuity/index.js should now include your code.
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/agentuity.yaml around lines 60 to 69, the bundler
ignore list includes "src/**", which excludes the entire source directory
including "src/agents" where your code resides, resulting in an empty bundle. To
fix this, remove the "src/**" pattern from the ignore list or narrow it to
exclude only irrelevant subdirectories, then run "agentuity bundle" again to
verify that the generated .agentuity/index.js includes your code.
| runner(true, import.meta.dirname).catch((err) => { | ||
| if (err instanceof Error) { |
There was a problem hiding this comment.
🛠️ Refactor suggestion
import.meta.dirname is not standard in Node ≤ 22
import.meta.dirname exists in Bun but not in current Node LTS. When the user follows the earlier suggestion to run node --env-file …, the code will crash with
Property 'dirname' does not exist on type 'ImportMeta'.
A portable alternative:
-runner(true, import.meta.dirname).catch((err) => {
+import { fileURLToPath } from 'node:url';
+import { dirname } from 'node:path';
+
+const __dirname = dirname(fileURLToPath(import.meta.url));
+
+runner(true, __dirname).catch((err) => {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| runner(true, import.meta.dirname).catch((err) => { | |
| if (err instanceof Error) { | |
| import { fileURLToPath } from 'node:url'; | |
| import { dirname } from 'node:path'; | |
| const __dirname = dirname(fileURLToPath(import.meta.url)); | |
| runner(true, __dirname).catch((err) => { | |
| if (err instanceof Error) { |
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/index.ts around lines 43 to 44, the use of
import.meta.dirname is not supported in Node.js versions up to 22, causing a
runtime error. Replace import.meta.dirname with a portable alternative by using
standard Node.js methods such as path.dirname and url.fileURLToPath with
import.meta.url to obtain the directory name in a way compatible with current
Node LTS versions.
| export default async function Agent(req: AgentRequest, resp: AgentResponse) { | ||
| try { | ||
| const raw = (await req.data.text()) || '{}'; | ||
| const body = JSON.parse(raw) as { perFeed?: number }; | ||
|
|
||
| const perFeed = body.perFeed ?? 3; // ← new default | ||
|
|
||
| const links = await getStartupLinks(perFeed); |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Validate perFeed to avoid abuse / errors
perFeed is blindly accepted. A client can send a negative or extremely large value leading to an empty slice or dozens of OpenAI calls.
-const perFeed = body.perFeed ?? 3;
+const perFeed = Math.min(Math.max(body.perFeed ?? 3, 1), 10); // 1-10 hard-limitReturn 400 when out of bounds to make the contract explicit.
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/src/agents/startup_scraper/index.ts around lines
17 to 24, the perFeed parameter is accepted without validation, which can lead
to negative or excessively large values causing errors or resource abuse. Add
validation to check if perFeed is within a reasonable range (e.g., a positive
integer and below a maximum threshold). If perFeed is out of bounds, respond
with a 400 status code and an appropriate error message to explicitly enforce
the contract.
| /* small helper to bucket common industry values */ | ||
| function normaliseIndustry(raw?: string): 'AI' | 'VC' | 'Tech' | 'Startups' | undefined { | ||
| if (!raw) return undefined; | ||
| const t = raw.toLowerCase(); | ||
| if (/(^| )(ai|artificial intelligence|machine learning|ml)( |$)/.test(t)) { return 'AI';} | ||
| if (/(vc|venture|fund|capital|investor)/.test(t)) { return 'VC';} | ||
| if (/(startup|accelerator|incubator|founder program)/.test(t)) { return 'Startups';} | ||
| if (/(tech|technology|robot|drone|fintech|crypto|software|hardware|cloud|SaaS)/.test(t)) { return 'Tech';} | ||
| return undefined; |
There was a problem hiding this comment.
Regex for “saas” never matches after .toLowerCase()
The text is lower-cased (t) but the pattern contains uppercase SaaS, and the regex is case-sensitive.
-if (/(tech|technology|robot|drone|fintech|crypto|software|hardware|cloud|SaaS)/.test(t)) { return 'Tech';}
+if (/(tech|technology|robot|drone|fintech|crypto|software|hardware|cloud|saas)/.test(t)) { return 'Tech';}Add the i flag or keep everything lower-case.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /* small helper to bucket common industry values */ | |
| function normaliseIndustry(raw?: string): 'AI' | 'VC' | 'Tech' | 'Startups' | undefined { | |
| if (!raw) return undefined; | |
| const t = raw.toLowerCase(); | |
| if (/(^| )(ai|artificial intelligence|machine learning|ml)( |$)/.test(t)) { return 'AI';} | |
| if (/(vc|venture|fund|capital|investor)/.test(t)) { return 'VC';} | |
| if (/(startup|accelerator|incubator|founder program)/.test(t)) { return 'Startups';} | |
| if (/(tech|technology|robot|drone|fintech|crypto|software|hardware|cloud|SaaS)/.test(t)) { return 'Tech';} | |
| return undefined; | |
| /* small helper to bucket common industry values */ | |
| function normaliseIndustry(raw?: string): 'AI' | 'VC' | 'Tech' | 'Startups' | undefined { | |
| if (!raw) return undefined; | |
| const t = raw.toLowerCase(); | |
| if (/(^| )(ai|artificial intelligence|machine learning|ml)( |$)/.test(t)) { return 'AI';} | |
| if (/(vc|venture|fund|capital|investor)/.test(t)) { return 'VC';} | |
| if (/(startup|accelerator|incubator|founder program)/.test(t)) { return 'Startups';} | |
| - if (/(tech|technology|robot|drone|fintech|crypto|software|hardware|cloud|SaaS)/.test(t)) { return 'Tech';} | |
| + if (/(tech|technology|robot|drone|fintech|crypto|software|hardware|cloud|saas)/.test(t)) { return 'Tech';} | |
| return undefined; | |
| } |
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/src/agents/startup_scraper/summarize.ts around
lines 75 to 83, the regex pattern for matching "SaaS" is case-sensitive and does
not match after the input string is converted to lowercase. Fix this by either
adding the case-insensitive flag `i` to the regex or changing "SaaS" to
lowercase "saas" to ensure the pattern matches correctly.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
agents/Startup_News_Scraper/README.md (4)
3-11: Add alt text to the deploy image
The second<img>tag (line 8) lacks analtattribute, which affects accessibility.- <img src="https://app.agentuity.com/img/deploy.svg" /> + <img src="https://app.agentuity.com/img/deploy.svg" alt="Deploy with Agentuity" />
69-72: Add missing comma for clarity
Insert a comma after "dashboard" and before "and choose" for proper pauses.-1. In your **Agentuity** dashboard open the deployed agent, click the **➕** icon under *IO Visualization* and choose **API**. +1. In your **Agentuity** dashboard, open the deployed agent, click the **➕** icon under *IO Visualization*, and choose **API**.
87-90: Remove redundant mention of "Code Node"
The heading "Do not adjust Code Node" and the subsequent list item both mention the Code node. Consider merging or rephrasing to avoid repetition.
117-121: Fix list indentation
The nested list under "You can also implement other n8n nodes:" uses 4-space indentation; markdownlint expects 2 spaces.- - Notion – create or append to a database - - Google Sheets – log each day’s stories - - Telegram / Teams – forward to other chat apps - - Webhooks – POST the digest JSON to any backend + - Notion – create or append to a database + - Google Sheets – log each day’s stories + - Telegram / Teams – forward to other chat apps + - Webhooks – POST the digest JSON to any backend
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
agents/Startup_News_Scraper/README.md(1 hunks)agents/Startup_News_Scraper/n8n_workflow/workflow.json(1 hunks)
🧰 Additional context used
🪛 LanguageTool
agents/Startup_News_Scraper/README.md
[uncategorized] ~70-~70: A comma might be missing here.
Context: ...y API endpoint 1. In your Agentuity dashboard open the deployed agent, click the *➕...
(AI_EN_LECTOR_MISSING_PUNCTUATION_COMMA)
[duplication] ~83-~83: Possible typo: you repeated a word.
Context: ...on** - Body Content Type – select JSON - JSON Body – {} (a single empty JSON objec...
(ENGLISH_WORD_REPEAT_RULE)
[grammar] ~87-~87: This phrase is duplicated. You should probably use “Code Node” only once.
Context: ...– leave blank ### 6. Do not adjust Code Node 1. Code node is preconfigured with the logic to tran...
(PHRASE_REPETITION)
[uncategorized] ~91-~91: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...eadable HTML. 2. Do not change. ### 7. Third Party Configuration (Email, Slack, Discord, e...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.17.2)
agents/Startup_News_Scraper/README.md
8-8: Images should have alternate text (alt text)
null
(MD045, no-alt-text)
36-36: Trailing punctuation in heading
Punctuation: ':'
(MD026, no-trailing-punctuation)
118-118: Unordered list indentation
Expected: 2; Actual: 4
(MD007, ul-indent)
119-119: Unordered list indentation
Expected: 2; Actual: 4
(MD007, ul-indent)
120-120: Unordered list indentation
Expected: 2; Actual: 4
(MD007, ul-indent)
121-121: Unordered list indentation
Expected: 2; Actual: 4
(MD007, ul-indent)
| { | ||
| "parameters": { | ||
| "method": "POST", | ||
| "options": {} | ||
| }, | ||
| "type": "n8n-nodes-base.httpRequest", | ||
| "typeVersion": 4.2, | ||
| "position": [ | ||
| -440, | ||
| 180 | ||
| ], | ||
| "id": "a1f3a427-c810-49d6-b3c6-dd1ce4fdda16", | ||
| "name": "HTTP Request" |
There was a problem hiding this comment.
HTTP Request URL placeholder must be configured
The HTTP Request node only specifies the method; the url field is blank and needs to be set to your Agentuity API endpoint as described in the README.
🤖 Prompt for AI Agents
In agents/Startup_News_Scraper/n8n_workflow/workflow.json around lines 23 to 35,
the HTTP Request node is missing the required 'url' parameter. Add the 'url'
field inside the "parameters" object and set it to the Agentuity API endpoint
URL as specified in the README to ensure the request is properly directed.
2 participants
Summary by CodeRabbit
New Features
Documentation
Chores