Skip to content

Add robots.txt to test sites #1

@adamschwartz

Description

@adamschwartz

As stated in the README, we deploy projects to both a testing site (cloudflare-docs.workers.dev) and a production site (workers.cloudflare.com):

Test sites deploy to:
```txt
https://[pathPrefix].cloudflare-docs.workers.dev/[pathPrefix]/
```
Production sites deploy to:
```txt
https://developers.cloudflare.com/[pathPrefix]/
```

We should instruct search engines to ignore the test sites via a robots.txt with:

User-agent: * Disallow: /

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions