diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index ff81e53fa..9eeef1045 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -3,7 +3,7 @@ name: Bug report about: Create a report to help us improve title: "[bug] Something went wrong" labels: bug -assignees: KCarretto +assignees: '' --- @@ -12,6 +12,7 @@ A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: + 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' @@ -24,15 +25,17 @@ A clear and concise description of what you expected to happen. If applicable, add screenshots to help explain your problem. **Desktop (please complete the following information):** - - OS: [e.g. iOS] - - Browser [e.g. chrome, safari] - - Version [e.g. 22] + +- OS: [e.g. iOS] +- Browser [e.g. chrome, safari] +- Version [e.g. 22] **Smartphone (please complete the following information):** - - Device: [e.g. iPhone6] - - OS: [e.g. iOS8.1] - - Browser [e.g. stock browser, safari] - - Version [e.g. 22] + +- Device: [e.g. iPhone6] +- OS: [e.g. iOS8.1] +- Browser [e.g. stock browser, safari] +- Version [e.g. 22] **Additional context** Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index dcf83a420..02f8cf7f6 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -2,7 +2,7 @@ name: Feature request about: Suggest an idea for this project title: "[feature] Something to do" -labels: enhancement +labels: feature assignees: '' --- diff --git a/.github/ISSUE_TEMPLATE/feedback.md b/.github/ISSUE_TEMPLATE/feedback.md new file mode 100644 index 000000000..3384c9522 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feedback.md @@ -0,0 +1,20 @@ +--- +name: Feedback +about: Provide feedback for this project +title: "[feedback] Something that went well / needs improvement" +labels: feedback +assignees: '' + +--- + +**Is your feedback related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshot here. diff --git a/README.md b/README.md index 829521a70..0e3cdb601 100644 --- a/README.md +++ b/README.md @@ -11,15 +11,43 @@ [![Rust Report Card](https://rust-reportcard.xuri.me/badge/github.com/spellshift/realm)](https://rust-reportcard.xuri.me/report/github.com/spellshift/realm) [![Docs](https://img.shields.io/badge/read%20our-docs-informational)](https://docs.realm.pub/) -Realm is a cross platform Red Team engagement platform with a focus on automation and reliability. +[Realm](https://github.com/spellshift/realm) is an Adversary Emulation Framework with a **focus on scalability, reliability, and automation**. It is highly performant and is designed for engagements of any size (up to many thousands of beacons). [Get started in minutes](https://docs.realm.pub/user-guide/getting-started). -https://github.com/spellshift/realm/assets/16250309/7b5834d9-a864-490a-96e5-8d83b276af11 + -## Features +## Feature Highlights + +- **Eldritch, a Pythonic DSL for Offensive Security:** Ditch clunky scripting and embrace [Eldritch](https://docs.realm.pub/user-guide/eldritch), Realm's Pythonic [Domain Specific Language (DSL)](https://en.wikipedia.org/wiki/Domain-specific_language) based on [Google Starlark](https://github.com/bazelbuild/starlark/blob/master/spec.md#starlark-language-specification). Write clear, concise, reuseable code that reflects your strategic thinking and streamlines offensive operations. [Eldritch](https://docs.realm.pub/user-guide/eldritch) is natively compiled to Rust, providing a performant abstraction for low-level system interactions. + +- **Effortless Multi-Host Management:** Juggling tasks across numerous machines during complex engagements? Realm simplifies the process, enabling you to control agents on multiple hosts simultaneously. + +- **Native GCP Integration:** Leverage the power and scalability of Google Cloud directly within your red team engagements. Realm seamlessly integrates with GCP services, boosting your attack capabilities without reinventing the wheel. + +- **Stateless Server Architecture:** While Realm officially supports GCP, you may deploy it's [stateless docker container](https://hub.docker.com/r/spellshift/tavern) to any environment that best fits your needs. + +- **Focus on Reliability:** Realm always prioritizes quality over quantity, enabling operators to focus on the engagement instead of spending hours troubleshooting bugs. Extensive testing and rigorous code review ensure unwavering reliability, while an intuitive design and clear documentation keep the learning curve minimal. After reaching a stable `1.0.0` release, Realm will follow [Semantic Versioning](https://semver.org/), ensuring the stability of older deployments. + +## Quick Start + +*To deploy a production ready instance see the [setup guide](https://docs.realm.pub/user-guide/tavern).* + +```bash +# Clone Realm +git clone https://github.com/spellshift/realm.git && cd realm + +# Start Tavern (Server) +go run ./tavern + +# In a new terminal, +# Start Imix (Agent) +cd realm/implants/imix && cargo run +``` + +## Project Components ### Agent (imix) -- Written in rust with support for MacOS, Linux, and Windows. +- Written in Rust with support for MacOS, Linux, and Windows. - Supports long running tasks by reading output from tasks in real time. - Interval callback times. - Simple file based configuration. @@ -41,29 +69,40 @@ https://github.com/spellshift/realm/assets/16250309/7b5834d9-a864-490a-96e5-8d83 - Remote execution over SSH. - And much much more: -## Quickstart guide +## Want to contribute? -*To deploy a production ready instance see the [tavern setup guide](https://docs.realm.pub/user-guide/tavern).* +Check out our [developer docs!](https://docs.realm.pub/dev-guide/introduction) -### Start the server +## Contact Support -```bash -git clone https://github.com/spellshift/realm.git && cd realm +Need a hand? We're here to help! If you're facing an issue with Realm, we're happy to assist! To ensure we can provide the best support, please [create an issue on our Github](https://github.com/spellshift/realm/issues/new?labels=bug&template=bug_report.md). -go run ./tavern +### Bug Support -# If you'd like to test without deploying an agent use the test data. -ENABLE_TEST_DATA=1 go run ./tavern -``` +When opening your issue, please include: -### Start the agent +- A clear and concise description of the problem you're encountering. +- Any relevant error messages or logs. +- Steps to reproduce the issue (if possible). +- Impacted Realm version and operating system. -```bash -git clone https://github.com/spellshift/realm.git -cd realm/implants/imix && cargo run +The more information you provide, the faster we can investigate and help you resolve the issue. -``` +### Feature Requests & Feedback + +Realm lives and breathes through its users. Your insights and experiences are crucial in guiding its development and ensuring it continues to empower your mission. Please don't hesitate to reach out! + +**Remember:** + +- Be respectful and constructive in your feedback ([code of conduct](https://github.com/spellshift/realm/blob/main/CODE_OF_CONDUCT.md)). +- Search for existing discussions or feature requests before creating new ones. +- The more details you provide, the better we can understand your needs and respond effectively. +- Together, we can shape Realm into an incredible framework. **Thank you for being part of the adventure!** + +#### Feature Requests + +Do you have an idea for a feature that would supercharge your workflow? We're all ears! [Open an issue on GitHub](https://github.com/spellshift/realm/issues/new?labels=feature&projects=&template=feature_request.md&title=%5Bfeature%5D+Something+to+do) and share your detailed proposal. Be sure to explain the problem you're facing, the solution you envision, and how it would benefit other users. The more information you provide, the better we can understand your needs and assess the feasibility of implementing your suggestion. -## Want to contribute start here +#### Provide Feedback - +Love something about Realm? Feel something could be improved? Let us know! Your feedback, good or bad, helps us make Realm better for everyone. [Open an issue on GitHub](https://github.com/spellshift/realm/issues/new?labels=feedback&projects=&template=feedback.md&title=%5Bfeedback%5D+Something+to+improve) outlining your thoughts, whether it's a praiseworthy feature, a usability concern, or a suggestion for improvement. Every bit of your feedback helps us refine Realm and make it an even more valuable tool in your red teaming toolbox. diff --git a/docs/README.md b/docs/README.md index 645ba4357..eccd0628a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,16 +1,23 @@ +# Updating Docs + +Note: All commands should be run from the docs folder (e.g. realm/docs) + ## Testing Changes -_All commands should be run from the docs folder (e.g. realm/docs)_ Ensure you have installed the dependencies: -``` + +```sh +gem install bundler -v '2.3.27' bundle install ``` Run the documentation locally with: -``` + +```sh bundle exec jekyll serve --livereload ``` ## Docsy Jekyll Template + * [Source on Github](https://github.com/vsoch/docsy-jekyll) * [View on Github Pages](https://vsoch.github.io/docsy-jekyll/) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index 97c7c8c26..aed542141 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -1,6 +1,11 @@ - title: About url: about + - title: User Guide - url: user-guide + url: user-guide/getting-started + +- title: Admin Guide + url: admin-guide/tavern + - title: Developer Guide - url: dev-guide + url: dev-guide/introduction diff --git a/docs/_data/toc.yml b/docs/_data/toc.yml index fc6df8e79..188e9e8d3 100644 --- a/docs/_data/toc.yml +++ b/docs/_data/toc.yml @@ -1,10 +1,8 @@ - title: User Guide - url: user-guide + url: user-guide/getting-started links: - title: "Getting Started" url: "user-guide/getting-started" - - title: "Tavern" - url: "user-guide/tavern" - title: "UI" url: "user-guide/ui" - title: "Imix" @@ -16,16 +14,29 @@ children: - title: Standard Library url: "user-guide/eldritch#standard-library" + - title: "Terminology" + url: "user-guide/terminology" + +- title: Admin Guide + url: admin-guide/tavern + links: + - title: "Tavern" + url: "admin-guide/tavern" + children: + - title: "Configuration" + url: "admin-guide/tavern#configuration" + - title: "User Interface" + url: "admin-guide/tavern#user-interface" + - title: "GraphQL API" + url: "admin-guide/tavern#graphql-api" - title: Developer Guide - url: dev-guide + url: dev-guide/introduction links: - title: "Introduction" url: "dev-guide/introduction" children: - title: "Contribution Guidelines" url: "dev-guide/introduction#contribution-guidelines" - - title: "Terminology" - url: "dev-guide/introduction#terminology" - title: "Project Structure" url: "dev-guide/introduction#project-structure" - title: "Where to Start?" @@ -33,16 +44,10 @@ - title: "Tavern" url: "dev-guide/tavern" children: - - title: "Configuration" - url: "dev-guide/tavern#configuration" - - title: "User Interface" - url: "dev-guide/tavern#user-interface" - title: "CDN API" - url: "dev-guide/tavern#cdn-api" - - title: "GraphQL API" - url: "dev-guide/tavern#graphql-api" + url: "admin-guide/tavern#cdn-api" - title: "Agent Development" - url: "dev-guide/tavern#agent-development" + url: "admin-guide/tavern#agent-development" - title: "Eldritch" url: "dev-guide/eldritch" children: @@ -53,8 +58,8 @@ - title: "OS Specific functions" url: "dev-guide/eldritch#os-specific-functions" - title: "Notes about using dictionary type Dict" - url: "dev-guide/eldritch#notes-about-using-dictionary-type-dict" + url: "dev-guide/eldritch#using-dict" - title: "Notes about asynchronous Eldritch code" - url: "dev-guide/eldritch#notes-about-asynchronous-eldritch-code" + url: "dev-guide/eldritch#using-async" - title: "About" url: "about" diff --git a/docs/_docs/admin-guide/tavern.md b/docs/_docs/admin-guide/tavern.md new file mode 100644 index 000000000..7dc7cd04d --- /dev/null +++ b/docs/_docs/admin-guide/tavern.md @@ -0,0 +1,383 @@ +--- +title: Tavern +tags: + - Admin Guide +description: Information on managing a Tavern deployment. +permalink: admin-guide/tavern +--- + +## Overview + +Tavern is a teamserver for Realm, providing a UI to control deployments and implants during an engagement. The majority of Tavern's functionality is exposed through a GraphQL API, which is used by both implants and the UI. + +If you would like to help contribute to Tavern, please take a look at our [open issues](https://github.com/spellshift/realm/issues?q=is%3Aopen+is%3Aissue+label%3Atavern). + +## Deployment + +This section will walk you through deploying a production ready instance of Tavern to GCP. If you're just looking to play around with Tavern, feel free to run the [docker image (kcarretto/tavern:latest)](https://hub.docker.com/repository/docker/kcarretto/tavern/general) locally. + +### 1. Create a GCP Project + +Navigate to the [GCP Console](https://console.cloud.google.com/) and [create a new GCP project](https://console.cloud.google.com/projectcreate). +![assets/img/tavern/deploy/create-gcp-project.png](/assets/img/tavern/deploy/create-gcp-project.png) + +Make a note of the created Project ID as you'll need that in a later step +![assets/img/tavern/deploy/gcp-project-info.png](/assets/img/tavern/deploy/gcp-project-info.png) + +### 2. Setup OAuth (Optional) + +_Note: These setup instructions assume you own a domain which you would like to host Tavern at._ + +If you want to configure OAuth for your Tavern Deployment, navigate to the [GCP OAuth Consent Screen](https://console.cloud.google.com/apis/credentials/consent) and create a new External consent flow. **If you do not configure OAuth, Tavern will not perform any authentication or authorization for requests.** + +![assets/img/tavern/deploy/gcp-new-oauth-consent.png](/assets/img/tavern/deploy/gcp-new-oauth-consent.png) + +Provide details that users will see when logging into Tavern, for example: + +* App Name: "Tavern" +* User Support Email: "" +* App Logo: Upload something cool if you'd like, but then you'll need to complete a verification process. +* App Domain: "" +* Authorized Domains: "mydomain.com" +* Developer Contact Information: "" + +Add the ".../auth/userinfo.profile" scope, used by Tavern to obtain user names and photourls. +![assets/img/tavern/deploy/gcp-oauth-scope.png](/assets/img/tavern/deploy/gcp-oauth-scope.png) + +Next, add yourself as a "Test User". **Until you publish your app, only test users may complete the OAuth consent flow.** If you didn't select any options that require verification, you may publish your app now (so you won't need to allowlist the users for your application). + +Navigate to the [Credentials Tool](https://console.cloud.google.com/apis/credentials) and select "Create Credentials" -> "OAuth client ID". Be sure to add an "Authorized redirect URI" so that the consent flow redirects to the appropriate Tavern endpoint. For example "mydomain.com/oauth/authorize". Save the resulting Client ID and Client secret for later. +![assets/img/tavern/deploy/oauth-new-creds.png](/assets/img/tavern/deploy/oauth-new-creds.png) + +Next, configure a CNAME record for the domain you'd like to host Tavern at (e.g. "tavern.mydomain.com") to point to "ghs.googlehosted.com.". +![assets/img/tavern/deploy/google-dns-cname.png](/assets/img/tavern/deploy/google-dns-cname.png) + +And that's it! In the below sections on deployment, please ensure you properly configure your OAUTH_CLIENT_ID, OAUTH_CLIENT_SECRET, and OAUTH_DOMAIN to ensure Tavern is properly configured. + +### 3. Google Cloud CLI + +Follow [these instructions](https://cloud.google.com/sdk/docs/install) to install the gcloud CLI. This will enable you to quickly obtain credentials that terraform will use to authenticate. Alternatively, you may create a service account (with appropriate permissions) and obtain [Application Default Credentials](https://cloud.google.com/sdk/gcloud/reference/auth/application-default) for it. See [these Authentication Instructions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#authentication) for more information on how to configure GCP authentication for Terraform. + +After installing the gcloud CLI, run `gcloud auth application-default login` to obtain Application Default Credentials. + +### 4. Terraform + +1. Follow [these instructions](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) to install the Terraform CLI. +2. Clone [the repo](https://github.com/spellshift/realm) and navigate to the `terraform` directory. +3. Run `terraform init` to install the Google provider for terraform. +4. Run `terraform apply -var="gcp_project=" -var="oauth_client_id=" -var="oauth_client_secret=" -var="oauth_domain="` to deploy Tavern! + +**Example:** + +```sh +terraform apply -var="gcp_project=new-realm-deployment" -var="oauth_client_id=12345.apps.googleusercontent.com" -var="oauth_client_secret=ABCDEFG" -var="oauth_domain=test-tavern.redteam.toys" +``` + +After terraform completes successfully, head to the [DNS mappings for Cloud Run](https://console.cloud.google.com/run/domains) and wait for a certificate to successfully provision. This may take a while, so go enjoy a nice cup of coffee ☕ + +After your certificate has successfully provisioned, it may still take a while (e.g. an hour or two) before you are able to visit Tavern using your custom OAuth Domain (if configured). + +#### CLI Variables + +|Name|Required|Description| +|----|--------|-----------| +|gcp_project|Yes|Project ID of the GCP Project created in step 1.| +|gcp_region|No|Region to deploy to.| +|mysql_user|No|The MySQL user to create and connect with.| +|mysql_passwd|No|The MySQL password to set and connect with. Autogenerated by default. | +|mysql_dbname|No|MySQL Database to create and connect to.| +|mysql_tier|No|The type of instance to run the Cloud SQL Database on.| +|oauth_client_id|Only if OAuth is configured|The OAuth ClientID Tavern will use to connect to the IDP (Google).| +|oauth_client_secret|Only if OAuth is configured|The OAuth Client Secret Tavern will use to connect to the IDP (Google).| +|oauth_domain|Only if OAuth is configured|The OAuth Domain that the IDP should redirect to e.g. tavern.mydomain.com (should be the domain you set a CNAME record for while configuring OAuth).| +|min_scale|No|The minimum number of containers to run, if set to 0 you may see cold boot latency.| +|max_scale|No|The maximum number of containers to run.| + +### Manual Deployment Tips + +Below are some deployment gotchas and notes that we try to address with Terraform, but can be a bit tricky if trying to deploy Tavern manually. + +* MySQL version 8.0 must be started with the flag `default_authentication_plugin=caching_sha2_password` for authentication to work properly. A new user must be created for authentication. +* When running in CloudRun, it's best to connect to CloudSQL via a unix socket (so ensure the `MYSQL_NET` env var is set to "unix" ). + * After adding a CloudSQL connection to your CloudRun instance, this unix socket is available at `/cloudsql/` (e.g. `/cloudsql/realm-379301:us-east4:tavern-db`). +* You must create a new database in your CloudSQL instance before launching Tavern and ensure the `MYSQL_DB` env var is set accordingly. + +## Configuration + +### Metrics + +By default, Tavern does not export metrics. You may use the below environment configuration variables to enable [Prometheus](https://prometheus.io/docs/introduction/overview/) metric collection. These metrics become available at the "/metrics" endpoint configured. These metrics are hosted on a separate HTTP server such that it can be restricted to localhost (default). This is because the endpoint is unauthenticated, and would leak sensitive information if it was accessible. + +| Env Var | Description | Default | Required | +| ------- | ----------- | ------- | -------- | +| ENABLE_METRICS | Set to any value to enable the "/metrics" endpoint. | Disabled | No | +| HTTP_METRICS_LISTEN_ADDR | Listen address for the metrics HTTP server, it must be different than the value of `HTTP_LISTEN_ADDR`. | `127.0.0.1:8080` | No | + +### MySQL + +By default, Tavern operates an in-memory SQLite database. To persist data, a MySQL backend is supported. In order to configure Tavern to use MySQL, the `MYSQL_ADDR` environment variable must be set to the `host[:port]` of the database (e.g. `127.0.0.1`, `mydb.com`, or `mydb.com:3306`). You can reference the [mysql.Config](https://pkg.go.dev/github.com/go-sql-driver/mysql#Config) for additional information about Tavern's MySQL configuration. + +The following environment variables are currently supported for additional MySQL Configuration: + +| Env Var | Description | Default | Required | +| ------- | ----------- | ------- | -------- | +| MYSQL_ADDR| Address of the MySQL server (e.g. `host[:port]`) | N/A | **_Yes_** | +| MYSQL_NET| Network type (e.g. unix) | tcp | No | +| MYSQL_USER| User to authenticate with | root | No | +| MYSQL_PASSWD| Password to authenticate with | no password | No | +| MYSQL_DB| Name of the database to use | tavern | No | +| MYSQL_MAX_IDLE_CONNS | Integer value of max idle mysql connections to keep open | 10 | No | +| MYSQL_MAX_OPEN_CONNS | Integer value of max mysql connections to open | 100 | No | +| MYSQL_MAX_CONN_LIFETIME | Integer value of max mysql connection lifetime (in seconds) | 3600 | No | + +Here is an example of running Tavern locally with a MySQL backend: + +```sh +MYSQL_USER="admin" MYSQL_ADDR="127.0.0.1:3306" go run ./tavern +``` + +When no value is set for `MYSQL_ADDR`, the default SQLite backend is used: + +```sh +MYSQL_USER="admin" go run ./tavern/ +2022/03/08 05:46:06 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite +``` + +### OAuth + +By default, user authentication is disabled for Tavern. This means that anyone can login and be granted a session. To restrict who is able to access your deployment, Tavern supports OAuth configuration (using [Google OAuth](https://developers.google.com/identity/protocols/oauth2)). + +To obtain a client_id and a client_secret for Google OAuth, please follow [their instructions](https://developers.google.com/identity/sign-in/web/sign-in#create_authorization_credentials) to create an application. + +The following environment variables are required for OAuth Configuration: + +| Env Var | Description | +| ------- | ----------- | +| OAUTH_CLIENT_ID | The [OAuth client_id](https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/) Tavern will use to communicate with an identity provider (Google) | +| OAUTH_CLIENT_SECRET | The [OAuth client_secret](https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/) Tavern will use to authenticate to an identity provider (Google) | +| OAUTH_DOMAIN | The domain Tavern is being hosted at, that the identity provider (Google) should redirect users to after completing the consent flow | + +Here is an example of running Tavern locally with OAuth configured: + +```sh +OAUTH_CLIENT_ID=123 OAUTH_CLIENT_SECRET=456 OAUTH_DOMAIN=127.0.0.1 go run ./tavern +2022/03/09 05:32:58 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite +2022/03/09 05:32:58 listening on 0.0.0.0:80 +``` + +When no OAuth configuration is provided, authentication is disabled: + +```sh +go run ./tavern +2022/03/09 05:24:43 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite +2022/03/09 05:24:43 WARNING: OAuth is not configured, authentication disabled +``` + +When partial OAuth configuration is provided, Tavern will error. This is to protect against inadvertently starting Tavern with authentication disabled. + +```sh +OAUTH_CLIENT_ID=123 go run ./tavern +2022/03/09 05:31:46 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite +2022/03/09 05:31:46 [FATAL] To configure OAuth, must provide value for environment var 'OAUTH_CLIENT_SECRET' +exit status 1 +``` + +#### How it Works + +Tavern hosts two endpoints to support OAuth: + +* A login handler (`/oauth/login`) which redirects users to Google's OAuth consent flow + * This endpoint sets a JWT cookie for the user, such that the [OAuth state parameter](https://auth0.com/docs/secure/attack-protection/state-parameters#csrf-attacks) can safely be verified later to prevent against CSRF attacks + * Currently the keys used to sign and verify JWTs are generated at server start, meaning if the server is restarted while a user is in the middle of an OAuth flow, it will fail and the user will need to restart the flow +* An authorization handler (`/oauth/authorize`) which users are redirected to by Google after completing Google's OAuth consent flow + * This handler is responsible for obtaining a user's profile information from Google using an OAuth access token, and creates the user's account if it does not exist yet + +##### Trust on First Use + +Tavern supports a Trust on First Use (TOFU) authentication model, meaning the first user to successfully authenticate will be granted admin permissions. Subsequent users that login will have accounts created, but will require activation before they can interact with any Tavern APIs. Only admin users may activate other users. + +##### CLI Application Authentication + +Tavern supports `access_tokens` in place of a session cookie for authentication, intended for use by CLI applications that require authenticated access to the Tavern API. Currently, we only support Golang clients via the provided `realm.pub/tavern/cli/auth` package. This package relies on the user's existing browser session (requiring OAuth) to relay authentication credentials (an `access_token`) to the application. Your CLI package may wish to cache these credentials securely to avoid unnecessary reauthentication. Here is an example of using this package: + +```golang +package main + +import ( + "context" + "fmt" + "net/http" + "time" + + "github.com/pkg/browser" + "realm.pub/tavern/cli/auth" +) + +func main() { + // Set Timeout (includes time for user login via browser) + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) + defer cancel() + + // Setup your Tavern URL (e.g. from env vars) + tavernURL := "http://127.0.0.1" + + // Configure Browser (uses the default system browser) + browser := auth.BrowserFunc(browser.OpenURL) + + // Open Browser and Obtain Access Token (via 127.0.0.1 redirect) + token, err := auth.Authenticate(ctx, browser, tavernURL) + if err != nil { + panic(err) + } + + // Example Tavern HTTP Request + req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/status", tavernURL), nil) + if err != nil { + panic(err) + } + + // Authenticate Request + token.Authenticate(req) + + // Send the request etc... +} +``` + +If you require authenticated access to the Tavern API outside of Golang, you may implement something similar to what this package does under the hood. **THIS IS NOT SUPPORTED**. We highly recommend relying on the provided Golang package instead, as our Authentication API may change and cause outages for your application. + +### Test Data + +Running Tavern with the `ENABLE_TEST_DATA` environment variable set will populate the database with test data. This is useful for UI development, testing, or just interacting with Tavern and seeing how it works. + +```sh +ENABLE_TEST_DATA=1 go run ./tavern +2023/02/24 01:02:37 [WARN] MySQL is not configured, using SQLite +2023/02/24 01:02:37 [WARN] OAuth is not configured, authentication disabled +2023/02/24 01:02:37 [WARN] Test data is enabled +2023/02/24 01:02:37 Starting HTTP server on 0.0.0.0:80 +``` + +### PPROF + +Running Tavern with the `ENABLE_PPROF` environment variable set will enable performance profiling information to be collected and accessible. This should never be set for a production deployment as it will be unauthenticated and may provide access to sensitive information, it is intended for development purposes only. Read more on how to use `pprof` with tavern in the [Developer Guide](/dev-guide/tavern#performance-profiling). + +## Build and publish tavern container + +If you want to deploy tavern without using the published version you'll have to build and publish your own container. + +### Build your container + +```bash +cd ./realm +docker build --tag tavern:dev --file ./docker/tavern.Dockerfile . +``` + +### Publish your container to docker hub + +If you haven't before [sign-up for a docker hub account](https://hub.docker.com/signup) and login with the CLI `docker login` + +```bash +docker tag tavern:dev /tavern:dev +docker push /tavern:dev +``` + +### Specify your container during terraform deploy + +```bash +terraform apply -var="gcp_project=" -var="oauth_client_id=" -var="oauth_client_secret=" -var="oauth_domain=" -var="tavern_container_image=/tavern:dev" +``` + +## User Interface + +## GraphQL API + +### Playground + +If you'd like to explore the Graph API and try out some queries, head to the `/graphiql` endpoint of your Tavern deployment. This endpoint exposes an interactive playground for you to experiment with GraphQL queries. Currently, this is used to fill gaps between developed backend functionality and in-development frontend functionality. + +![/assets/img/tavern/graphiql.png](/assets/img/tavern/graphiql.png) + +#### Some sample queries to get started + +##### List all beacons + +```graphql +query listBeacons { + beacons { + id + identifier + name + host { + platform + identifier + primaryIP + } + } +} +``` + +##### Create a tome + +```graphql +mutation CreateTome ($input: CreateTomeInput!) { + createTome(input: $input) { + id + name + } +} +``` + +```json +{ + "input": { + "name": "Test tome", + "author":"Hulto", + "description": "Just a sample", + "paramDefs": "[{\"name\":\"print_string\",\"label\":\"Print String\",\"type\":\"string\",\"placeholder\":\"A message to print\"}]", + "eldritch": "print(input_params['print_string'])", + "fileIDs": [], + "tactic": "IMPACT" + } +} +``` + +##### Create a task + +```graphql +mutation createQuest($input: CreateQuestInput!, $beaconIDs:[ID!]!){ + createQuest(input: $input, beaconIDs: $beaconIDs) { + id + } +} +``` + +```json +{ + "input": { + "name": "Run test tome", + "tomeID": "21474836488", + "parameters": "{\"print_string\":\"Hello World\"}" + }, + "beaconIDs": ["8589934593"] +} +``` + +##### Get all task and quest output + +```graphql +query get_task_res { + quests { + tasks { + id + output + quest { + tome { + eldritch + } + parameters + } + execFinishedAt + } + } +} +``` diff --git a/docs/_docs/dev-guide/introduction.md b/docs/_docs/dev-guide/introduction.md index f62d84527..6d417dbcf 100644 --- a/docs/_docs/dev-guide/introduction.md +++ b/docs/_docs/dev-guide/introduction.md @@ -54,38 +54,6 @@ At the time of writing, the Tavern UI is still in an early stage, and therefore In an attempt to reduce the complexity of merges, we enforce a linear history for Realm. This means that when your PR is merged, a "squash & merge" will be enforced so that only one commit is added onto the main branch. This means you can feel free to commit and push as often as you'd like, since all of your commits will be combined before merging your final changes. -# Terminology - -Throughout the documentation terms like "agent" or "implant" are used to reference various components (or types of components) in our codebase. Below we attempt to define some of those terms, to add some clarity to that other documentation. - -### Host - -A Host is a system that is in-scope for the current engagement. It is used to establish a logical boundary between different systems in an engagement (e.g. between a webserver and a database). This enables operations to target a particular system, for example you may want to list files on a web server in your engagement scope. - -### Implant - -References malicious code or persistence mechanisms that are deployed to compromise target systems. - -### Agent - -An Agent is a type of implant which retrieves execution instructions by connecting to our backend infrastructure (calling back) and querying for new tasks. - -### Beacon - -A Beacon is a running instance of an Agent. A Host may have multiple active Beacons that use the same underlying Agent. - -### Task - -A Task represents a set of instructions for an Agent to perform. For example, listing files could be a Task. When listing files across various Beacons, one Task per Beacon will be created for tracking the individual execution output. - -### Eldritch - -Eldritch is our Pythonic Domain Specific Language (DSL), which can be used to progammatically define red team operations. Many of the language's built-in features do not rely on system binaries. For more information, please see the [Eldritch section](/user-guide/eldritch) of the documentation. - -### Tome - -A Tome is a prebuilt Eldritch bundle, which provides execution instructions to a Beacon. Tomes can embed files and accept parameters to change their behavior at runtime. Tavern's built-in Tomes are defined [here](https://github.com/spellshift/realm/tree/main/tavern/tomes). - # Project Structure * **[.devcontainer](https://github.com/spellshift/realm/tree/main/.devcontainer)** contains settings required for configuring a VSCode dev container that can be used for Realm development diff --git a/docs/_docs/dev-guide/tavern.md b/docs/_docs/dev-guide/tavern.md index 45600f456..dc5ee5312 100644 --- a/docs/_docs/dev-guide/tavern.md +++ b/docs/_docs/dev-guide/tavern.md @@ -8,286 +8,10 @@ permalink: dev-guide/tavern ## Overview -Tavern is a teamserver for Realm, providing a UI to control deployments and implants during an engagement. The majority of Tavern's functionality is exposed through a GraphQL API, which is used by both implants and the UI. - -If you would like to help contribute to Tavern, please take a look at our [open issues](https://github.com/spellshift/realm/issues?q=is%3Aopen+is%3Aissue+label%3Atavern). - -## Configuration - -### Metrics - -By default, Tavern does not export metrics. You may use the below environment configuration variables to enable [Prometheus](https://prometheus.io/docs/introduction/overview/) metric collection. These metrics become available at the "/metrics" endpoint configured. These metrics are hosted on a separate HTTP server such that it can be restricted to localhost (default). This is because the endpoint is unauthenticated, and would leak sensitive information if it was accessible. - -| Env Var | Description | Default | Required | -| ------- | ----------- | ------- | -------- | -| ENABLE_METRICS | Set to any value to enable the "/metrics" endpoint. | Disabled | No | -| HTTP_METRICS_LISTEN_ADDR | Listen address for the metrics HTTP server, it must be different than the value of `HTTP_LISTEN_ADDR`. | `127.0.0.1:8080` | No | - -### MySQL - -By default, Tavern operates an in-memory SQLite database. To persist data, a MySQL backend is supported. In order to configure Tavern to use MySQL, the `MYSQL_ADDR` environment variable must be set to the `host[:port]` of the database (e.g. `127.0.0.1`, `mydb.com`, or `mydb.com:3306`). You can reference the [mysql.Config](https://pkg.go.dev/github.com/go-sql-driver/mysql#Config) for additional information about Tavern's MySQL configuration. - -The following environment variables are currently supported for additional MySQL Configuration: - -| Env Var | Description | Default | Required | -| ------- | ----------- | ------- | -------- | -| MYSQL_ADDR| Address of the MySQL server (e.g. `host[:port]`) | N/A | **_Yes_** | -| MYSQL_NET| Network type (e.g. unix) | tcp | No | -| MYSQL_USER| User to authenticate with | root | No | -| MYSQL_PASSWD| Password to authenticate with | no password | No | -| MYSQL_DB| Name of the database to use | tavern | No | -| MYSQL_MAX_IDLE_CONNS | Integer value of max idle mysql connections to keep open | 10 | No | -| MYSQL_MAX_OPEN_CONNS | Integer value of max mysql connections to open | 100 | No | -| MYSQL_MAX_CONN_LIFETIME | Integer value of max mysql connection lifetime (in seconds) | 3600 | No | - -Here is an example of running Tavern locally with a MySQL backend: - -```sh -MYSQL_USER="admin" MYSQL_ADDR="127.0.0.1:3306" go run ./tavern -``` - -When no value is set for `MYSQL_ADDR`, the default SQLite backend is used: - -```sh -MYSQL_USER="admin" go run ./tavern/ -2022/03/08 05:46:06 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite -``` - -### OAuth - -By default, user authentication is disabled for Tavern. This means that anyone can login and be granted a session. To restrict who is able to access your deployment, Tavern supports OAuth configuration (using [Google OAuth](https://developers.google.com/identity/protocols/oauth2)). - -To obtain a client_id and a client_secret for Google OAuth, please follow [their instructions](https://developers.google.com/identity/sign-in/web/sign-in#create_authorization_credentials) to create an application. - -The following environment variables are required for OAuth Configuration: - -| Env Var | Description | -| ------- | ----------- | -| OAUTH_CLIENT_ID | The [OAuth client_id](https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/) Tavern will use to communicate with an identity provider (Google) | -| OAUTH_CLIENT_SECRET | The [OAuth client_secret](https://www.oauth.com/oauth2-servers/client-registration/client-id-secret/) Tavern will use to authenticate to an identity provider (Google) | -| OAUTH_DOMAIN | The domain Tavern is being hosted at, that the identity provider (Google) should redirect users to after completing the consent flow | - -Here is an example of running Tavern locally with OAuth configured: - -```sh -OAUTH_CLIENT_ID=123 OAUTH_CLIENT_SECRET=456 OAUTH_DOMAIN=127.0.0.1 go run ./tavern -2022/03/09 05:32:58 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite -2022/03/09 05:32:58 listening on 0.0.0.0:80 -``` - -When no OAuth configuration is provided, authentication is disabled: - -```sh -go run ./tavern -2022/03/09 05:24:43 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite -2022/03/09 05:24:43 WARNING: OAuth is not configured, authentication disabled -``` - -When partial OAuth configuration is provided, Tavern will error. This is to protect against inadvertently starting Tavern with authentication disabled. - -```sh -OAUTH_CLIENT_ID=123 go run ./tavern -2022/03/09 05:31:46 no value found for environment var 'MYSQL_ADDR', starting tavern with SQLite -2022/03/09 05:31:46 [FATAL] To configure OAuth, must provide value for environment var 'OAUTH_CLIENT_SECRET' -exit status 1 -``` - -#### How it Works - -Tavern hosts two endpoints to support OAuth: - -* A login handler (`/oauth/login`) which redirects users to Google's OAuth consent flow - * This endpoint sets a JWT cookie for the user, such that the [OAuth state parameter](https://auth0.com/docs/secure/attack-protection/state-parameters#csrf-attacks) can safely be verified later to prevent against CSRF attacks - * Currently the keys used to sign and verify JWTs are generated at server start, meaning if the server is restarted while a user is in the middle of an OAuth flow, it will fail and the user will need to restart the flow -* An authorization handler (`/oauth/authorize`) which users are redirected to by Google after completing Google's OAuth consent flow - * This handler is responsible for obtaining a user's profile information from Google using an OAuth access token, and creates the user's account if it does not exist yet - -##### Trust on First Use - -Tavern supports a Trust on First Use (TOFU) authentication model, meaning the first user to successfully authenticate will be granted admin permissions. Subsequent users that login will have accounts created, but will require activation before they can interact with any Tavern APIs. Only admin users may activate other users. - -##### CLI Application Authentication - -Tavern supports `access_tokens` in place of a session cookie for authentication, intended for use by CLI applications that require authenticated access to the Tavern API. Currently, we only support Golang clients via the provided `realm.pub/tavern/cli/auth` package. This package relies on the user's existing browser session (requiring OAuth) to relay authentication credentials (an `access_token`) to the application. Your CLI package may wish to cache these credentials securely to avoid unnecessary reauthentication. Here is an example of using this package: - -```golang -package main - -import ( - "context" - "fmt" - "net/http" - "time" - - "github.com/pkg/browser" - "realm.pub/tavern/cli/auth" -) - -func main() { - // Set Timeout (includes time for user login via browser) - ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) - defer cancel() - - // Setup your Tavern URL (e.g. from env vars) - tavernURL := "http://127.0.0.1" - - // Configure Browser (uses the default system browser) - browser := auth.BrowserFunc(browser.OpenURL) - - // Open Browser and Obtain Access Token (via 127.0.0.1 redirect) - token, err := auth.Authenticate(ctx, browser, tavernURL) - if err != nil { - panic(err) - } - - // Example Tavern HTTP Request - req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("%s/status", tavernURL), nil) - if err != nil { - panic(err) - } - - // Authenticate Request - token.Authenticate(req) - - // Send the request etc... -} -``` - -If you require authenticated access to the Tavern API outside of Golang, you may implement something similar to what this package does under the hood. **THIS IS NOT SUPPORTED**. We highly recommend relying on the provided Golang package instead, as our Authentication API may change and cause outages for your application. - -### Test Data - -Running Tavern with the `ENABLE_TEST_DATA` environment variable set will populate the database with test data. This is useful for UI development, testing, or just interacting with Tavern and seeing how it works. - -```sh -ENABLE_TEST_DATA=1 go run ./tavern -2023/02/24 01:02:37 [WARN] MySQL is not configured, using SQLite -2023/02/24 01:02:37 [WARN] OAuth is not configured, authentication disabled -2023/02/24 01:02:37 [WARN] Test data is enabled -2023/02/24 01:02:37 Starting HTTP server on 0.0.0.0:80 -``` - -### PPROF - -Running Tavern with the `ENABLE_PPROF` environment variable set will enable performance profiling information to be collected and accessible. This should never be set for a production deployment as it will be unauthenticated and may provide access to sensitive information, it is intended for development purposes only. Read more on how to use `pprof` with tavern under the [Performance Profiling](#performance-profiling) section of this guide. - -## Build and publish tavern container - -If you want to deploy tavern without using the published version you'll have to build and publish your own container. - -### Build your container - -```bash -cd ./realm -docker build --tag tavern:dev --file ./docker/tavern.Dockerfile . -``` - -### Publish your container to docker hub - -If you haven't before [sign-up for a docker hub account](https://hub.docker.com/signup) and login with the CLI `docker login` - -```bash -docker tag tavern:dev /tavern:dev -docker push /tavern:dev -``` - -### Specify your container during terraform deploy - -```bash -terraform apply -var="gcp_project=" -var="oauth_client_id=" -var="oauth_client_secret=" -var="oauth_domain=" -var="tavern_container_image=/tavern:dev" -``` - -## User Interface +Before reading this guide, please check out the [admin guide](/admin-guide/tavern) to better understand how Tavern is deployed and managed. If you would like to help contribute to Tavern, please take a look at our [open issues](https://github.com/spellshift/realm/issues?q=is%3Aopen+is%3Aissue+label%3Atavern). ## GraphQL API -### Playground - -If you'd like to explore the Graph API and try out some queries, head to the `/graphiql` endpoint of your Tavern deployment. This endpoint exposes an interactive playground for you to experiment with GraphQL queries. - -![/assets/img/tavern/graphiql.png](/assets/img/tavern/graphiql.png) - -#### Some sample queries to get started - -##### List all beacons - -```graphql -query get_beacons { - beacons { - id - identifier - name - hostPlatform - hostIdentifier - hostPrimaryIP - } -} -``` - -##### Create a tome - -```graphql -mutation CreateTome ($input: CreateTomeInput!) { - createTome(input: $input) { - id - name - } -} -``` - -```json -{ - "input": { - "name": "Test tome", - "description": "Just a sample", - "eldritch": "print(input_params['print_string'])", - "fileIDs": [], - "paramDefs": "[{\"name\":\"print_string\",\"label\":\"Print String\",\"type\":\"string\",\"placeholder\":\"A message to print\"}]" - } -} -``` - -##### Create a task - -```graphql -mutation createQuest($input: CreateQuestInput!, $beaconIDs:[ID!]!){ - createQuest(input: $input, beaconIDs: $beaconIDs) { - id - } -} -``` - -```json -{ - "input": { - "name": "Run test tome", - "tomeID": "21474836488", - "parameters": "{\"print_string\":\"Hello World\"}" - }, - "sess": ["8589934593"] -} -``` - -##### Get all task and quest output - -```graphql -query get_task_res { - quests { - tasks { - id - output - quest { - tome { - eldritch - } - parameters - } - execFinishedAt - } - } -} -``` - ### Creating a New Model 1. Initialize the schema `cd tavern && go run entgo.io/ent/cmd/ent init ` diff --git a/docs/_docs/user-guide/getting-started.md b/docs/_docs/user-guide/getting-started.md index 52d9b2e39..93c73405c 100644 --- a/docs/_docs/user-guide/getting-started.md +++ b/docs/_docs/user-guide/getting-started.md @@ -8,23 +8,92 @@ permalink: user-guide/getting-started ## Getting Started -*To deploy a production ready instance see the [tavern setup guide](https://docs.realm.pub/user-guide/tavern).* +Welcome, intrepid red teamer! This guide will launch you into a sandboxed Realm experience, perfect for experimenting and exploring its features. Think of it as your personal training ground before unleashing it in real-world engagements. But first, let's make sure you're in the right place. -### Start the server +* Want to deploy Realm for production? Check out our [deployment guide](/admin-guide/tavern#deployment). +* Already using a deployed Realm instance? Check out our [UI guide](/user-guide/ui). + +Otherwise, let's delve into Realm! Break, bend, and explore to your heart's content. This is your chance to unlock Realm's potential and get comfortable before your next engagement. Time for an adventure! + +### Tavern (C2 / Teamserver) + +In the bustling world of Realm, the Tavern serves as your red team's central operations hub. It is your virtual war room, managing [Beacons](/user-guide/terminology#beacon), those carefully planted footholds within your target landscape. Think of them as listening posts, constantly relaying intel back to the Tavern and awaiting [Quests](/user-guide/terminology#quest). + +But the Tavern is more than just a C2. Imagine it as a vast, ever-evolving map of the target environment, a knowledge graph meticulously built from collected data. It will help you navigate the terrain, identify vulnerabilities, and plan your next move with strategic precision. Remember, a well-stocked Tavern paves the way for successful campaigns! + +To start Tavern, open a terminal and run the following: ```bash git clone https://github.com/spellshift/realm.git cd realm && go run ./tavern - -# If you'd like to test without deploying an agent use the test data. -ENABLE_TEST_DATA=1 go run ./tavern ``` -### Start the agent +![starting-tavern](/assets/img/user-guide/getting-started/starting-tavern.png) + +The warnings you see here indicate that there are settings recommended for production environments that have not been configured. For now, it's safe to ignore these and continue on. + +### Imix (Agent / Beacon) + +[Imix](/user-guide/imix) is the supported [Agent](/user-guide/terminology#agent) for Realm. This fiery trickster resides within your target network, eagerly awaiting your commands whispered through magical [Eldritch](/user-guide/terminology#eldritch) [Tomes](/user-guide/terminology#tome). These [Tomes](/user-guide/terminology#tome), inspired by [MITRE ATT&CK tactics](https://attack.mitre.org/matrices/enterprise/), empower [Imix](/user-guide/imix) to scout, exploit weaknesses, and launch attacks against target systems. ```bash -git clone https://github.com/spellshift/realm.git -cd realm/implants/imix && cargo run +# Assumes you have already cloned the repository and are in the 'realm' directory +cd ./implants/imix && cargo run ``` -Want to work with the API? Check out the [sample queries](https://docs.realm.pub/dev-guide/tavern#graphql-api) +![starting-imix](/assets/img/user-guide/getting-started/starting-imix.png) + +Here we run the `imix` [Agent](/user-guide/terminology#agent), and with this debug-mode, it will log information to keep you informed of its operations. When built for production, `imix` does not generate these logs (see [our imix guide](/user-guide/imix)). + +You'll notice two key components of the configuration: + +* `callback_uri` - Defines the URI for Tavern, where this [Beacon](/user-guide/terminology#beacon) should callback to. +* `interval` - Defines how often this [Beacon](/user-guide/terminology#beacon) should perform callbacks (in seconds). + +These configurations can be controlled via Environment Variables at `imix` compile-time. See the [imix configuration docs](/user-guide/imix#configuration) for more information. + +### Quests + +Now it's time to provide our [Beacon](/user-guide/terminology#beacon) it's first [Task](/user-guide/terminology#task). We do this, by creating a [Quest](/user-guide/terminology#quest) in the UI, which represents a collection of [Tasks](/user-guide/terminology#task) across one or more [Hosts](/user-guide/terminology#host). Let's open our UI, which should be available at [http://127.0.0.1:80/](http://127.0.0.1:80/). + +#### Beacon Selection + +Now from the left navigation menu, select "Create new quest". + +![create-new-quest](/assets/img/user-guide/getting-started/create-new-quest.png) + +This view will show all of our active [Beacons](/user-guide/terminology#beacon) available for tasking. For more details on filtering and selecting [Beacons](/user-guide/terminology#beacon), see our [UI guide](/user-guide/ui). For now, simply select your creatively-named [Beacon](/user-guide/terminology#beacon) and click "Continue". + +#### Tome Selection + +A [Tome](/user-guide/terminology#tome) is an [Eldritch](/user-guide/terminology#eldritch) package ready for execution by `imix`. By default, Tavern includes several core [Tomes](/user-guide/terminology#tome) to get you started. But don't worry, for more customization and advanced capabilities, it's easy to write your own [Tomes](/user-guide/terminology#tome)! These [Tomes](/user-guide/terminology#tome) are designed to be cross-platform, so it shouldn't matter which platform you're running on. You can view the underlying [Eldritch](/user-guide/terminology#eldritch) code a [Tome](/user-guide/terminology#tome) will run by clicking on the "details" dropdown. Feel free to take a minute to read through some of the core [Tomes](/user-guide/terminology#tome) available to you. + +![select-tome](/assets/img/user-guide/getting-started/select-tome.png) + +Let's select "hostname", which will simply print the hostname of the [Host](/user-guide/terminology#host) `imix` is running on. You may filter [Tomes](/user-guide/terminology#tome) using the search bar above. Certain [Tomes](/user-guide/terminology#tome) will allow / require you to specify parameters to inform their execution, which will be displayed in this step (for relevant [Tomes](/user-guide/terminology#tome)). When you're ready, select "Continue". + +#### Confirm and Submit + +Lastly, we'll be greeted with a prompt displaying a summary of the [Quest](/user-guide/terminology#quest) you're about to launch. Check that everything looks correct, and click "Submit"! + +![confirm-quest](/assets/img/user-guide/getting-started/confirm-quest.png) + +#### Results + +Now, in your `imix` logs you'll see that when it calls back, it will obtain the [Tome](/user-guide/terminology#tome) from Tavern and evaluate it. On it's next callback, it will report the results. + +![imix-tome-eval](/assets/img/user-guide/getting-started/imix-tome-eval.png) + +By refreshing the UI (polling not yet supported), you should see the output is now available! + +![quest-result](/assets/img/user-guide/getting-started/quest-result.png) + +This concludes your initial foray into Realm, but the true adventure lies ahead. Having navigated this introduction, you now possess the foundational knowledge to embark on your red teaming journey with Realm. + +**Remember:** + +* **Craft Strategic Campaigns:** Leverage [Eldritch](/user-guide/terminology#eldritch) [Tomes](/user-guide/terminology#tome) to orchestrate Imix's actions, aligning them with your specific objectives. +* **Analyze and Adapt:** Use the ever-expanding knowledge graph to inform your decision-making and refine your attack strategy. +* **Harness the Community:** Engage with the growing Realm community to share knowledge, gain insights, and contribute to its evolution. + +Farewell, adventurer, and may your [Quests](/user-guide/terminology#quest) with [Imix](/user-guide/imix) be fruitful and fiery! diff --git a/docs/_docs/user-guide/tavern.md b/docs/_docs/user-guide/tavern.md deleted file mode 100644 index b733e499f..000000000 --- a/docs/_docs/user-guide/tavern.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -title: Tavern -tags: - - User Guide -description: Tavern User Guide -permalink: user-guide/tavern ---- - -## Overview - -Welcome to Tavern! - -This section outlines some basic usage and gotchas. - -### Creating quests - -Quests are how you interact with beacons think actions/tasks in other c2s. - -Quests by default are group actions so scaling your activity is easy and built-in. - -Each quest is made up of three main parts: -- Beacons - What you'll be executing on -- Tomes - What you'll be executing -- Input parameters - arguments passed to the tome and eldritch interpreter as `input_params{}` - -#### To start click "Create new quest" - -First you'll be prompted to select beacons you want this quest to apply to. - -Beacons can be searched by name, group tag, and service tag. - -Just start typing what you're looking for and Tavern will search across all three fields. - -Now you can select individual beacons or use the "Select all" button to add everything that matches your search criteria. - -*Note: `Service + group` searches are and'd while `service + service` or `group + group` searches are or'd.* - -#### Next select the tome you want to run. - -Tomes can be searched by name - -If the tome has parameters you'll be prompted. Consult the placeholder text if you're unsure how to format the input. - -#### Finally review and confirm your quest - -The final page gives an overview of what beacons, tome and parameters have been selected. - -Lastly give your quest a descriptive name so you can reference back to it later if you need to lookup the information again. (We recommend this so you can avoid duplicating work and exposure on the system). - -## Deployment - -This section will walk you through deploying a production ready instance of Tavern to GCP. If you're just looking to play around with Tavern, feel free to run the [docker image (kcarretto/tavern:latest)](https://hub.docker.com/repository/docker/kcarretto/tavern/general) locally. - -### 1. Create a GCP Project - -Navigate to the [GCP Console](https://console.cloud.google.com/) and [create a new GCP project](https://console.cloud.google.com/projectcreate). -![assets/img/tavern/deploy/create-gcp-project.png](/assets/img/tavern/deploy/create-gcp-project.png) - -Make a note of the created Project ID as you'll need that in a later step -![assets/img/tavern/deploy/gcp-project-info.png](/assets/img/tavern/deploy/gcp-project-info.png) - -### 2. Setup OAuth (Optional) - -_Note: These setup instructions assume you own a domain which you would like to host Tavern at._ - -If you want to configure OAuth for your Tavern Deployment, navigate to the [GCP OAuth Consent Screen](https://console.cloud.google.com/apis/credentials/consent) and create a new External consent flow. **If you do not configure OAuth, Tavern will not perform any authentication or authorization for requests.** - -![assets/img/tavern/deploy/gcp-new-oauth-consent.png](/assets/img/tavern/deploy/gcp-new-oauth-consent.png) - -Provide details that users will see when logging into Tavern, for example: - -* App Name: "Tavern" -* User Support Email: "YOUR_EMAIL@EXAMPLE.COM" -* App Logo: Upload something cool if you'd like, but then you'll need to complete a verification process. -* App Domain: "https://tavern.mydomain.com/" -* Authorized Domains: "mydomain.com" -* Developer Contact Information: "YOUR_EMAIL@EXAMPLE.COM" - -Add the ".../auth/userinfo.profile" scope, used by Tavern to obtain user names and photourls. -![assets/img/tavern/deploy/gcp-oauth-scope.png](/assets/img/tavern/deploy/gcp-oauth-scope.png) - -Next, add yourself as a "Test User". **Until you publish your app, only test users may complete the OAuth consent flow.** If you didn't select any options that require verification, you may publish your app now (so you won't need to allowlist the users for your application). - -Navigate to the [Credentials Tool](https://console.cloud.google.com/apis/credentials) and select "Create Credentials" -> "OAuth client ID". Be sure to add an "Authorized redirect URI" so that the consent flow redirects to the appropriate Tavern endpoint. For example "mydomain.com/oauth/authorize". Save the resulting Client ID and Client secret for later. -![assets/img/tavern/deploy/oauth-new-creds.png](/assets/img/tavern/deploy/oauth-new-creds.png) - -Next, configure a CNAME record for the domain you'd like to host Tavern at (e.g. "tavern.mydomain.com") to point to "ghs.googlehosted.com.". -![assets/img/tavern/deploy/google-dns-cname.png](/assets/img/tavern/deploy/google-dns-cname.png) - -And that's it! In the below sections on deployment, please ensure you properly configure your OAUTH_CLIENT_ID, OAUTH_CLIENT_SECRET, and OAUTH_DOMAIN to ensure Tavern is properly configured. - -### 3. Google Cloud CLI - -Follow [these instructions](https://cloud.google.com/sdk/docs/install) to install the gcloud CLI. This will enable you to quickly obtain credentials that terraform will use to authenticate. Alternatively, you may create a service account (with appropriate permissions) and obtain [Application Default Credentials](https://cloud.google.com/sdk/gcloud/reference/auth/application-default) for it. See [these Authentication Instructions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#authentication) for more information on how to configure GCP authentication for Terraform. - -After installing the gcloud CLI, run `gcloud auth application-default login` to obtain Application Default Credentials. - -### 4. Terraform - -1. Follow [these instructions](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) to install the Terraform CLI. -2. Clone [the repo](https://github.com/spellshift/realm) and navigate to the `terraform` directory. -3. Run `terraform init` to install the Google provider for terraform. -4. Run `terraform apply -var="gcp_project=" -var="oauth_client_id=" -var="oauth_client_secret=" -var="oauth_domain="` to deploy Tavern! - -**Example:** - -```sh -terraform apply -var="gcp_project=new-realm-deployment" -var="oauth_client_id=12345.apps.googleusercontent.com" -var="oauth_client_secret=ABCDEFG" -var="oauth_domain=test-tavern.redteam.toys" -``` - -After terraform completes successfully, head to the [DNS mappings for Cloud Run](https://console.cloud.google.com/run/domains) and wait for a certificate to successfully provision. This may take a while, so go enjoy a nice cup of coffee ☕ - -After your certificate has successfully provisioned, it may still take a while (e.g. an hour or two) before you are able to visit Tavern using your custom OAuth Domain (if configured). - -#### CLI Variables - -|Name|Required|Description| -|----|--------|-----------| -|gcp_project|Yes|Project ID of the GCP Project created in step 1.| -|gcp_region|No|Region to deploy to.| -|mysql_user|No|The MySQL user to create and connect with.| -|mysql_passwd|No|The MySQL password to set and connect with. Autogenerated by default. | -|mysql_dbname|No|MySQL Database to create and connect to.| -|mysql_tier|No|The type of instance to run the Cloud SQL Database on.| -|oauth_client_id|Only if OAuth is configured|The OAuth ClientID Tavern will use to connect to the IDP (Google).| -|oauth_client_secret|Only if OAuth is configured|The OAuth Client Secret Tavern will use to connect to the IDP (Google).| -|oauth_domain|Only if OAuth is configured|The OAuth Domain that the IDP should redirect to e.g. tavern.mydomain.com (should be the domain you set a CNAME record for while configuring OAuth).| -|min_scale|The minimum number of containers to run, if set to 0 you may see cold boot latency.| -|max_scale|The maximum number of containers to run.| - -### Manual Deployment Tips - -Below are some deployment gotchas and notes that we try to address with Terraform, but can be a bit tricky if trying to deploy Tavern manually. - -* MySQL version 8.0 must be started with the flag `default_authentication_plugin=caching_sha2_password` for authentication to work properly. A new user must be created for authentication. -* When running in CloudRun, it's best to connect to CloudSQL via a unix socket (so ensure the `MYSQL_NET` env var is set to "unix" ). - * After adding a CloudSQL connection to your CloudRun instance, this unix socket is available at `/cloudsql/` (e.g. `/cloudsql/realm-379301:us-east4:tavern-db`). -* You must create a new database in your CloudSQL instance before launching Tavern and ensure the `MYSQL_DB` env var is set accordingly. diff --git a/docs/_docs/user-guide/terminology.md b/docs/_docs/user-guide/terminology.md new file mode 100644 index 000000000..970c247bf --- /dev/null +++ b/docs/_docs/user-guide/terminology.md @@ -0,0 +1,43 @@ +--- +title: Terminology +tags: + - User Guide +description: Various terms used throughout Realm. +permalink: user-guide/terminology +--- + +## Terminology + +Throughout the documentation terms like "agent" or "implant" are used to reference various components of Realm. Below we attempt to define some of those terms, to add some clarity to that other documentation. + +### Host + +A Host is a system that is in-scope for the current engagement. It is used to establish a logical boundary between different systems in an engagement (e.g. between a webserver and a database). This enables operations to target a particular system, for example you may want to list files on a web server in your engagement scope. + +### Implant + +References malicious code or persistence mechanisms that are deployed to compromise target systems. Imix is the primary implant used with Realm. + +### Agent + +An Agent is a type of implant which retrieves execution instructions by connecting to our backend infrastructure (calling back) and querying for new tasks. + +### Beacon + +A Beacon is a running instance of an Agent. A Host may have multiple active Beacons that use the same underlying Agent. + +### Task + +A Task represents a set of instructions for an Agent to perform. For example, listing files could be a Task. When listing files across various Beacons, one Task per Beacon will be created for tracking the individual execution output. + +### Quest + +A Quest represents a collection of tasks, each with a unique host. This is how Realm enables multi-host management. + +### Eldritch + +Eldritch is our Pythonic Domain Specific Language (DSL), which can be used to progammatically define red team operations. Many of the language's built-in features do not rely on system binaries. For more information, please see the [Eldritch section](/user-guide/eldritch) of the documentation. + +### Tome + +A Tome is a prebuilt Eldritch bundle, which provides execution instructions to a Beacon. Tomes can embed files and accept parameters to change their behavior at runtime. Tavern's built-in Tomes are defined [here](https://github.com/spellshift/realm/tree/main/tavern/tomes). diff --git a/docs/assets/img/user-guide/getting-started/confirm-quest.png b/docs/assets/img/user-guide/getting-started/confirm-quest.png new file mode 100644 index 000000000..1d373ef6a Binary files /dev/null and b/docs/assets/img/user-guide/getting-started/confirm-quest.png differ diff --git a/docs/assets/img/user-guide/getting-started/create-new-quest.png b/docs/assets/img/user-guide/getting-started/create-new-quest.png new file mode 100644 index 000000000..293fe2791 Binary files /dev/null and b/docs/assets/img/user-guide/getting-started/create-new-quest.png differ diff --git a/docs/assets/img/user-guide/getting-started/imix-tome-eval.png b/docs/assets/img/user-guide/getting-started/imix-tome-eval.png new file mode 100644 index 000000000..a49588943 Binary files /dev/null and b/docs/assets/img/user-guide/getting-started/imix-tome-eval.png differ diff --git a/docs/assets/img/user-guide/getting-started/quest-result.png b/docs/assets/img/user-guide/getting-started/quest-result.png new file mode 100644 index 000000000..14816ba17 Binary files /dev/null and b/docs/assets/img/user-guide/getting-started/quest-result.png differ diff --git a/docs/assets/img/user-guide/getting-started/select-tome.png b/docs/assets/img/user-guide/getting-started/select-tome.png new file mode 100644 index 000000000..72adb6249 Binary files /dev/null and b/docs/assets/img/user-guide/getting-started/select-tome.png differ diff --git a/docs/assets/img/user-guide/getting-started/starting-imix.png b/docs/assets/img/user-guide/getting-started/starting-imix.png new file mode 100644 index 000000000..f540eef46 Binary files /dev/null and b/docs/assets/img/user-guide/getting-started/starting-imix.png differ diff --git a/docs/assets/img/user-guide/getting-started/starting-tavern.png b/docs/assets/img/user-guide/getting-started/starting-tavern.png new file mode 100644 index 000000000..7b462f53b Binary files /dev/null and b/docs/assets/img/user-guide/getting-started/starting-tavern.png differ diff --git a/docs/pages/index.md b/docs/pages/index.md index 84eebfeca..44b735deb 100644 --- a/docs/pages/index.md +++ b/docs/pages/index.md @@ -4,60 +4,60 @@ title: Realm Docs permalink: / --- -# Welcome to Realm! - -## User Guide -Realm is a cross platform Red Team engagement platform with a focus on automation and reliability. - -Want to get hands on now? Follow this guide to setup a dev instance of realm. - -![realm-logo](/assets/img/realm_create_quest.png) - -
Server (tavern)
-
    -
  • Web interface.
  • -
  • Group actions.
  • -
  • Graphql backend for easy API access.
  • -
  • OAuth login support.
  • -
  • Cloud native deployment with pre-made terraform for production deployments.
  • -
- -
Agent (imix)
-
    -
  • Written in rust with support for MacOS, Linux, and Windows.
  • -
  • Supports long running tasks by reading output from tasks in real time.
  • -
  • Interval callback times.
  • -
  • Simple file based configuration.
  • -
  • Embedded files.
  • -
  • Built-in interpreter.
  • -
- -
Standalone interpreter (golem)
-
    -
  • Interactive shell for testing hands-on testing.
  • -
  • Embedded files that execute autonomously on execuiton.
  • -
- -
Built-in interpreter (eldritch)
-
    -
  • Reflective DLL Loader.
  • -
  • Port scanning.
  • -
  • Remote execution over SSH.
  • -
  • And more.
  • -
- -## Developer Guide - -
-
- {% for post in site.docs %} - - {% if post.url contains 'dev-guide' %} - {% else %} - {% continue %} - {% endif %} -
-
{{ post.title }}
-

{{ post.description }}

-
{% endfor %} -
\ No newline at end of file +## Welcome to Realm + +[Realm](https://github.com/spellshift/realm) is an Adversary Emulation Framework with a **focus on scalability, reliability, and automation**. It is highly performant and is designed for engagements of any size (up to many thousands of beacons). [Get started in minutes](user-guide/getting-started). + +### Feature Highlights + +* **Focus on Reliability:** Realm always prioritizes quality over quantity, enabling operators to focus on the engagement instead of spending hours troubleshooting bugs. Extensive testing and rigorous code review ensure unwavering reliability, while an intuitive design and clear documentation keep the learning curve minimal. After reaching a stable `1.0.0` release, Realm will follow [Semantic Versioning](https://semver.org/), ensuring the stability of older deployments. + +* **Eldritch, a Pythonic DSL for Offensive Security:** Ditch clunky scripting and embrace [Eldritch](/user-guide/eldritch), Realm's Pythonic [Domain Specific Language (DSL)](https://en.wikipedia.org/wiki/Domain-specific_language) based on [Google Starlark](https://github.com/bazelbuild/starlark/blob/master/spec.md#starlark-language-specification). Write clear, concise, reuseable code that reflects your strategic thinking and streamlines offensive operations. [Eldritch](/user-guide/eldritch) is natively compiled to Rust, providing a performant abstraction for low-level system interactions. + +* **Effortless Multi-Host Management:** Juggling tasks across numerous machines during complex engagements? Realm simplifies the process, enabling you to control agents on multiple hosts simultaneously. + +* **Native GCP Integration:** Leverage the power and scalability of Google Cloud directly within your red team engagements. Realm seamlessly integrates with GCP services, boosting your attack capabilities without reinventing the wheel. + +* **Stateless Server Architecture:** While Realm officially supports GCP, you may deploy it's [stateless docker container](https://hub.docker.com/r/spellshift/tavern) to any environment that best fits your needs. + +### Documentation Overview + +Our three sections of documentation are split by intended audience, but feel free to explore! + +* [User Guide](/user-guide) provides documentations for operators using Realm. +* [Admin Guide](/admin-guide) provides documentation for managing a Realm deployment. +* [Developer Guide](/dev-guide) provides documentation for Realm contributors. + +### Contact Support + +Need a hand? We're here to help! If you're facing an issue with Realm, we're happy to assist! To ensure we can provide the best support, please [create an issue on our Github](https://github.com/spellshift/realm/issues/new?labels=bug&template=bug_report.md). + +#### Bug Support + +When opening your issue, please include: + +* A clear and concise description of the problem you're encountering. +* Any relevant error messages or logs. +* Steps to reproduce the issue (if possible). +* Impacted Realm version and operating system. + +The more information you provide, the faster we can investigate and help you resolve the issue. + +### Feature Requests & Feedback + +Realm lives and breathes through its users. Your insights and experiences are crucial in guiding its development and ensuring it continues to empower your mission. Please don't hesitate to reach out! + +**Remember:** + +* Be respectful and constructive in your feedback ([code of conduct](https://github.com/spellshift/realm/blob/main/CODE_OF_CONDUCT.md)). +* Search for existing discussions or feature requests before creating new ones. +* The more details you provide, the better we can understand your needs and respond effectively. +* Together, we can shape Realm into an incredible framework. **Thank you for being part of the adventure!** + +#### Feature Requests + +Do you have an idea for a feature that would supercharge your workflow? We're all ears! [Open an issue on GitHub](https://github.com/spellshift/realm/issues/new?labels=feature&projects=&template=feature_request.md&title=%5Bfeature%5D+Something+to+do) and share your detailed proposal. Be sure to explain the problem you're facing, the solution you envision, and how it would benefit other users. The more information you provide, the better we can understand your needs and assess the feasibility of implementing your suggestion. + +#### Provide Feedback + +Love something about Realm? Feel something could be improved? Let us know! Your feedback, good or bad, helps us make Realm better for everyone. [Open an issue on GitHub](https://github.com/spellshift/realm/issues/new?labels=feedback&projects=&template=feedback.md&title=%5Bfeedback%5D+Something+to+improve) outlining your thoughts, whether it's a praiseworthy feature, a usability concern, or a suggestion for improvement. Every bit of your feedback helps us refine Realm and make it an even more valuable tool in your red teaming toolbox.