Skip to content

fix(database): ensure git uses proper credentials #403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ReenigneArcher merged 1 commit into from
May 16, 2025
Merged

fix(database): ensure git uses proper credentials #403

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 33 additions & 30 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,36 +37,39 @@ platforms such as GitHub discussions/issues might be added in the future.
:exclamation: if using Docker these can be arguments.
:warning: Never publicly expose your tokens, secrets, or ids.

| variable | required | default | description |
|----------------------------------|----------|------------------------------------------------------|-------------------------------------------------------------------------|
| DAILY_TASKS | False | `true` | Daily tasks on or off. |
| DAILY_RELEASES | False | `true` | Send a message for each game released on this day in history. |
| DAILY_CHANNEL_ID | False | `None` | Required if daily_tasks is enabled. |
| DAILY_TASKS_UTC_HOUR | False | `12` | The hour to run daily tasks. |
| DATA_REPO | False | `https://github.com/LizardByte/support-bot-data` | Repository to store persistent data. This repository should be private! |
| DATA_REPO_BRANCH | False | `master` | Branch to store persistent data. |
| DISCORD_BOT_TOKEN | True | `None` | Token from Bot page on discord developer portal. |
| DISCORD_CLIENT_ID | True | `None` | Discord OAuth2 client id. |
| DISCORD_CLIENT_SECRET | True | `None` | Discord OAuth2 client secret. |
| DISCORD_GITHUB_STATUS_CHANNEL_ID | True | `None` | Channel ID to send GitHub status updates to. |
| DISCORD_REDDIT_CHANNEL_ID | True | `None` | Channel ID to send Reddit post updates to. |
| DISCORD_REDIRECT_URI | False | `https://localhost:8080/discord/callback` | The redirect uri for OAuth2. Must be publicly accessible. |
| DISCORD_SPONSORS_CHANNEL_ID | True | `None` | Channel ID to send sponsorship updates to. |
| GITHUB_CLIENT_ID | True | `None` | GitHub OAuth2 client id. |
| GITHUB_CLIENT_SECRET | True | `None` | GitHub OAuth2 client secret. |
| GITHUB_REDIRECT_URI | False | `https://localhost:8080/github/callback` | The redirect uri for OAuth2. Must be publicly accessible. |
| GITHUB_TOKEN | True | `None` | GitHub personal access token. Must have `read:org` |
| GITHUB_WEBHOOK_SECRET_KEY | True | `None` | A secret value to ensure webhooks are from trusted sources. |
| GRAVATAR_EMAIL | False | `None` | Gravatar email address for bot avatar. |
| IGDB_CLIENT_ID | False | `None` | Required if daily_releases is enabled. |
| IGDB_CLIENT_SECRET | False | `None` | Required if daily_releases is enabled. |
| PRAW_CLIENT_ID | True | `None` | `client_id` from reddit app setup page. |
| PRAW_CLIENT_SECRET | True | `None` | `client_secret` from reddit app setup page. |
| PRAW_SUBREDDIT | True | `None` | Subreddit to monitor (reddit user should be moderator of the subreddit) |
| REDDIT_USERNAME | True | `None` | Reddit username |
| REDDIT_PASSWORD | True | `None` | Reddit password |
| SUPPORT_COMMANDS_REPO | False | `https://github.com/LizardByte/support-bot-commands` | Repository for support commands. |
| SUPPORT_COMMANDS_BRANCH | False | `master` | Branch for support commands. |
| variable | required | default | description |
|----------------------------------|----------|------------------------------------------------------|--------------------------------------------------------------------------------------------|
| DAILY_TASKS | False | `true` | Daily tasks on or off. |
| DAILY_RELEASES | False | `true` | Send a message for each game released on this day in history. |
| DAILY_CHANNEL_ID | False | `None` | Required if daily_tasks is enabled. |
| DAILY_TASKS_UTC_HOUR | False | `12` | The hour to run daily tasks. |
| DATA_REPO | False | `https://github.com/LizardByte/support-bot-data` | Repository to store persistent data. This repository should be private! |
| DATA_REPO_BRANCH | False | `master` | Branch to store persistent data. |
| DISCORD_BOT_TOKEN | True | `None` | Token from Bot page on discord developer portal. |
| DISCORD_CLIENT_ID | True | `None` | Discord OAuth2 client id. |
| DISCORD_CLIENT_SECRET | True | `None` | Discord OAuth2 client secret. |
| DISCORD_GITHUB_STATUS_CHANNEL_ID | True | `None` | Channel ID to send GitHub status updates to. |
| DISCORD_REDDIT_CHANNEL_ID | True | `None` | Channel ID to send Reddit post updates to. |
| DISCORD_REDIRECT_URI | False | `https://localhost:8080/discord/callback` | The redirect uri for OAuth2. Must be publicly accessible. |
| DISCORD_SPONSORS_CHANNEL_ID | True | `None` | Channel ID to send sponsorship updates to. |
| GIT_USER_EMAIL | True | `None` | Email address for git commits. |
| GIT_USER_NAME | True | `None` | Username for git commits. |
| GIT_TOKEN | True | `None` | GitHub personal access token. Must have `repo` write access. Falls back to `GITHUB_TOKEN`. |
| GITHUB_CLIENT_ID | True | `None` | GitHub OAuth2 client id. |
| GITHUB_CLIENT_SECRET | True | `None` | GitHub OAuth2 client secret. |
| GITHUB_REDIRECT_URI | False | `https://localhost:8080/github/callback` | The redirect uri for OAuth2. Must be publicly accessible. |
| GITHUB_TOKEN | True | `None` | GitHub personal access token. Must have `read:org` |
| GITHUB_WEBHOOK_SECRET_KEY | True | `None` | A secret value to ensure webhooks are from trusted sources. |
| GRAVATAR_EMAIL | False | `None` | Gravatar email address for bot avatar. |
| IGDB_CLIENT_ID | False | `None` | Required if daily_releases is enabled. |
| IGDB_CLIENT_SECRET | False | `None` | Required if daily_releases is enabled. |
| PRAW_CLIENT_ID | True | `None` | `client_id` from reddit app setup page. |
| PRAW_CLIENT_SECRET | True | `None` | `client_secret` from reddit app setup page. |
| PRAW_SUBREDDIT | True | `None` | Subreddit to monitor (reddit user should be moderator of the subreddit) |
| REDDIT_USERNAME | True | `None` | Reddit username |
| REDDIT_PASSWORD | True | `None` | Reddit password |
| SUPPORT_COMMANDS_REPO | False | `https://github.com/LizardByte/support-bot-commands` | Repository for support commands. |
| SUPPORT_COMMANDS_BRANCH | False | `master` | Branch for support commands. |

### Start

Expand Down
62 changes: 57 additions & 5 deletions src/common/database.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,35 @@
self.repo_branch = os.getenv("DATA_REPO_BRANCH", "master")
self.db_dir = os.path.join(self.db_dir, "support-bot-data")

# Get Git user configuration from environment variables
self.git_user_name = os.environ["GIT_USER_NAME"]
self.git_user_email = os.environ["GIT_USER_EMAIL"]

Check warning on line 41 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L40-L41 

Added lines #L40 - L41 were not covered by tests

# Git credentials for authentication (required for private repo)
self.git_token = os.getenv("GIT_TOKEN") or os.getenv("GITHUB_TOKEN")
if not self.git_token:
raise ValueError("GIT_TOKEN or GITHUB_TOKEN must be provided for private repository access")

Check warning on line 46 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L44-L46 

Added lines #L44 - L46 were not covered by tests

# Prepare URL with credentials for private repo access
protocol, repo_path = self.repo_url.split("://", 1)
clone_url = f"{protocol}://{self.git_user_name}:{self.git_token}@{repo_path}"

Check warning on line 50 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L49-L50 

Added lines #L49 - L50 were not covered by tests

if not os.path.exists(self.db_dir):
# Clone repo if it doesn't exist
print(f"Cloning repository {self.repo_url} to {self.db_dir}")
try:
# Try cloning with the specified branch
self.repo = git.Repo.clone_from(self.repo_url, self.db_dir, branch=self.repo_branch)
self.repo = git.Repo.clone_from(clone_url, self.db_dir, branch=self.repo_branch)

Check warning on line 57 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L57 

Added line #L57 was not covered by tests
# Configure the repo
self._configure_repo()

Check warning on line 59 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L59 

Added line #L59 was not covered by tests
except git.exc.GitCommandError as e:
# Check if the error is due to branch not found
if "Remote branch" in str(e) and "not found in upstream origin" in str(e):
print(f"Branch '{self.repo_branch}' not found in remote. Creating a new empty branch.")
# Clone with default branch first
self.repo = git.Repo.clone_from(self.repo_url, self.db_dir)
self.repo = git.Repo.clone_from(clone_url, self.db_dir)

Check warning on line 65 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L65 

Added line #L65 was not covered by tests
# Configure the repo
self._configure_repo()

Check warning on line 67 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L67 

Added line #L67 was not covered by tests

# Create a new orphan branch (not based on any other branch)
self.repo.git.checkout('--orphan', self.repo_branch)
Expand Down Expand Up @@ -91,6 +108,8 @@
else:
# Use existing repo
self.repo = git.Repo(self.db_dir)
# Configure the repo
self._configure_repo()

Check warning on line 112 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L112 

Added line #L112 was not covered by tests

# Make sure the correct branch is checked out
if self.repo_branch not in [ref.name.split('/')[-1] for ref in self.repo.refs]:
Expand Down Expand Up @@ -136,6 +155,34 @@
indent=4,
)

def _configure_repo(self):
"""Configure the Git repository with user identity from environment variables."""
if self.repo:
with self.repo.config_writer() as config:

Check warning on line 161 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L160-L161 

Added lines #L160 - L161 were not covered by tests
# Set user name and email for this repository
config.set_value("user", "name", self.git_user_name)
config.set_value("user", "email", self.git_user_email)

Check warning on line 164 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L163-L164 

Added lines #L163 - L164 were not covered by tests

# Configure credentials for private repo access
domain = self.repo_url.split("://")[-1].split("/")[0]

Check warning on line 167 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L167 

Added line #L167 was not covered by tests

# Set credential store helper
config.set_value("credential", "helper", "store")

Check warning on line 170 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L170 

Added line #L170 was not covered by tests

# Set credential helper specific to this domain
if self.git_user_name and self.git_token:
config.set_value(f"credential \"{domain}\"", "username", self.git_user_name)

Check warning on line 174 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L173-L174 

Added lines #L173 - L174 were not covered by tests

# Update origin URL with credentials to ensure push works
protocol, repo_path = self.repo_url.split("://", 1)
new_url = f"{protocol}://{self.git_user_name}:{self.git_token}@{repo_path}"
try:
origin = self.repo.remote('origin')
origin.set_url(new_url)
except git.exc.GitCommandError as e:
print(f"Failed to update remote URL: {str(e)}")

Check warning on line 183 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L177-L183 

Added lines #L177 - L183 were not covered by tests
# Continue anyway, might work with stored credentials

def _check_for_migration(self):
# Check if migration is needed (shelve exists but json doesn't)
# No extension is used on Linux
Expand Down Expand Up @@ -253,15 +300,20 @@

# Check if we have anything to commit after adding
if self.repo.git.status('--porcelain'):
# Ensure the repository is configured with user identity
self._configure_repo()

Check warning on line 304 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L304 

Added line #L304 was not covered by tests

# Commit all changes at once with a general message
commit_message = "Update database files"
self.repo.git.commit('-m', commit_message)
print("Committed changes to git data repository")

# Push to remote
# Push to remote with credentials
try:
origin = self.repo.remote('origin')
origin.push()
# Ensure we're using the credentials for push
protocol, repo_path = self.repo_url.split("://", 1)
push_url = f"{protocol}://{self.git_user_name}:{self.git_token}@{repo_path}"
self.repo.git.push(push_url, self.repo_branch)

Check warning on line 316 in src/common/database.py

View check run for this annotation

Codecov / codecov/patch

src/common/database.py#L314-L316 

Added lines #L314 - L316 were not covered by tests
print("Pushed changes to remote git data repository")
except git.exc.GitCommandError as e:
print(f"Failed to push changes: {str(e)}")
Expand Down
Loading