Skip to content

Tidy up scripts #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
C0rn3j wants to merge 2 commits into
base: code
Choose a base branch
from
Open

Tidy up scripts #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2296af6
Tidy up scripts
C0rn3j Dec 2, 2025
7f12a70
Add requirements to docs
C0rn3j Dec 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README-scripts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
Requirements:
python-psycopg2
postgresql

After initial checkout:
(1) run setup.sh to setup the git-disk repo
(2) edit psqldb-env.sh to suit your environment
Expand Down Expand Up @@ -30,7 +34,3 @@ and rerun update.sh.
The creation of the tmpfs mounts is for both speed and to reduce SSD wear.
The mounts are not strictly needed and the scripts can easily be hacked to avoid
requiring them.




3 changes: 2 additions & 1 deletion doit.pl
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/usr/bin/env perl
use Data::Dumper;
use Cwd;

Expand Down Expand Up @@ -62,7 +63,7 @@
}

#
# Read existing entires.
# Read existing entries.
# If option A and `newId` is defined is used the commit will be assumed done.
# If option B is used and `cached` is defined the commit messages
# will be re-done, but the contents will not be re-done.
Expand Down
17 changes: 8 additions & 9 deletions init-psqldb.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
. ./psqldb-env.sh
#!/usr/bin/env bash
Copy link
Member

@kevina kevina Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason bash is needed here and #!/bin/sh won't work? The same applies to the other shell scripts. I have always used sh <script> and have not run into issues. On my system at least sh alias to dash and not bash.

If bash is needed for specific features that's OK. But I rather stick to the standard shell script unless there is a compelling reason not to.

Copy link
Author

@C0rn3j C0rn3j Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar enough with what's pure sh implementation and what's dash/bash when I write shellcode, so I default to bash.

I don't mind switching it to sh, though every system, the CI included, will have bash available.

Copy link
Member

@kevina kevina Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dash is a bit faster than bash, but it’s not something I am set on. I always use #!/bin/sh unless I have a good reason not to.

If you want to take a stab at getting the CI integration working, then switching to bash is fine if that’s what you are more comfortable with. Otherwise, I would prefer to stick with sh, as that’s what I would use.

set -euo pipefail

set -e
# Create the psql database

# create the psql database
. ./psqldb-env.sh

if findmnt psqldb
then
pgctl stop
sudo umount psqldb
if findmnt psqldb && [[ -e ${PGDIR} ]]; then
pgctl stop
sudo umount psqldb
fi

mkdir -p psqldb
Expand All @@ -28,7 +28,6 @@ max_wal_size = 512MB
EOF
pgctl start

createdb $DBNAME
createdb "$DBNAME"
psql scowl -f comp/util-fun.sql
psql scowl -f comp/comp-init.sql

8 changes: 3 additions & 5 deletions init.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
# create the tmp git repo

set -e
#!/usr/bin/env bash
set -euo pipefail

mkdir -p git
sudo mount -t tmpfs -o size=512M none git
cd git
ln -s ../git-disk/.git

ln -s ../git-disk/.git .
11 changes: 6 additions & 5 deletions psqldb-env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# set this to the absolute path of the root directly you are running the
# scripts from.
ROOTDIR=/home/kevina/wordlist/diff/
ROOTDIR="${HOME}/wordlist/diff/"

export DBNAME=scowl

export PGVER=11
export PGBINDIR=/usr/lib/postgresql/11/bin
Copy link
Member

@kevina kevina Dec 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not hard code PGVER. It should take the PGVER from the environment and if it's not set to the system default, and if that fails, fall back to 11. The PGBINDIR should then use PGVER to find the correct path.

Copy link
Author

@C0rn3j C0rn3j Dec 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what exactly is wanted here, deleting the PGVER line?

I simply bumped it to current release since 11 has been EOL for a while.

Copy link
Member

@kevina kevina Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something like:

PGVER="${PGVAR:-11}"

I am not set on defaulting to 11. It is just what I use and is known to work. My system is also older and doesn't have the latest version available. If we get this working on CI than I am okay with bumping this to the current version that is support by the CI environment as I will no longer need to run it on my system.

export PGVER=18
export PGBINDIR=/usr/lib/postgresql/18/bin

export PATH="$PGBINDIR":"$PATH"

Expand All @@ -15,7 +15,8 @@ export PGDIR="$DBROOT"/scowl
export PGHOST="$DBROOT"
export PGPORT=${PGPORT:-5437}

alias psql=`which psql`
alias psql=$(command -v psql)

pgctl () {
pg_ctl -D "$PGDIR" -l "$PGDIR"/log "$@"
pg_ctl -D "$PGDIR" -l "$PGDIR"/log "$@"
}
6 changes: 4 additions & 2 deletions push.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
git push diff diff \
$( for f in `git tag -l | fgrep diff/`; do echo $f:`basename $f`; done )
#!/usr/bin/env bash
set -euo pipefail

git push diff diff $( for f in $(git tag -l | grep -F diff/); do echo $f:$(basename $f); done )
8 changes: 5 additions & 3 deletions setup.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
# setup the git-disk repo
#!/usr/bin/env bash
set -euo pipefail

set -e
# Setup the git-disk repo

rm -rf git-disk
git init git-disk
cd git-disk
git remote add src git@github.com:en-wl/wordlist.git
#git remote add src /home/kevina/wordlist/v2-pub
#git remote add src "${HOME}/wordlist/v2-pub"
git remote add diff git@github.com:en-wl/wordlist-diff.git
git fetch src v1
git fetch src
Expand Down
16 changes: 8 additions & 8 deletions update.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
#!/bin/sh
#!/usr/bin/env bash
set -euo pipefail

# this will update an already populated git-disk

set -ex
# This will update an already populated git-disk

: ${SCOWL_BRANCH:=v2}
export SCOWL_BRANCH
: ${DIFF_BRANCH:=diff}
export DIFF_BRANCH
ROOTDIR="$PWD"

cd git-disk

git reset --hard
Expand All @@ -23,9 +21,11 @@ git reset --hard src/"$SCOWL_BRANCH"

cd ..

if ! mountpoint -q git; then sh init.sh; fi
if ! mountpoint -q git; then
./init.sh
fi

cd git
PATH="$ROOTDIR/bin:$PATH" perl ../doit.pl
PATH="$ROOTDIR/bin:$PATH" ../doit.pl

echo 'now do: cd git; sh ../push.sh'
echo 'now do: cd git; ../push.sh'