Tidy up scripts

[C0rn3j]

This all executables as such and starts using shebangs.

There's changes and safeguards added while I was trying to run this.

Sort of WIP since I never got it to run the setup properly, I do not get what "${HOME}/wordlist/v2-pub" is supposed to be.

Please squash when merging.

[kevina]

I do not get what "${HOME}/wordlist/v2-pub" is supposed to be.

That is a local hack to use the local git repo on my local machine. I just pushed some updates to sync with what I had on my machine and remove the local hack.

[C0rn3j]

It gets further now, but it looks like it creates a broken symlink.

make: Leaving directory '/home/c0rn3j/Projects/wordlist-diff/git/speller'
Can't exec "comp/comp.sh": No such file or directory at ../doit.pl line 86.

[kevina]

Sorry. There were more local hacks and a bunch of files I didn't commit. I just did a forced update on code branch. I have verified it at least works on my machine in a fresh checkout. The diff branch is now also uptodate with en-wl/wordlist. See the updated README for how to rerun the scripts on existing commits.

[C0rn3j]

I got it to run with some minor changes.

Does this do a large operation per each diff?

It's been running on my somewhat decent laptop for quite a while now (for the last 111 commits).

I'm not sure how well GitHub CI will take this.

[kevina]

Thank you for continuing to work on this.

Does this do a large operation per each diff?

What do you mean by large operation? It should take a few minutes per diff. The results are cached though in the git repo itself so it won't do unnecessary work.

[kevina]

That you for keeping the changes minimal. There is just a few minor things.

[kevina]

Let's not hard code PGVER. It should take the PGVER from the environment and if it's not set to the system default, and if that fails, fall back to 11. The PGBINDIR should then use PGVER to find the correct path.

[C0rn3j]

I'm not sure what exactly is wanted here, deleting the PGVER line?

I simply bumped it to current release since 11 has been EOL for a while.

[kevina]

Maybe something like:

PGVER="${PGVAR:-11}"

I am not set on defaulting to 11. It is just what I use and is known to work. My system is also older and doesn't have the latest version available. If we get this working on CI than I am okay with bumping this to the current version that is support by the CI environment as I will no longer need to run it on my system.

[kevina]

Is there a reason bash is needed here and #!/bin/sh won't work? The same applies to the other shell scripts. I have always used sh <script> and have not run into issues. On my system at least sh alias to dash and not bash.

If bash is needed for specific features that's OK. But I rather stick to the standard shell script unless there is a compelling reason not to.

[C0rn3j]

I'm not familiar enough with what's pure sh implementation and what's dash/bash when I write shellcode, so I default to bash.

I don't mind switching it to sh, though every system, the CI included, will have bash available.

[kevina]

Dash is a bit faster than bash, but it’s not something I am set on. I always use #!/bin/sh unless I have a good reason not to.

If you want to take a stab at getting the CI integration working, then switching to bash is fine if that’s what you are more comfortable with. Otherwise, I would prefer to stick with sh, as that’s what I would use.

[kevina]

I should add the CI task does need to commit the changes when done, so there might need to be some precautions to parallel runs that can cause commit conflicts.

What I would ultimately like to see is that when a P.R. is submitted some sort of link to the diff in this repo is added to the P.R. There also needs to be some sort way to map the P.R. (and other branches) into a parallel branch in the this repo.

I have very little experience with setting something like this up (and at this point not even sure if it possible). If you are willing to try and make this work I can give you the necessary access to do so.