Skip to content

Improvements to the configuration logic #744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pombredanne merged 6 commits into from
Sep 19, 2017
Merged

Improvements to the configuration logic #744

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d5b01d6
Automatically (re-)configure if Git commits have changed
sschuberth Aug 30, 2017
a138c65
Do not write a space after ellipses
sschuberth Aug 30, 2017
5478e65
When configuring, print which config is being used even in quiet mode
sschuberth Aug 30, 2017
24f125d
Move the comment about development mode to the SCANCODE_DEV_MODE file
sschuberth Aug 30, 2017
773bffa
Remove any stale development tag file from previous configure runs
sschuberth Aug 30, 2017
c95017a
Do not shadow the os package with a variable name
sschuberth Aug 30, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# ScanCode special files
/SCANCODE_DEV_MODE
/SCANCODE_GIT_MODE

# Python compiled files
*.py[cod]
Expand Down
5 changes: 3 additions & 2 deletions MANIFEST.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ graft src
graft etc
graft thirdparty
graft samples

graft tests

prune src/scancode_toolkit.egg-info
Expand All @@ -27,8 +26,10 @@ include scancode
include scancode.bat
include extractcode
include extractcode.bat
include .travis.yml
include appveyor.yml

include .travis.yml appveyor.yml
exclude SCANCODE_DEV_MODE
exclude SCANCODE_GIT_MODE

global-exclude *.py[co] __pycache__ *.*~
25 changes: 20 additions & 5 deletions etc/conf/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ def unsupported(platform):

sys_platform = str(sys.platform).lower()
if 'linux' in sys_platform:
os = 'linux'
platform = 'linux'
elif'win32' in sys_platform:
os = 'win'
platform = 'win'
elif 'darwin' in sys_platform:
os = 'mac'
platform = 'mac'
else:
unsupported(sys_platform)

Expand All @@ -38,9 +38,24 @@ def unsupported(platform):
'mac': ['64',],
}

arches = supported_combos[os]
arches = supported_combos[platform]
if arch not in arches:
unsupported(os + arch)
unsupported(platform + arch)


def clear_dev_mode():
"""
Remove any stale development tag file from previous configure runs.
"""
import os
from scancode import root_dir
try:
os.remove(os.path.join(root_dir, 'SCANCODE_DEV_MODE'))
except OSError:
pass


clear_dev_mode()


"""
Expand Down
9 changes: 3 additions & 6 deletions etc/conf/dev/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,11 @@


def setup_dev_mode():
"""
Create the development mode tag file. In development mode, ScanCode does
not rely on license data to remain untouched and will always check the
license index cache for consistency, rebuilding it if necessary.
"""
from scancode import root_dir
with open(os.path.join(root_dir, 'SCANCODE_DEV_MODE'), 'wb') as sdm:
sdm.write('This is a tag file to notify that ScanCode is used in development mode.')
sdm.write('This is a tag file to notify that ScanCode is used in development mode. '
'In this mode, ScanCode does not rely on license data to remain untouched and will '
'always check the license index cache for consistency, rebuilding it if necessary.')


setup_dev_mode()
4 changes: 3 additions & 1 deletion etc/configure.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ def create_virtualenv(std_python, root_dir, tpp_dirs, quiet=False):

def activate(root_dir):
""" Activate a virtualenv in the current process."""
print("* Activating ...")
print("* Activating...")
bin_dir = os.path.join(root_dir, 'bin')
activate_this = os.path.join(bin_dir, 'activate_this.py')
with open(activate_this) as f:
Expand Down Expand Up @@ -365,6 +365,8 @@ def get_conf_files(config_dir_paths, root_dir, file_names=requirements):
abs_path = os.path.join(root_dir, path)
if os.path.exists(abs_path):
configs.append(path)
print('Using configuration directory:\n'
' %(path)s' % locals())
else:
print()
print('WARNING: Skipping missing Configuration directory:\n'
Expand Down
5 changes: 4 additions & 1 deletion etc/release/MANIFEST.in.release
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,10 @@ include scancode
include scancode.bat
include extractcode
include extractcode.bat
include .travis.yml appveyor.yml
include .travis.yml
include appveyor.yml

exclude SCANCODE_DEV_MODE
exclude SCANCODE_GIT_MODE

global-exclude *.py[co] __pycache__ *.*~
25 changes: 24 additions & 1 deletion scancode
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,10 +105,33 @@ _canonicalize_file_path() {
SCANCODE_BIN="$( realpath "${BASH_SOURCE[0]}" )"
SCANCODE_ROOT_DIR="$( cd "$( dirname "${SCANCODE_BIN}" )" && pwd )"

SCANCODE_CONFIG_DIR="etc/conf"

GIT_STATUS=$(git -C "$SCANCODE_ROOT_DIR" status --porcelain 2> /dev/null)
Copy link
Member

@pombredanne pombredanne Aug 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this also implicitly checks for Git being available locally? Just curious what the different with a plain git status?

Copy link
Collaborator Author

@sschuberth sschuberth Aug 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. It implicitly checks for $SCANCODE_ROOT_DIR to be a Git working tree (git status returns code 128 otherwise), and also for the git executable to be present (in this case it's Bash that returns code 127).

if [ $? -eq 0 ]; then
# This is a Git working tree, check whether it is clean.
if [ -z "$GIT_STATUS" ]; then
# This is a clean working tree, check whether configure has been run
# before for this commit.
GIT_COMMIT_CONFIGURED=$(cat "$SCANCODE_ROOT_DIR/SCANCODE_GIT_MODE" 2> /dev/null)
GIT_COMMIT_HEAD=$(git -C "$SCANCODE_ROOT_DIR" rev-parse HEAD)
if [ "$GIT_COMMIT_CONFIGURED" != "$GIT_COMMIT_HEAD" ]; then
echo "* (Re-)Configuring ScanCode because Git commits have changed..."
CONFIGURE_QUIET=1 $SCANCODE_ROOT_DIR/configure $SCANCODE_CONFIG_DIR
Copy link
Member

@pombredanne pombredanne Aug 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO to be really safe, you should first issue a configure --clean but let see first if this is enough.
There could be cases where files have been removed and there could be dangling .pyc files left over in some rare cases

Copy link
Collaborator Author

@sschuberth sschuberth Aug 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I tried to avoid a full clean by doing e.g. 773bffa.

fi
echo $GIT_COMMIT_HEAD > "$SCANCODE_ROOT_DIR/SCANCODE_GIT_MODE"
else
echo "* (Re-)Configuring ScanCode because Git working tree is not clean..."
CONFIGURE_QUIET=1 $SCANCODE_ROOT_DIR/configure $SCANCODE_CONFIG_DIR
fi
else
rm -f "$SCANCODE_ROOT_DIR/SCANCODE_GIT_MODE"
fi

SCANCODE_CONFIGURED_PYTHON=$SCANCODE_ROOT_DIR/bin/python
if [ ! -f "$SCANCODE_CONFIGURED_PYTHON" ]; then
echo "* Configuring ScanCode for first use..."
CONFIGURE_QUIET=1 $SCANCODE_ROOT_DIR/configure etc/conf
CONFIGURE_QUIET=1 $SCANCODE_ROOT_DIR/configure $SCANCODE_CONFIG_DIR
fi

$SCANCODE_ROOT_DIR/bin/scancode "$@"