Release new version by nbirillo · Pull Request #118 · hyperskill/hyperstyle

nbirillo · 2021-08-26T18:02:25Z

Add penalty algorithm (if you don't send history, it does not work, but the output JSON has a new field in all cases)
Merge subcategories into five main categories (to disable this feature you shout use ‑‑with‑all‑categories flag)
Add difficulty level for each issue (Note, if you use this flag, the output format will be changed. To use this feature you should use ‑‑group‑by‑difficulty flag. README file contains ax example of this format)
Some code refactoring
Update grade thresholds according to the empirical analysis (discussed with Kolya)
Update lintes for Java, Kotlin and Python
Support Java15

Add wps-light support

Added support for flake8-broken-line, flake8-string-format, flake8-commas (they are WPS dependencies). Added support for cohesion.

Added new inspector: Radon. Added maintainability index.

Updated versions of dependencies Added django dictionary support Fixed tests

Added xlsx_tool_run.py – script to run the tool on multiple code samples stored in xlsx file

Add penalty system

* Updated penalty coefficients; * Added new tests; * Fixed bug with _get_issue_class_to_influence, which caused the dictionary to get issues that should not be penalized. * Fixed two cases in test_categorize. There were two identical issues in the list of previously made issues.

* Refactor the tool for evaluation: add multithreading, add a possibility to handle CSV arguments * Add script to filter solutions by language and drop duplicates * Add script to distribute grades from unique solutions to all solutions * Add script to find diffs between two graded dataframes * Add tests and descriptions

# Conflicts: # src/python/evaluation/common/util.py # src/python/evaluation/evaluation_config.py

* Add origin class for maintainability index * Add WPS518 to ignore due to collision with C0200 by Pylint

* Added a new category INFO, in order not to take into account some issues in the evaluation.

Added a utility for dataset marking

# Conflicts: # src/python/evaluation/evaluation_config.py # whitelist.txt

Added a script for plots

# Conflicts: # .github/workflows/build.yml # VERSION.md # requirements-test.txt # requirements.txt # src/python/common/tool_arguments.py # src/python/review/application_config.py # src/python/review/common/file_system.py # src/python/review/inspectors/detekt/detekt.py # src/python/review/inspectors/eslint/eslint.py # src/python/review/inspectors/flake8/.flake8 # src/python/review/inspectors/flake8/flake8.py # src/python/review/inspectors/intellij/intellij.py # src/python/review/inspectors/issue.py # src/python/review/inspectors/parsers/checkstyle_parser.py # src/python/review/inspectors/pmd/files/bin/basic.xml # src/python/review/inspectors/pmd/issue_types.py # src/python/review/inspectors/pmd/pmd.py # src/python/review/inspectors/pyast/python_ast.py # src/python/review/inspectors/pylint/pylint.py # src/python/review/inspectors/radon/radon.py # src/python/review/inspectors/spotbugs/spotbugs.py # src/python/review/inspectors/springlint/springlint.py # src/python/review/reviewers/common.py # src/python/review/reviewers/perform_review.py # src/python/review/reviewers/utils/code_statistics.py # src/python/review/reviewers/utils/print_review.py # src/python/review/run_tool.py # test/python/functional_tests/conftest.py # test/python/functional_tests/test_multi_file_project.py # test/python/functional_tests/test_range_of_lines.py # test/python/functional_tests/test_single_file_json_format.py # test/python/inspectors/test_checkstyle_inspector.py # test/python/inspectors/test_flake8_inspector.py # test/python/inspectors/test_local_review.py # test/python/inspectors/test_pmd_inspector.py # test/resources/inspectors/python/case37_wildcard_import.py # whitelist.txt

* Fixed #109 * Fixed #112 * Fixed #116

src/python/review/common/file_system.py

src/python/review/common/subprocess_runner.py

* Added output examples with --group-by-difficulty * typo fix

* Added some new words * Added W0511 * Added WPS428 * Added many exceptions * Sort whitelists (Github Actions) * Comments fix Co-authored-by: GirZ0n <GirZ0n@users.noreply.github.com>

* Updated detekt * Added new words * Sort whitelists (Github Actions) Co-authored-by: GirZ0n <GirZ0n@users.noreply.github.com>

* Fixed #122 * Sort whitelists (Github Actions) Co-authored-by: GirZ0n <GirZ0n@users.noreply.github.com>

nbirillo · 2021-08-30T17:02:55Z

@vyahhi @andrvb could you please check this PR?

andrvb · 2021-09-07T21:44:18Z

Are file like test/resources/evaluation/xlsx_target_files/target_sorted_order.xlsx and similar files in evaluation dir needed?

* Delete intellij inspector * Delete spotbugs inspector

* Delete intellij inspector * Delete spoibugs inspector * Download Detekt in Dockerfile * Install curl and unzip * Fix Dockerfile * Try to delete detekt sources * Check env variables * Try to get env variable * Use my docker image * Small code refactoring and also download detekt-jar file * Use my docker image * Use my docker image

* Delete intellij inspector * Delete spoibugs inspector * Download Detekt in Dockerfile * Install curl and unzip * Fix Dockerfile * Try to delete detekt sources * Check env variables * Try to get env variable * Use my docker image * Small code refactoring and also download detekt-jar file * Use my docker image * Use my docker image * Delete springlint inspector

* Delete checkstyle sources * Fix curl command

* Delete PMD sources * Fix flake8 and echo env variables * Fix pmd tests

nbirillo · 2021-09-13T20:08:55Z

@andrvb I deleted unused inspectors (since they are too slow and we don't plan to use them in the tool) Also I deleted all inspectors sources since it can violate licenses. And of course, I deleted unnecessary resources.

In GitHub actions I have to use my own docker image since an intermediate version is not pished into stepik docker hub and does not contain all necessary changes (I built my docker image by Dockerfile in the repo)

Sorry for the big PR (I decreased it as it was possible). But in this PR we updated linters and it means we have to change configs and issues categorization. Several configs are too big, also we have some additional big files (like whitelists for flake8 in the repo and in the tool).

You can ignore the following files:
whitelist.txt (both of them)
issue_types.py (for all inspectors)
xml configs for linters

It is about 1200 new lines of code. Also since we added ‑‑group‑by‑difficulty and ‑‑with‑all‑categories we changed a lot of tests, it also takes a lot of new code lines.

vyahhi · 2021-09-13T20:28:06Z

vyahhi · 2021-09-16T11:37:30Z

@andrvb let's merge this PR?

andrvb · 2021-09-16T20:16:43Z

Dockerfile

+# Install PMD
+RUN curl -sSLO https://github.com/pmd/pmd/releases/download/pmd_releases/${PMD_VERSION}/pmd-bin-${PMD_VERSION}.zip \
+    && unzip pmd-bin-${PMD_VERSION}.zip -d ${PMD_DIRECTORY}
+


It would be better to move the linters installation before the hyperstyle code installation. This will allow you to use the Docker cache because the hyperstyle code changes more frequently than the linter versions.
See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#leverage-build-cache

Or (maybe it even more better) move the linters installation to base docker image. In this case, you can continue to use the base image in GitHub actions.

@andrvb @vyahhi Should I do it in this PR? Or in another one?``

@nbirillo, whatever works better for you.

Let's do it in the next PR

vyahhi · 2021-09-22T17:42:15Z

OK, I'm merging this PR since it has no major issues to fix anymore.

GirZ0n and others added 30 commits May 3, 2021 15:58

wps-light support (#17)

051260c

Add wps-light support

New flake8 plugins support (#18)

f3d42f3

Added support for flake8-broken-line, flake8-string-format, flake8-commas (they are WPS dependencies). Added support for cohesion.

Radon support (#19)

4b47996

Added new inspector: Radon. Added maintainability index.

Requirements upgrade (#21)

388f2fa

Updated versions of dependencies Added django dictionary support Fixed tests

xlsx-run-tool (#24)

eef3f41

Added xlsx_tool_run.py – script to run the tool on multiple code samples stored in xlsx file

Penalty system (#25)

b14b5b9

Add penalty system

Update version to 1.2.0

045deff

Merge branch 'main' into main-upd

595f758

Rename output_format to format

7e765fb

Fix double quotes

afa44ec

Fix small issues

60c93bf

Fix small issues

f3c2ff9

Fix double quotes

5db1ca3

Fix double quotes

057031f

Merge branch 'main-upd' into develop

9125521

# Conflicts: # src/python/evaluation/common/util.py # src/python/evaluation/evaluation_config.py

Inspectors fix (#39)

c5ffb19

* Add origin class for maintainability index * Add WPS518 to ignore due to collision with C0200 by Pylint

New category (#38)

b3d7c70

* Added a new category INFO, in order not to take into account some issues in the evaluation.

Merge branch 'main-upd' into develop

1b72ce3

Dataset labeling utility (#31)

aa05fc4

Added a utility for dataset marking

resolved merge conflicts

77e03b2

Merge branch 'main' into main-upd

bf40554

Fixed trailing commas

0e82bf7

Delete create_directory function

ba15cb3

Delete create_directory function

1de2aff

Merge branch 'main-upd' into develop

b69e709

# Conflicts: # src/python/evaluation/evaluation_config.py # whitelist.txt

Update version

e7b103f

Graph plotter (#41)

a56cb28

Added a script for plots

updated whitelist to resolve merge conflicts

6961e87

nbirillo and others added 2 commits August 26, 2021 14:56

Issues fix (#117)

a6c2405

* Fixed #109 * Fixed #112 * Fixed #116

nbirillo commented Aug 26, 2021

View reviewed changes

nbirillo mentioned this pull request Aug 27, 2021

Release pip package for hyperstyle #94

Closed

GirZ0n and others added 2 commits August 27, 2021 13:42

Updated README (#119)

1442316

* Added output examples with --group-by-difficulty * typo fix

Delete analysis part (#120)

1c3a75f

nbirillo requested review from andrvb and vyahhi August 27, 2021 10:52

GirZ0n and others added 3 commits August 30, 2021 12:29

Pylint categorization update (#121)

fab3503

* Added some new words * Added W0511 * Added WPS428 * Added many exceptions * Sort whitelists (Github Actions) * Comments fix Co-authored-by: GirZ0n <GirZ0n@users.noreply.github.com>

Detekt update (#123)

d762788

* Updated detekt * Added new words * Sort whitelists (Github Actions) Co-authored-by: GirZ0n <GirZ0n@users.noreply.github.com>

Issue fix (#127)

93bf94d

* Fixed #122 * Sort whitelists (Github Actions) Co-authored-by: GirZ0n <GirZ0n@users.noreply.github.com>

nbirillo and others added 8 commits September 13, 2021 12:13

Delete evaluation resources

e0fb60a

Delete Intellij and spotbugs inspectors (#131)

78d4f7a

* Delete intellij inspector * Delete spotbugs inspector

Fix teamcity build

7da76b0

Delete checkstyle sources (#134)

dfaa3d9

* Delete checkstyle sources * Fix curl command

Delete pmd sources (#135)

a5283ee

* Delete PMD sources * Fix flake8 and echo env variables * Fix pmd tests

Delete ses evaluation resources

5997bf8

nbirillo mentioned this pull request Sep 13, 2021

Delete source code of the linters from the project #130

Closed

Add a guideline about tool installation (#137)

60edc0e

andrvb approved these changes Sep 16, 2021

View reviewed changes

vyahhi merged commit 44e5025 into main Sep 22, 2021

nbirillo deleted the develop branch October 6, 2021 08:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release new version#118

Release new version#118
vyahhi merged 90 commits intomainfrom
develop

nbirillo commented Aug 26, 2021 •

edited by vyahhi

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nbirillo commented Aug 30, 2021

Uh oh!

andrvb commented Sep 7, 2021

Uh oh!

nbirillo commented Sep 13, 2021

Uh oh!

vyahhi commented Sep 13, 2021

Uh oh!

vyahhi commented Sep 16, 2021

Uh oh!

andrvb Sep 16, 2021

Uh oh!

nbirillo Sep 17, 2021 •

edited

Loading

Uh oh!

andrvb Sep 20, 2021

Uh oh!

nbirillo Sep 20, 2021

Uh oh!

vyahhi commented Sep 22, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

nbirillo commented Aug 26, 2021 • edited by vyahhi Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nbirillo commented Aug 30, 2021

Uh oh!

andrvb commented Sep 7, 2021

Uh oh!

nbirillo commented Sep 13, 2021

Uh oh!

vyahhi commented Sep 13, 2021

Uh oh!

vyahhi commented Sep 16, 2021

Uh oh!

andrvb Sep 16, 2021

Choose a reason for hiding this comment

Uh oh!

nbirillo Sep 17, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrvb Sep 20, 2021

Choose a reason for hiding this comment

Uh oh!

nbirillo Sep 20, 2021

Choose a reason for hiding this comment

Uh oh!

vyahhi commented Sep 22, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

nbirillo commented Aug 26, 2021 •

edited by vyahhi

Loading

nbirillo Sep 17, 2021 •

edited

Loading