extends eval engine to work with additional task type active learning#1184
Merged
janvanrijn merged 2 commits intodevelopfrom Feb 28, 2023
Merged
extends eval engine to work with additional task type active learning#1184janvanrijn merged 2 commits intodevelopfrom
janvanrijn merged 2 commits intodevelopfrom
Conversation
PGijsbers
approved these changes
Feb 27, 2023
Contributor
PGijsbers
left a comment
PGijsbers
left a comment
There was a problem hiding this comment.
So I assume this will just generate whatever task splits the evaluation procedure declares when the task is created? What does this mean for other parts of processing the task (e.g., the predictions)? Do they simply not get processed (which is OK, I think. They would add an evaluation engine after the server reimplementation)?
thanks for this catch! GitHub should get a Grammerly plugin Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl>
Member
Author
|
currently, the evaluation engine relies on the php webOS to only give it tasks where it can use splits for. This is of course waiting for a problem to happen, because we want these checks on the same place. the Java engine will crash if it can't handle the task type, but still |
janvanrijn
added a commit
that referenced
this pull request
Jan 8, 2024
* add local changes * skis minio url for sparse arff * Update .htaccess * cleanup * fix in data fork * fix for CORS * merging Api_data * merging ES * updated PHP and ES * minio_url to parquet_url rename * ES update * set htaccess back to development state (?) * Revert "set htaccess back to development state (?)" This reverts commit 9181f73. * Develop with changes (#1172) * merging Api_data * merging ES * updated PHP and ES Co-authored-by: root <root@openml.test.tue.nl> * Revert "Develop with changes (#1172)" (#1173) This reverts commit 031dd12. * added active testing (#1183) * extends eval engine to work with additional task type active learning (#1184) * extends eval engine to work with additional task type active learning * Update openml_OS/controllers/Api_splits.php thanks for this catch! GitHub should get a Grammerly plugin Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> --------- Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> * Update README.md * Add qa tasktype (#1193) * adds records * bugfix * updated file * update record * update record * adds text tasks * extended with task * typo fix * adds splits option * task answers * fix dataset field * adds default task to test server reinstantiation (#1195) * Point .pq files to datasets bucket * Update to dataset description in test server, remove old minio_url field --------- Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> Co-authored-by: root <root@openml.test.tue.nl> Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> Co-authored-by: Jos van der Velde <j.d.v.d.velde@tue.nl>
janvanrijn
added a commit
that referenced
this pull request
Jan 10, 2024
* add new feature * updated error codes * updates api data feature description * pull develop back in branch (#1168) * add local changes * skis minio url for sparse arff * Update .htaccess * cleanup * fix in data fork * fix for CORS * minio_url to parquet_url rename * ES update Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> * pull develop into branch (#1169) * add local changes * skis minio url for sparse arff * Update .htaccess * cleanup * fix in data fork * fix for CORS * minio_url to parquet_url rename * ES update * set htaccess back to development state (?) * Revert "set htaccess back to development state (?)" This reverts commit 9181f73. Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> * pull feature description into develop (#1170) * add local changes * skis minio url for sparse arff * Update .htaccess * cleanup * fix in data fork * fix for CORS * minio_url to parquet_url rename * ES update * set htaccess back to development state (?) * Revert "set htaccess back to development state (?)" This reverts commit 9181f73. Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> * pull develop into feature description (#1171) * add local changes * skis minio url for sparse arff * Update .htaccess * cleanup * fix in data fork * fix for CORS * minio_url to parquet_url rename * ES update * set htaccess back to development state (?) * Revert "set htaccess back to development state (?)" This reverts commit 9181f73. Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> * align htaccess with develop * Develop with changes (#1174) * add local changes * skis minio url for sparse arff * Update .htaccess * cleanup * fix in data fork * fix for CORS * merging Api_data * merging ES * updated PHP and ES * minio_url to parquet_url rename * ES update * set htaccess back to development state (?) * Revert "set htaccess back to development state (?)" This reverts commit 9181f73. Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> Co-authored-by: root <root@openml.test.tue.nl> * fixes problem * fixes problem * bugfix in old frontend * feature description update (#1205) * add local changes * skis minio url for sparse arff * Update .htaccess * cleanup * fix in data fork * fix for CORS * merging Api_data * merging ES * updated PHP and ES * minio_url to parquet_url rename * ES update * set htaccess back to development state (?) * Revert "set htaccess back to development state (?)" This reverts commit 9181f73. * Develop with changes (#1172) * merging Api_data * merging ES * updated PHP and ES Co-authored-by: root <root@openml.test.tue.nl> * Revert "Develop with changes (#1172)" (#1173) This reverts commit 031dd12. * added active testing (#1183) * extends eval engine to work with additional task type active learning (#1184) * extends eval engine to work with additional task type active learning * Update openml_OS/controllers/Api_splits.php thanks for this catch! GitHub should get a Grammerly plugin Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> --------- Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> * Update README.md * Add qa tasktype (#1193) * adds records * bugfix * updated file * update record * update record * adds text tasks * extended with task * typo fix * adds splits option * task answers * fix dataset field * adds default task to test server reinstantiation (#1195) * Point .pq files to datasets bucket * Update to dataset description in test server, remove old minio_url field --------- Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> Co-authored-by: root <root@openml.test.tue.nl> Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> Co-authored-by: Jos van der Velde <j.d.v.d.velde@tue.nl> * ontologies to feature view * typo fix * added if/else construct for ontologies * typo fix * typo fix * adds datafeature description * typo fix * typo fix * typo fix * typo fix * typo fix * query fixes * error_handling * xml fixes * typo fix * typo fix * fix api call * api change * fix * fix * revert weird change * adds data feature to SQL (needed for ontologies) * update reinstall script * update cron * adds data processed * updated * adds feature index to query * small update * updates data feature value sql * data feature * address comments * update checks * update * error code * fix mistake * fix of database table for large files (#1207) Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> --------- Co-authored-by: root <root@openml.org> Co-authored-by: joaquinvanschoren <joaquin.vanschoren@gmail.com> Co-authored-by: root <root@openml.test.tue.nl> Co-authored-by: Pieter Gijsbers <p.gijsbers@tue.nl> Co-authored-by: Jos van der Velde <j.d.v.d.velde@tue.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.