From 28f8c7715e38e64675b15636d4ac83859919e539 Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Fri, 11 Nov 2022 14:07:53 +0800 Subject: [PATCH 01/11] Dev (#32) * Update README.md * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Dev (#13) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * Dev (#24) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#25) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * simplify codes * refactor: more robust freq print implementation * update readme * Dev (#14) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * Dev (#24) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#25) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Dev (#26) * Update README.md * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Update README.md (#28) * simplify codes * refactor: more robust freq print implementation * update readme * fix(exp_loader): fix a bug of loaded experiments information print * fix param name * feat(complex-data-recorder): add image recorder * feat(exp_manager): add hyper-parameter log in `code` dir * docs(readme): add gitignore document From c20545bf32bc2103188abc33f6d6bdfdc518de99 Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Fri, 11 Nov 2022 15:38:38 +0800 Subject: [PATCH 02/11] Dev (#33) * Update README.md * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Dev (#13) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * Dev (#24) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#25) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * simplify codes * refactor: more robust freq print implementation * update readme * Dev (#14) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * Dev (#24) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#25) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Dev (#26) * Update README.md * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Update README.md (#28) * simplify codes * refactor: more robust freq print implementation * update readme * fix(exp_loader): fix a bug of loaded experiments information print * fix param name * feat(complex-data-recorder): add image recorder * feat(exp_manager): add hyper-parameter log in `code` dir * docs(readme): add gitignore document From 977dfac223252821701accf69cfaa68adc9fbf8e Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Fri, 11 Nov 2022 15:48:52 +0800 Subject: [PATCH 03/11] fix(example): fix ignored conflicts --- example/simplest_code/project/main.py | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/example/simplest_code/project/main.py b/example/simplest_code/project/main.py index 84ac737..b8e1458 100644 --- a/example/simplest_code/project/main.py +++ b/example/simplest_code/project/main.py @@ -28,16 +28,6 @@ def get_param(): task_name = 'demo_task' rla_data_root = '../' -<<<<<<< HEAD -<<<<<<< HEAD -exp_manager.configure(task_name, rla_config='../rla_config.yaml', data_root=rla_data_root, - ignore_file_path='./ignore') -======= -exp_manager.configure(task_name, rla_config='../rla_config.yaml', data_root=rla_data_root) ->>>>>>> 9bd402e505cc920aa4329f451ac34fb3b12f6347 -======= -exp_manager.configure(task_name, rla_config='../rla_config.yaml', data_root=rla_data_root) ->>>>>>> 29ab768949f26c307e4bdb07fd9d0dc15047a69d exp_manager.log_files_gen() exp_manager.print_args() From 53397766e139c145631eba9732db6c5625a88126 Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Wed, 13 Jul 2022 12:28:31 +0800 Subject: [PATCH 04/11] fix: fix bugs of torch-version ckp loader --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index 055d82a..a29fbf9 100644 --- a/README.md +++ b/README.md @@ -277,6 +277,5 @@ PS: - [ ] add an auto integration script. - [ ] download / upload experiment logs through timestamp. - [ ] add a document to the plot function. -- [ ] allow sync LOG only or ALL TYPE LOGS. -- [x] support aim and smarter logger. +- [ ] allow sync LOG only or ALL TYPE LOGS. - [ ] add unit_test to ckp loader. From 969a0f5f1b5ac6be4ba92c955ac7bccce7f82aa9 Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Thu, 14 Jul 2022 10:58:03 +0800 Subject: [PATCH 05/11] Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility --- RLA/easy_log/exp_loader.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/RLA/easy_log/exp_loader.py b/RLA/easy_log/exp_loader.py index 56f7c4a..c2206b1 100644 --- a/RLA/easy_log/exp_loader.py +++ b/RLA/easy_log/exp_loader.py @@ -82,8 +82,9 @@ def load_from_record_date(self, var_prefix: Optional[str] = None, variable_list: """ if self.is_valid_config: loaded_tester = Tester.load_tester(self.load_date, self.task_name, self.data_root) - print("attrs of the loaded tester") - pprint(loaded_tester.__dict__) + if verbose: + print("attrs of the loaded tester") + pprint(loaded_tester.__dict__) # load checkpoint load_res = {} if var_prefix is not None: From d3f274b465390b41ab97f4301d0e8dc194ac33e7 Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Thu, 14 Jul 2022 16:49:26 +0800 Subject: [PATCH 06/11] Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading --- RLA/easy_log/tester.py | 1 + 1 file changed, 1 insertion(+) diff --git a/RLA/easy_log/tester.py b/RLA/easy_log/tester.py index 31fa723..f25b472 100644 --- a/RLA/easy_log/tester.py +++ b/RLA/easy_log/tester.py @@ -695,6 +695,7 @@ def load_checkpoint(self, ckp_index=None): pprint.pprint(all_ckps) if ckp_index is None: ckp_index = all_ckps[-1].split('checkpoint-')[1].split('.pt')[0] + print("loaded checkpoints:", "checkpoint-{}.pt".format(ckp_index)) return ckp_index, torch.load(self.checkpoint_dir + "checkpoint-{}.pt".format(ckp_index)) def auto_parse_info(self): From e78641b935e6ce474d8be18aa909238361ce6fbb Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Thu, 11 Aug 2022 01:14:13 +0800 Subject: [PATCH 07/11] Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability --- RLA/easy_log/exp_loader.py | 5 ++--- RLA/easy_log/tester.py | 2 +- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/RLA/easy_log/exp_loader.py b/RLA/easy_log/exp_loader.py index c2206b1..56f7c4a 100644 --- a/RLA/easy_log/exp_loader.py +++ b/RLA/easy_log/exp_loader.py @@ -82,9 +82,8 @@ def load_from_record_date(self, var_prefix: Optional[str] = None, variable_list: """ if self.is_valid_config: loaded_tester = Tester.load_tester(self.load_date, self.task_name, self.data_root) - if verbose: - print("attrs of the loaded tester") - pprint(loaded_tester.__dict__) + print("attrs of the loaded tester") + pprint(loaded_tester.__dict__) # load checkpoint load_res = {} if var_prefix is not None: diff --git a/RLA/easy_log/tester.py b/RLA/easy_log/tester.py index f25b472..66f7807 100644 --- a/RLA/easy_log/tester.py +++ b/RLA/easy_log/tester.py @@ -15,6 +15,7 @@ import datetime import os.path as osp import pprint +import numpy as np from RLA.easy_log.time_step import time_step_holder @@ -695,7 +696,6 @@ def load_checkpoint(self, ckp_index=None): pprint.pprint(all_ckps) if ckp_index is None: ckp_index = all_ckps[-1].split('checkpoint-')[1].split('.pt')[0] - print("loaded checkpoints:", "checkpoint-{}.pt".format(ckp_index)) return ckp_index, torch.load(self.checkpoint_dir + "checkpoint-{}.pt".format(ckp_index)) def auto_parse_info(self): From 58bfe0948fd793deef6aa4234d31d9431eda5a28 Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Tue, 4 Oct 2022 20:17:08 +0800 Subject: [PATCH 08/11] Dev (#14) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * Dev (#24) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#25) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Dev (#26) * Update README.md * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Update README.md (#28) * simplify codes * refactor: more robust freq print implementation * update readme --- example/simplest_code/project/main.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/example/simplest_code/project/main.py b/example/simplest_code/project/main.py index b8e1458..a142802 100644 --- a/example/simplest_code/project/main.py +++ b/example/simplest_code/project/main.py @@ -28,6 +28,9 @@ def get_param(): task_name = 'demo_task' rla_data_root = '../' + +exp_manager.configure(task_name, rla_config='../rla_config.yaml', data_root=rla_data_root, + ignore_file_path='./ignore') exp_manager.log_files_gen() exp_manager.print_args() From 049a3a24555cea534edfe1190363f955fde802dd Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Fri, 11 Nov 2022 15:40:06 +0800 Subject: [PATCH 09/11] Dev (#17) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * Dev (#24) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#25) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Dev (#26) * Update README.md * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * Update README.md (#28) * simplify codes * refactor: more robust freq print implementation * update readme * Dev (#29) * Update README.md * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#10) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#11) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * Dev (#12) * Update README.md * Dev (#20) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * Dev (#21) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * Dev (#22) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * Dev (#23) * fix: fix bugs of torch-version ckp loader * refactor: add sync_timestep for hp loader * fix: minor changes for version compatibility * fix: a bug of sorting in torch-version checkpoint loading * refactor: robust multi-key plot implementation * feat: supoort pretty plotter * refactor(log plotter): record scores of the log plotter * fix(exp_loader): add parameter ckp_index * update readme * rm unsolved merge * feat: tf-v2 compatible * refactor: add timestep recorder. refactor on exp_loader * test: add test data * feat(plot): track the hyper-parameter from the exp_manager instead of the experiment name. refactor the plot_func for better readability * test(plot): add user cases and documents * test(plot): add user cases * simplify codes * refactor: more robust freq print implementation * update readme * fix(exp_loader): fix a bug of loaded experiments information print * fix param name * feat(complex-data-recorder): add image recorder * feat(exp_manager): add hyper-parameter log in `code` dir * docs(readme): add gitignore document From dc037cb8914ae5429b19449bec9c1ccf5f46770d Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Fri, 11 Nov 2022 17:32:08 +0800 Subject: [PATCH 10/11] update readme --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index a29fbf9..613f5d1 100644 --- a/README.md +++ b/README.md @@ -270,6 +270,10 @@ PS: 2. An alternative way is building your own NFS for your physical machines and locate data_root to the NFS. +## further usage demos + +We write the usage of RLA as unit tests. You can check the scripts in `test` folder for more details (include test_plot.py, test_scripts.py, and test_proj/proj/test_manger.py). + # TODO - [ ] support custom data structure saving and loading. - [ ] support video visualization. From 8b114ff7f87c6aad257e81ee7b446bb2985bedfe Mon Sep 17 00:00:00 2001 From: Xiong-Hui Chen Date: Fri, 11 Nov 2022 19:49:48 +0800 Subject: [PATCH 11/11] update readme --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 613f5d1..763e8e5 100644 --- a/README.md +++ b/README.md @@ -270,7 +270,7 @@ PS: 2. An alternative way is building your own NFS for your physical machines and locate data_root to the NFS. -## further usage demos +## more demos We write the usage of RLA as unit tests. You can check the scripts in `test` folder for more details (include test_plot.py, test_scripts.py, and test_proj/proj/test_manger.py).