Feature implementation from commits b46bf82..6a47416

CHANGES.rst

@@ -85,7 +85,7 @@ Version 0.4.0
 -------------
 - Add `data` package that holds all data-related codes
 - Reform the data provider structure
-- Create a server for data centralized management `qlib-server<https://amc-msra.visualstudio.com/trading-algo/_git/qlib-server>`_
+- Create a server for data centralized management `qlib-server <https://amc-msra.visualstudio.com/trading-algo/_git/qlib-server>`_
 - Add a `ClientProvider` to work with server
 - Add a pluggable cache mechanism
 - Add a recursive backtracking algorithm to inspect the furthest reference date for an expression
@@ -166,12 +166,12 @@ Version 0.8.0
     - Nested decision execution framework is supported
     - There are lots of changes for daily trading, it is hard to list all of them. But a few important changes could be noticed
         - The trading limitation is more accurate;
-            - In `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/backtest/exchange.py#L160>`_, longing and shorting actions share the same action.
-            - In `current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/backtest/exchange.py#L304>`_, the trading limitation is different between logging and shorting action.
+            - In `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/backtest/exchange.py#L160>`__, longing and shorting actions share the same action.
+            - In `current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/backtest/exchange.py#L304>`__, the trading limitation is different between logging and shorting action.
         - The constant is different when calculating annualized metrics.
-            - `Current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/contrib/evaluate.py#L42>`_ uses more accurate constant than `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/evaluate.py#L22>`_
-        - `A new version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/tests/data.py#L17>`_ of data is released. Due to the unstability of Yahoo data source, the data may be different after downloading data again.
-        - Users could check out the backtesting results between  `Current version <https://github.com/microsoft/qlib/tree/7c31012b507a3823117bddcc693fc64899460b2a/examples/benchmarks>`_ and `previous version <https://github.com/microsoft/qlib/tree/v0.7.2/examples/benchmarks>`_
+            - `Current version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/contrib/evaluate.py#L42>`_ uses more accurate constant than `previous version <https://github.com/microsoft/qlib/blob/v0.7.2/qlib/contrib/evaluate.py#L22>`__
+        - `A new version <https://github.com/microsoft/qlib/blob/7c31012b507a3823117bddcc693fc64899460b2a/qlib/tests/data.py#L17>`__ of data is released. Due to the unstability of Yahoo data source, the data may be different after downloading data again.
+        - Users could check out the backtesting results between  `Current version <https://github.com/microsoft/qlib/tree/7c31012b507a3823117bddcc693fc64899460b2a/examples/benchmarks>`__ and `previous version <https://github.com/microsoft/qlib/tree/v0.7.2/examples/benchmarks>`__
 
 
 Other Versions

docs/advanced/alpha.rst

@@ -38,7 +38,7 @@ Example
 
         DIF = \frac{EMA(CLOSE, 12) - EMA(CLOSE, 26)}{CLOSE}
 
-    `DEA`means a 9-period EMA of the DIF.
+    `DEA` means a 9-period EMA of the DIF.
 
     .. math::
 

docs/advanced/task_management.rst

@@ -18,7 +18,7 @@ With this module, users can run their ``task`` automatically at different period
 
 This whole process can be used in `Online Serving <../component/online.html>`_.
 
-An example of the entire process is shown `here <https://github.com/microsoft/qlib/tree/main/examples/model_rolling/task_manager_rolling.py>`_.
+An example of the entire process is shown `here <https://github.com/microsoft/qlib/tree/main/examples/model_rolling/task_manager_rolling.py>`__.
 
 Task Generating
 ===============
@@ -31,9 +31,10 @@ Here is the base class of ``TaskGen``:
 
 .. autoclass:: qlib.workflow.task.gen.TaskGen
     :members:
+    :noindex:
 
 ``Qlib`` provides a class `RollingGen <https://github.com/microsoft/qlib/tree/main/qlib/workflow/task/gen.py>`_ to generate a list of ``task`` of the dataset in different date segments.
-This class allows users to verify the effect of data from different periods on the model in one experiment. More information is `here <../reference/api.html#TaskGen>`_.
+This class allows users to verify the effect of data from different periods on the model in one experiment. More information is `here <../reference/api.html#TaskGen>`__.
 
 Task Storing
 ============
@@ -53,8 +54,9 @@ Users need to provide the MongoDB URL and database name for using ``TaskManager`
 
 .. autoclass:: qlib.workflow.task.manage.TaskManager
     :members:
+    :noindex:
 
-More information of ``Task Manager`` can be found in `here <../reference/api.html#TaskManager>`_.
+More information of ``Task Manager`` can be found in `here <../reference/api.html#TaskManager>`__.
 
 Task Training
 =============
@@ -64,11 +66,13 @@ An easy way to get the ``task_func`` is using ``qlib.model.trainer.task_train``
 It will run the whole workflow defined by ``task``, which includes *Model*, *Dataset*, *Record*.
 
 .. autofunction:: qlib.workflow.task.manage.run_task
+    :noindex:
 
 Meanwhile, ``Qlib`` provides a module called ``Trainer``. 
 
 .. autoclass:: qlib.model.trainer.Trainer
     :members:
+    :noindex:
 
 ``Trainer`` will train a list of tasks and return a list of model recorders.
 ``Qlib`` offer two kinds of Trainer, TrainerR is the simplest way and TrainerRM is based on TaskManager to help manager tasks lifecycle automatically. 

docs/component/data.rst

@@ -24,8 +24,8 @@ The introduction of ``Data Layer`` includes the following parts.
 Here is a typical example of Qlib data workflow
 
 - Users download data and converting data into Qlib format(with filename suffix `.bin`).  In this step, typically only some basic data are stored on disk(such as OHLCV).
-- Creating some basic features based on Qlib's expression Engine(e.g. "Ref($close, 60) / $close", the return of last 60 trading days). Supported operators in the expression engine can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/ops.py>`_. This step is typically implemented in Qlib's `Data Loader <https://qlib.readthedocs.io/en/latest/component/data.html#data-loader>`_ which is a component of `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ .
-- If users require more complicated data processing (e.g. data normalization),  `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ support user-customized processors to process data(some predefined processors can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`_).  The processors are different from operators in expression engine. It is designed for some complicated data processing methods which is hard to supported in operators in expression engine.
+- Creating some basic features based on Qlib's expression Engine(e.g. "Ref($close, 60) / $close", the return of last 60 trading days). Supported operators in the expression engine can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/ops.py>`__. This step is typically implemented in Qlib's `Data Loader <https://qlib.readthedocs.io/en/latest/component/data.html#data-loader>`_ which is a component of `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ .
+- If users require more complicated data processing (e.g. data normalization),  `Data Handler <https://qlib.readthedocs.io/en/latest/component/data.html#data-handler>`_ support user-customized processors to process data(some predefined processors can be found `here <https://github.com/microsoft/qlib/blob/main/qlib/data/dataset/processor.py>`__).  The processors are different from operators in expression engine. It is designed for some complicated data processing methods which is hard to supported in operators in expression engine.
 - At last, `Dataset <https://qlib.readthedocs.io/en/latest/component/data.html#dataset>`_ is responsible to prepare model-specific dataset from the processed data of Data Handler
 
 Data Preparation
@@ -37,7 +37,7 @@ Qlib Format Data
 We've specially designed a data structure to manage financial data, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information.
 Such data will be stored with filename suffix `.bin` (We'll call them `.bin` file, `.bin` format, or qlib format). `.bin` file is designed for scientific computing on finance data.
 
-``Qlib`` provides two different off-the-shelf datasets, which can be accessed through this `link <https://github.com/microsoft/qlib/blob/main/qlib/contrib/data/handler.py>`_:
+``Qlib`` provides two different off-the-shelf datasets, which can be accessed through this `link <https://github.com/microsoft/qlib/blob/main/qlib/contrib/data/handler.py>`__:
 
 ========================  =================  ================
 Dataset                   US Market          China Market
@@ -47,7 +47,7 @@ Alpha360                  √                  √
 Alpha158                  √                  √
 ========================  =================  ================
 
-Also, ``Qlib`` provides a high-frequency dataset. Users can run a high-frequency dataset example through this `link <https://github.com/microsoft/qlib/tree/main/examples/highfreq>`_.
+Also, ``Qlib`` provides a high-frequency dataset. Users can run a high-frequency dataset example through this `link <https://github.com/microsoft/qlib/tree/main/examples/highfreq>`__.
 
 Qlib Format Dataset
 -------------------
@@ -332,6 +332,7 @@ Here are some interfaces of the ``QlibDataLoader`` class:
 
 .. autoclass:: qlib.data.dataset.loader.DataLoader
     :members:
+    :noindex:
 
 API
 ---
@@ -361,6 +362,7 @@ Here are some important interfaces that ``DataHandlerLP`` provides:
 
 .. autoclass:: qlib.data.dataset.handler.DataHandlerLP
     :members: __init__, fetch, get_cols
+    :noindex:
 
 
 If users want to load features and labels by config, users can define a new handler and call the static method `parse_config_to_fields` of ``qlib.contrib.data.handler.Alpha158``.
@@ -451,6 +453,7 @@ The ``DatasetH`` class is the `dataset` with `Data Handler`. Here is the most im
 
 .. autoclass:: qlib.data.dataset.__init__.DatasetH
     :members:
+    :noindex:
 
 API
 ---
@@ -470,9 +473,11 @@ Global Memory Cache
 
 .. autoclass:: qlib.data.cache.MemCacheUnit
     :members:
+    :noindex:
 
 .. autoclass:: qlib.data.cache.MemCache
     :members:
+    :noindex:
 
 
 ExpressionCache
@@ -487,6 +492,7 @@ The following shows the details about the interfaces:
 
 .. autoclass:: qlib.data.cache.ExpressionCache
     :members:
+    :noindex:
 
 ``Qlib`` has currently provided implemented disk cache `DiskExpressionCache` which inherits from `ExpressionCache` . The expressions data will be stored in the disk.
 
@@ -502,6 +508,7 @@ The following shows the details about the interfaces:
 
 .. autoclass:: qlib.data.cache.DatasetCache
     :members:
+    :noindex:
 
 ``Qlib`` has currently provided implemented disk cache `DiskDatasetCache` which inherits from `DatasetCache` . The datasets' data will be stored in the disk.
 
@@ -512,7 +519,7 @@ Data and Cache File Structure
 
 We've specially designed a file structure to manage data and cache, please refer to the `File storage design section in Qlib paper <https://arxiv.org/abs/2009.11189>`_ for detailed information. The file structure of data and cache is listed as follows.
 
-.. code-block:: json
+.. code-block::
 
     - data/
         [raw data] updated by data providers

docs/component/model.rst

@@ -20,6 +20,7 @@ The base class provides the following interfaces:
 
 .. autoclass:: qlib.model.base.Model
     :members:
+    :noindex:
 
 ``Qlib`` also provides a base class `qlib.model.base.ModelFT <../reference/api.html#qlib.model.base.ModelFT>`_, which includes the method for finetuning the model.
 

docs/component/online.rst

@@ -1,4 +1,4 @@
-.. _online:
+.. _online_serving:
 
 ==============
 Online Serving
@@ -32,21 +32,25 @@ Online Manager
 
 .. automodule:: qlib.workflow.online.manager
     :members:
+    :noindex:
 
 Online Strategy
 ===============
 
 .. automodule:: qlib.workflow.online.strategy
     :members:
+    :noindex:
 
 Online Tool
 ===========
 
 .. automodule:: qlib.workflow.online.utils
     :members:
+    :noindex:
 
 Updater
 =======
 
 .. automodule:: qlib.workflow.online.update
     :members:
+    :noindex:

docs/component/recorder.rst

@@ -61,6 +61,7 @@ The ``ExpManager`` module in ``Qlib`` is responsible for managing different expe
 
 .. autoclass:: qlib.workflow.expm.ExpManager
     :members: get_exp, list_experiments
+    :noindex:
 
 For other interfaces such as `create_exp`, `delete_exp`, please refer to `Experiment Manager API <../reference/api.html#experiment-manager>`_.
 
@@ -71,6 +72,7 @@ The ``Experiment`` class is solely responsible for a single experiment, and it w
 
 .. autoclass:: qlib.workflow.exp.Experiment
     :members: get_recorder, list_recorders
+    :noindex:
 
 For other interfaces such as `search_records`, `delete_recorder`, please refer to `Experiment API <../reference/api.html#experiment>`_.
 
@@ -85,6 +87,7 @@ Here are some important APIs that are not included in the ``QlibRecorder``:
 
 .. autoclass:: qlib.workflow.recorder.Recorder
     :members: list_artifacts, list_metrics, list_params, list_tags
+    :noindex:
 
 For other interfaces such as `save_objects`, `load_object`, please refer to `Recorder API <../reference/api.html#recorder>`_.
 

docs/component/report.rst

@@ -51,6 +51,7 @@ API
 
 .. automodule:: qlib.contrib.report.analysis_position.report
     :members:
+    :noindex:
 
 Graphical Result
 ~~~~~~~~~~~~~~~~
@@ -93,6 +94,7 @@ API
 
 .. automodule:: qlib.contrib.report.analysis_position.score_ic
     :members:
+    :noindex:
 
 
 Graphical Result
@@ -151,6 +153,7 @@ API
 
 .. automodule:: qlib.contrib.report.analysis_position.risk_analysis
     :members:
+    :noindex:
 
 
 Graphical Result
@@ -174,6 +177,7 @@ Graphical Result
                 The `Information Ratio` without cost.
             - `excess_return_with_cost`
                 The `Information Ratio` with cost.
+
             To know more about `Information Ratio`, please refer to `Information Ratio – IR <https://www.investopedia.com/terms/i/informationratio.asp>`_.
         -  `max_drawdown`
             - `excess_return_without_cost`
@@ -269,6 +273,7 @@ API
 
 .. automodule:: qlib.contrib.report.analysis_model.analysis_model_performance
     :members:
+    :noindex:
 
 
 Graphical Results

docs/component/rl/framework.rst

@@ -28,7 +28,7 @@ In QlibRL, EnvWrapper is a subclass of gym.Env, so it implements all necessary i
 
 EnvWrapper will organically organize these components. Such decomposition allows for better flexibility in development. For example, if the developers want to train multiple types of policies in the same environment, they only need to design one simulator and design different state interpreters/action interpreters/reward functions for different types of policies.
 
-QlibRL has well-defined base classes for all these 4 components. All the developers need to do is define their own components by inheriting the base classes and then implementing all interfaces required by the base classes. The API for the above base components can be found `here <../../reference/api.html#module-qlib.rl>`_.
+QlibRL has well-defined base classes for all these 4 components. All the developers need to do is define their own components by inheriting the base classes and then implementing all interfaces required by the base classes. The API for the above base components can be found `here <../../reference/api.html#module-qlib.rl>`__.
 
 Policy
 ------------
@@ -42,4 +42,4 @@ As you may have noticed, a training vessel itself holds all the required compone
 
 With a training vessel, the trainer could finally launch the training pipeline by simple, Scikit-learn-like interfaces (i.e., ``trainer.fit()``).
 
-The API for Trainer and TrainingVessel and can be found `here <../../reference/api.html#module-qlib.rl.trainer>`_.
+The API for Trainer and TrainingVessel and can be found `here <../../reference/api.html#module-qlib.rl.trainer>`__.

docs/component/strategy.rst

@@ -80,6 +80,7 @@ TopkDropoutStrategy
         In most cases, ``TopkDrop`` algorithm sells and buys `Drop` stocks every trading day, which yields a turnover rate of 2$\times$`Drop`/$K$.
 
         The following images illustrate a typical scenario.
+
         .. image:: ../_static/img/topk_drop.png
             :alt: Topk-Drop
 

docs/conf.py

@@ -77,7 +77,7 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This patterns also effect to html_static_path and html_extra_path
-exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "hidden"]
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = "sphinx"

docs/developer/code_standard_and_dev_guide.rst

@@ -15,7 +15,8 @@ Continuous Integration (CI) tools help you stick to the quality standards by run
 When you submit a PR request, you can check whether your code passes the CI tests in the "check" section at the bottom of the web page.
 
 1. Qlib will check the code format with black. The PR will raise error if your code does not align to the standard of Qlib(e.g. a common error is the mixed use of space and tab).
- You can fix the bug by inputing the following code in the command line.
+
+   You can fix the bug by inputting the following code in the command line.
 
 .. code-block:: bash
 
@@ -32,15 +33,17 @@ When you submit a PR request, you can check whether your code passes the CI test
 
 
 3. Qlib will check your code style flake8. The checking command is implemented in [github action workflow](https://github.com/microsoft/qlib/blob/0e8b94a552f1c457cfa6cd2c1bb3b87ebb3fb279/.github/workflows/test.yml#L73).
- You can fix the bug by inputing the following code in the command line.
+
+   You can fix the bug by inputing the following code in the command line.
 
 .. code-block:: bash
 
     flake8 --ignore E501,F541,E402,F401,W503,E741,E266,E203,E302,E731,E262,F523,F821,F811,F841,E713,E265,W291,E712,E722,W293 qlib
 
 
 4. Qlib has integrated pre-commit, which will make it easier for developers to format their code.
- Just run the following two commands, and the code will be automatically formatted using black and flake8 when the git commit command is executed.
+
+   Just run the following two commands, and the code will be automatically formatted using black and flake8 when the git commit command is executed.
 
 .. code-block:: bash
 

docs/hidden/client.rst

@@ -81,6 +81,7 @@ If running on Windows, open **NFS** features and write correct **mount_path**, i
     * Open ``Programs and Features``.
     * Click ``Turn Windows features on or off``.
     * Scroll down and check the option ``Services for NFS``, then click OK
+
     Reference address: https://graspingtech.com/mount-nfs-share-windows-10/
 2.config correct mount_path
     * In windows, mount path must be not exist path and root path,
@@ -161,7 +162,7 @@ Limitations
 API
 ***
 
-The client is based on `python-socketio<https://python-socketio.readthedocs.io>`_ which is a framework that supports WebSocket client for Python language. The client can only propose requests and receive results, which do not include any calculating procedure.
+The client is based on `python-socketio <https://python-socketio.readthedocs.io>`_ which is a framework that supports WebSocket client for Python language. The client can only propose requests and receive results, which do not include any calculating procedure.
 
 Class
 -----

docs/index.rst

@@ -56,6 +56,12 @@ Document Structure
    Task Management <advanced/task_management.rst>
    Point-In-Time database <advanced/PIT.rst>
 
+.. toctree::
+   :maxdepth: 3
+   :caption: FOR DEVELOPERS:
+
+   Code Standard & Development Guidance <developer/code_standard_and_dev_guide.rst>
+
 .. toctree::
    :maxdepth: 3
    :caption: REFERENCE: