Documentation Update Part 2

[stelladk]

TODO:

• Check all documentation
• Update docs for convolution attributes
• Create example for GrowingDAG
• Reduce warnings
• Check that all tests use named arguments
• Explain differences between GrowingContainer, GrowingModel, and SequentialGrowingContainer

Sébastian suggestions:

• Example with complete training-growing loop
• Tuning hyperparameters is not clear

Alex suggestions:

• More examples on the containers
• Conditional experiment setups (ligher for CI)

Reminder to address Issues:

• Current implementation of *_volume in GrowingModule is wrong #153
• Implement in_features, in_volume, input_shape... #140
• Remove functionalities for experiments #71
• Warning in test_GrowableDAG.py #36

Notes for documentation:
I managed to grow layers that have an average pooling in between. Use case: GrowingDAG -> downsampling -> GrowingDAG. Now the hidden dimensions between GrowingContainers can be increased.
Constraints: (1) each GrowingDAG starts and ends with a MergeGrowingModule so effectively the order of operations is Merge -> Merge. (2) the first MergeGrowingModule needs to be aware of the correct output shape in order to grow so the pooling operation needs to be inside the post_merge_function. It is not possible to grow if it appears directly in the forward function. (PR #179)

I fixed the issue with the GlobalAveragePooling by correcting the padding. Now we can also grow Use case: MergeGrowingModule -> GlobalAveragePooling ->MergeGrowingModule. (PR #179)
I found a way to also grow Use case: Conv -> GlobalAveragePooling -> Flatten -> Lin. For the GrowingDAG Conv -> Lin is equivalent to Conv -> Conv1x1.
Constraints: (1) the global average pooling is still in the post_merge_function (2) the flattening CANNOT be in the post_merge_function as it does not allow the unfolding of the activity (reminder: the activity is the post-activity and if we flatten in the module the activity is flat) (no, I cannot just not unfold the activity because it will change the tensor_s shape). Instead Flatten should be called in the forward function. (3) Because this creates a shape mismatch in ωσ(αx) I added a new attribute in Conv2dMergeGrowingModule that keeps track of potential reshapes (e.g. flatten before a linear layer) and applies it when necessary. One of this situations is the calculation of the output_volume of the module. (PR #177) (4) When growing Conv -> Lin the previous layers α should use the tensor_s of the last Conv2dMergeGrowingModule and the next layers ω should use the tensor_s of the first LinearMergeGrowingModule. So now the second MergeGrowingModule in a Merge -> Merge sequence is always updating the shape of its tensor_s. (PR #179)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
unittests	93.96% <ø> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[stelladk]

There is still an open discussion about ruff checks SIM102, SIM117, and E501.

• "SIM102", # forces non-nested if statements
• "SIM117", # nested with statements
• "E501", # line too long

SIM102, SIM117 seems good rules.
E501 is abit of a pain but may worth it.

The suggestions from SIM102 and SIM117 seem unreadable to me, but I will leave this to the decision of the majority. E501 was clashing with the pre-commit limit

[TheoRudkiewicz]

Does this https://growingnet.github.io/gromo/auto_examples/plot_growing_container_tutorial.html correspond to Alex and Sebastien first suggestion ? If not, we could discuss what is missing.
About Sebastien second point unfortunately I don't know how to properly do it, so I don't think we can document it.

[TheoRudkiewicz]

For the example, what I would recommend is to create a branch doc-aux on you repository and edit the the workflow of this branch with:

on:
  push:
    branches: [ main, docs-aux ]
  pull_request:
    branches: [ main, docs-aux ]

then you can merge this branch (docs) into docs-aux and push docs-aux so that the https://stelladk.github.io/gromo/auto_examples/index.html shows the example already built.

Thank you for changing hidden_features to hidden_neurons.