This repository contains all code for my bachelor thesis "Structural Comparison of Winning Tickets from the Lottery Ticket Hypothesis".
In folder code_LTH, all code for generating, checking and saving the Winning Tickets can be found. This includes one jupyter notebook for each of the three datasets (CIFAR-10, CINIC-10 and SVHN). All Winning and random tickets are saved in the tickets folder by saving their weights as h5 file. For each ticket, the history of 10 epochs of training from the original model and the subnetwork is uploaded. This enables the comparison whether the subnetwork counts as Winning Ticket. Furthermore it includes a conv2.json file, containing the structure of a Conv-2 architecture and a zeros.h5 file, containing the weights information for a network which has been initialized with zeros everywhere.
In folder code_Distances, the three distance measures are implemented, including the computation and saving of 30 distance values for all 10 conditions in the NNSTD modified and the SaSD first-layer-first notebook. The computed distances are saved as csv files into the dists folder.
In the stat_analysis folder, the Rmarkdown (Rmd) files are stored. The file Make dists tidy.Rmd merges all distance files into a tidy table for all datasets and for each dataset separately and saved those in the dists folder as well. Tidy in this case means that each observation (each distance) has its own row, all factors that belong to this distance like the used method, the condition, layer and a unique ID for each comparison of two networks are saved in a column each. Analyse_dists.Rmd makes most of the plots and all statistical analyses for the main part of my thesis. One plot for the second part is composed there as well. For both Rmd files, the corresponding HTML file is included in the folder as well. The notebooks are concerned with the second part of the thesis. The notebook Comparing Structures conv1 layers contains all code for comparing structural aspects from Winning Tickets and random tickets. In Pruning and comparing conv1 layers, the main findings from the last notebook are used for special pruning methods, to identify how much these structural aspects are influencing the outcome of SaSD.