As deep learning models become increasingly prevalent in various applications, their computational demands grow, posing significant challenges for deployment on resource-constrained devices. One promising approach to address this issue is model pruning, which has demonstrated the ability to achieve high network compression levels with surprisingly little degradation in model accuracy. However, pruning often requires retraining the pruned model and also specialized hardware, making it infeasible in scenarios with strict hardware limitations. Channel merging, i.e., a technique that combines multiple structures (i.e., channels or neurons in Conv2D/Linear layers), offers an elegant solution to reduce model complexity and speed up inference without the need of retraining or specialized hardware. The goal of this thesis is to extend our previous work by investigating the impact of channel merging on each model’s class, quantifying which samples are affected, i.e., Pruning Identified Exemplars, and comparing these results with our earlier findings.
Download as PDF
Student Target Groups:
- Students in ICE
- Students in Computer Science
- Students Software Engineering
Thesis Type:
- Master Thesis / Master Project
Goal and Tasks:
- Conduct a thorough literature review on model compression and the properties of examples that the network forgets;
- Develop a theory by examining examples that the model forgets, possibly by estimating the impact of model merging on internal computations (see https://arxiv.org/pdf/2404.11534);
- Test the findings on semi-supervised or unsupervised trained networks;
- Present the results of your work and summarize the outcomes in a written report.
Recommended Prior Knowledge:
- Knowledge of neural networks;
- Programming skills in Python;
- Knowledge of PyTorch.
Used Tools & Equipment
- A compute cluster of TU Graz.
Start:
Contact: