Compressing Pre-trained Language Models by Matrix Decomposition

Keyphrases

• Number of Parameters 100%
• Pre-trained Language Model 100%
• Matrix Decomposition 100%
• Internal Representation 50%
• Computational Requirements 50%
• Compression Method 50%
• Benchmark Dataset 50%
• Memory Requirements 50%
• Stage Model 50%
• Model Size 50%
• Model Inference 50%
• Inference Speed 50%
• NLP Tasks 50%
• Inference Time 50%
• Metric Performance 50%
• Green AI 50%
• Small Matrices 50%
• Minimal Losses 50%
• BERT Base 50%
• Pre-trained Large Language Models 50%
• Model Compression 50%
• Feature Distillation 50%
• GLUE Benchmark 50%

Computer Science

• Green AI 50%