パラメータ削減

書いてみる

関連タグ

#モデル (16,362)
#ランク (922)
#GELU活性化 (1)
#GLU活性化 (1)
#MABViT (1)
#VisionTransformer (13)

Characterizing the Accuracy - Efficiency Trade-off of Low-rank Decomposition in Language Models

Ikemen Mas Kot
8か月前

Compute Better Spent: Replacing Dense Layers with Structured Matrices

Ikemen Mas Kot
8か月前

MABViT -- Modified Attention Block Enhances Vision Transformers

Ikemen Mas Kot
9か月前

Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity

Ikemen Mas Kot
1年前