what is the multi_query_group_num in config.json?
#75
by
lg920810
- opened
Hi~ Can anyone know the multi_query_group_num? Please help to understand the meaning of it.
"multi_query_attention": true,
"multi_query_group_num": 2,
"num_attention_heads": 32,
"num_layers": 28,