Found a total of 1 related content
Kuaishou's research result SAMP was recognized at the EMNLP2023 International Artificial Intelligence Conference
Article Introduction:With the wide application of deep learning models in fields such as natural language processing, model inference speed and performance have become important issues. Recently, the research result led by Kuaishou "SAMP: Post-training Quantitative Model Inference Library Based on Adaptive Mixed Precision" was successfully selected into the top conference EMNLP2023, and was demonstrated and shared in Singapore. The research proposed an inference acceleration tool called SAMP, Adaptive mixed-precision technology significantly improves inference speed while maintaining model performance. It contains an adaptive mixed-precision encoder and a series of advanced fusion strategies. The adaptive mixed-precision encoder can find the optimal floating-point and fixed-point mixed-precision combination among a large number of general matrix multiplication (GEMM) operations and Transformer layers to make the model
2023-12-15comment 0519