PUMA: Secure inference of LLaMA-7B in five minutesOACSCD

Transformer models(e.g., Bert and GPT) have shown their dominance in machine learning tasks. Many cloud companies have begun to provide services based on Transformer models, examples include translation and text-speech conversion. However, such a service inevitably requires access to the client''s data, which might contain sensitive information. Theoretically, running the services under secure multi-party computation(MPC)could protect clients'' privacy. However, current MPC frameworks are still limited in terms of model performance, efficiency, deployment, and functionality, especially when facing complex Transformer models. To this end, we propose an MPC framework Pum A to enable secure and efficient Transformer model inference. We first design high-quality approximations for the bottleneck functions in Transformers such as GELU and Softmax, reducing about20-76% computation and communication costs than state-of-the-art works without performance drop. Then, we provide concrete instantiations for secure Embedding and Layer Norm.These implementations produce correct results and integrate compatible system architectures of cleartext Transformer models. Finally, we conducted extensive experiments on six popular benchmarks: text classification/generation/summarization/translation, audio-to-text,and image-to-text. Results show that Pum A can finish most tasks in several minutes, with comparable model performance(e.g., accuracy) as cleartext, and even evaluate LLa MA-7B in less than 5 minutes to generate 1 token.