from yogthos@lemmy.ml to technology@lemmy.ml on 02 Jul 14:11
https://lemmy.ml/post/32583279
This paper introduces DiffuCoder, a 7B-scale open-source masked diffusion large language model (dLLM) specifically designed for code generation.
The research provides insights into how dLLMs generate content, distinguishing their decoding behavior from that of autoregressive (AR) models. Unlike AR models, dLLMs can intrinsically adjust their generation causality and increasing sampling temperature diversifies not just token choices but also their generation order, creating a rich search space for reinforcement learning (RL).
This flexibility allows dLLMs to be more non-autoregressive and generate tokens in a less sequential, more “human-like” code writing manner.
To leverage this diversity and improve performance, the paper proposes coupled-GRPO RL algorithm. This method utilizes a coupled-sampling scheme that constructs complementary mask noise during training to reduce the variance of token log-likelihood estimates while maintaining training efficiency.
Experimentally, coupled-GRPO significantly boosts DiffuCoder’s performance on code generation benchmarks, notably improving EvalPlus scores by 4.4% with training on only 21K samples. The research also shows that coupled-GRPO trained models experience a smaller performance drop when decoding steps are halved (resulting in a 2x speedup), indicating increased parallelism and reduced reliance on AR bias during decoding.
available at huggingface.co/apple/DiffuCoder-7B-cpGRPO
threaded - newest