Tag:pre-train
All the articles with the tag "pre-train".
如何做 continued pre-train
Posted on:July 4, 2023 at 12:00 AM介绍一下 continued pre-train
CodeLLM Training Recipe
Posted on:July 26, 2023 at 12:00 AM一个偏综述的文章,总结 codeLLM 相关 paper 从 data collection 到 training 中间的一些细节
RoPE and Length Scaling
Posted on:August 10, 2023 at 12:00 AMIntroduce some basic concepts of Position Encoding, RoPE and length extrapolation related it.