【英文原版】Gemma技术报告-16页

2024-02-21Gemma:OpenModelsBasedonGeminiResearchandTechnologyGemmaTeam,GoogleDeepMind11SeeContributionsandAcknowledgmentssectionforfullauthorlist.Pleasesendcorrespondencetogemma-1-report@google.com.ThisworkintroducesGemma,afamilyoflightweight,state-of-theartopenmodelsbuiltfromtheresearchandtechnologyusedtocreateGeminimodels.Gemmamodelsdemonstratestrongperformanceacrossacademicbenchmarksforlanguageunderstanding,reasoning,andsafety.Wereleasetwosizesofmodels(2billionand7billionparameters),andprovidebothpretrainedandfine-tunedcheckpoints.Gemmaoutperformssimilarlysizedopenmodelson11outof18text-basedtasks,andwepresentcomprehensiveevaluationsofsafetyandresponsibilityaspectsofthemodels,alongsideadetaileddescriptionofmodeldevelopment.WebelievetheresponsiblereleaseofLLMsiscriticalforimprovingthesafetyoffrontiermodels,andforenablingthenextwaveofLLMinnovations.Introductionmentmethodologies.WepresentGemma,afamilyofopenmodelsGemmaadvancesstate-of-the-artperformancebasedonGoogle’sGeminimodels(GeminiTeam,relativetocomparable-scale(andsomelarger),2023).openmodels(Almazroueietal.,2023;Jiangetal.,2023;Touvronetal.,2023a,b)acrossaWetrainedGemmamodelsonupto6Tto-widerangeofdomainsincludingbothautomatedkensoftext,usingsimilararchitectures,data,benchmarksandhumanevaluation.Exampledo-andtrainingrecipesastheGeminimodelfamily.mainsincl...