代码生成研究脉络

https://paperswithcode.com/sota/code-generation-on-conala

1	PanGu-Coder-FT-I	44.32			Fine-Tuning Large Language Models for Answering Programming Questions with Code Snippets			2023
2	RoBERTaMarian	35.74	13.8		Leveraging pre-trained language models for code generation			2023
3	MarianCG	34.43	10.2		MarianCG: a code generation transformer model inspired by machine translation			2022
4	TranX + BERT w/mined	34.2	5.8		The impact of lexical and grammatical processing on generating code from natural language			2022
5	BERT + TAE	33.41			Code Generation from Natural Language with Less Prior and More Monolingual Data			2021
6	BERTMarian	32.46	12.40		Leveraging pre-trained language models for code generation			2023
7	External Knowledge With API + Reranking	32.26			Incorporating External Knowledge through Pre-training for Natural Language to Code Generation			2020
8	External Knowledge With API	30.69			Incorporating External Knowledge through Pre-training for Natural Language to Code Generation			2020
9	BART W/ Mined	30.55			Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation			2021
10	ELECTRAMarian	30.18	10.0		Leveraging pre-trained language models for code generation			2023
11	Reranker	30.11	2.8		Reranking for Neural Semantic Parsing			2019
12	LUKEMarian	29.83	7.6		Leveraging pre-trained language models for code generation			2023
13	BART Base	26.24			Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation			2021
14	TranX	24.30			TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation			2018

引用：

官网：

CoNaLa: The Code/Natural Language Challenge

CMU官方预处理脚本：

GitHub - conala-corpus/conala-baseline: Baseline for the Conala: Code/Natural Language Challenge

Second SOTA使用的数据集：CoNaLa-Large
https://huggingface.co/datasets/AhmedSSoliman/CoNaLa-Large