nlpaug 自然语言 文字增强

1.同义词替换

import nlpaug.augmenter.word as naw
text = "hello Word!"
aug = naw.SynonymAug(aug_src='wordnet')
ouput = aug.augment(text)

>>'hi Word!'

2.上下文词嵌入增强器Contextual Word Embeddings Augmenter

aug = naw.ContextualWordEmbsAug(
    model_path='bert-base-uncased', action="substitute")
ouput = aug.augment(text)
>>'that word!'

参考：https://towardsdatascience.com/nlp-extract-contextualized-word-embeddings-from-bert-keras-tf-67ef29f60a7b
文档：
https://nlpaug.readthedocs.io/en/latest/example/example.html
https://github.com/makcedward/nlpaug/blob/master/example/textual_augmenter.ipynb

下载谷歌预训练模型

curl -o - https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz | gunzip > ./GoogleNews-vectors-negative300.bin