Python列表的元素删除问题

python列表的元素删除问题

案例：查询Google词向量当中的停用词

• 描述：为了确定google词向量当中的停用词，本人收集了nltk的停用词表，同时自己添加了所有的英文符号到停用词表当中，通过遍历每个停用词表的元素，查询该词是否在停用词表当中；以下为实现的代码：

import gensim
# 导入模型
path = "Google_news_vec/GoogleNews-vectors-negative300.bin"
print("Loading the model...")
model = gensim.models.KeyedVectors.load_word2vec_format(path, binary=True)
print("Loading successfully.")

stopwords = ['i', 'me', 'my', 'myself', 'we', 'our', 'ours', 'ourselves', 'you', "you're", "you've", "you'll", "you'd", 'your', 'yours', 'yourself', 'yourselves', 'he', 'him', 'his', 'himself', 'she', "she's", 'her', 'hers', 'herself', 'it', "it's", 'its', 'itself', 'they', 'them', 'their', 'theirs', 'themselves', 'what', 'which', 'who', 'whom', 'this', 'that', "that'll", 'these', 'those', 'am', 'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had', 'having', 'do', 'does', 'did', 'doing', 'a', 'an', 'the', 'and', 'but', 'if', 'or', 'because', 'as', 'until', 'while', 'of', 'at', 'by', 'for', 'with', 'about', 'against', 'between', 'into', 'through', 'during', 'before', 'after', 'above', 'below', 'to', 'from', 'up', 'down', 'in', 'out', 'on', 'off', 'over', 'under', 'again', 'further', 'then', 'once', 'here', 'there', 'when', 'where', 'why', 'how', 'all', 'any', 'both', 'each', 'few', 'more', 'most', 'other', 'some', 'such', 'no', 'nor', 'not', 'only', 'own', 'same', 'so', 'than', 'too', 'very', 's', 't', 'can', 'will', 'just', 'don', "don't", 'should', "should've", 'now', 'd', 'll', 'm', 'o', 're', 've', 'y', 'ain', 'aren', "aren't", 'couldn', "couldn't", 'didn', "didn't", 'doesn', "doesn't", 'hadn', "hadn't", 'hasn', "hasn't", 'haven', "haven't", 'isn', "isn't", 'ma', 'mightn', "mightn't", 'mustn', "mustn't", 'needn', "needn't", 'shan', "shan't", 'shouldn', "shouldn't", 'wasn', "wasn't", 'weren', "weren't", 'won', "won't", 'wouldn', "wouldn't"]
# 查询单词是否在模型的词表当中，在则删除
for item in stopwords:
    if item in model.vocab:
        stopwords.remove(item)
print(stopwords)

• 但是上述查询的输出却是这样的（间隔输出）：
  [‘me’, ‘myself’, ‘our’, ‘ourselves’, “you’re”, “you’ll”, ‘your’, ‘yourself’, …等等
• 重新查找单词“me”的词向量，发现是存在的；

print(model.get_vector('me'))
# 输出：[ 1.38671875e-01 -9.17968750e-02  3.49121094e-02  1.50390625e-01
 -2.57873535e-03  8.30078125e-02 -6.22558594e-02 -1.06445312e-01
 -1.06933594e-01  1.70898438e-01 -1.50390625e-01 -2.39257812e-01
  2.19726562e-02 -2.98828125e-01 -3.39843750e-01  2.34375000e-01 ...

???

• 为什么？这里就是python列表方法remove()的坑， 在对列表中的元素进行删除时，被删除元素的所在位置的空间会被释放，其后的元素会被填充到当前位置，导致被删除元素之后的元素的索引发生改变，在for循环遍历的过程中跳过该元素。所以
  for循环遍历不要删除元素！！！
  for循环遍历不要删除元素！！！
  for循环遍历不要删除元素！！！
• 同理也不要插入元素！！！

附：Google预训练词向量地址(需外网)或者github地址

• 也可以通过以下方式下载

import gensim.downloader as api

api.load("word2vec-google-news-300") # download and load the model
api.info() # 查看gensim自带的语料和模型