实现一个简单的电影评论语料库的全文检索系统
#!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Wed Apr 4 15:28:11 2018 @author: dag """ #coding:utf-8 import nltk import re def raw(file): contents = open(file).read() contents = re.sub(r'<.*?>', ' ', contents) contents = re.sub('\s+', ' ', contents) return contents def snippet(doc, term): # buggy text = ' '*30 + raw(doc) + ' '*30 pos = text.index(term) return text[pos-30:pos+30] print ("Building Index...") files = nltk.corpus.movie_reviews.abspaths() idx = nltk.Index((w, f) for f in files for w in raw(f).split()) query = '' while query != "quit": query = input("query> ") if query in idx: for doc in idx[query]: print (snippet(doc, query)) else: print ("Not found")
output:
runfile('/Users/dag/.spyder-py3/filesearch.py', wdir='/Users/dag/.spyder-py3') Building Index... query> natural drug usage as being a hip and natural part of the art scene twister , this is yet another natural disaster movie that do , such as bull durham and the natural . no curve balls here great abandon . rejecting her natural abilities , she has sp due to the exhausting of our natural resources . each membe rn kentucky . it's beauty and natural goodness is being slow ilming in order to get a more natural expression of fear out