1.Variational Reasoning for QuestionAnswering with Knowledge Grap
Summary:
The problem with QA: the problem is noisy; it takes multiple rounds of reasoning to find the answer.
Proposed an end-to-end deep learning model, which can effectively solve the above two problems, and achieved the best results on a series of data sets
Disadvantages of the kernel method: integrate the entire knowledge base into a large table, which lacks logical information and cannot be used for multiple rounds of reasoning. Suitable for single-round reasoning.
For example, who wrote this book with Tony? Tony author book author
The traditional remedy is to use relationship completion to create a new relationship for two non-adjacent entities. But this will be extremely complicated, not in line with the natural way of thinking, and will encounter many problems in storage and computing.
The second challenge is: how to locate the subject entity. Many are simple string matching, which is often inaccurate. The cause of this problem is also the existence of noise (expression, input errors, etc.), as well as the problem of aliases, such as Trump and Trump. There are many technologies to improve the accuracy of entity recognition. Especially the errors in speech recognition, the superposition of names and place names, cause the performance of the entire QA system to decline.
Most of the training data simply provides questions and answers, and does not have a detailed description of the relationship or logical reasoning process. It is necessary to find entities and relationships at the same time, as well as unknowable reasoning steps.
We propose the VRN model to improve these problems
1. Probability-based end-to-end model, which can deal with the problems of inaccurate entity recognition and multiple rounds of reasoning at the same time
2. We propose a deep learning model similar to the propagation algorithm, which performs logical reasoning on the probability model
3. We will strengthen the algorithm, reduce variables, and conduct system end-to-end training
4. We have achieved very good results in MetaQA
We are based on the reality and do not come to the fore. We can deal with problems from two sources (text and spoken language), as well as problems that cannot be solved by the current model.
Related work
Sentiment analysis
Matching problems use certain representation methods or logical inferences. These inferences require grammar, rules and relatively perfect annotations in the professional field. Of course he can't handle the noise.
Nuclear method
Attention mechanism, the effect is better in a single round of reasoning, and cannot be applied to multiple rounds
Representation learning (vector filling)
Sentences or fine-grained problems, but not suitable for large-scale knowledge graphs (this does not seem to indicate the shortcomings of learning...)
Multiple rounds of reasoning
In order to complete multiple rounds of reasoning, some traverse through path sampling and dynamic processing
Structure overview
Contains two probability models
Entity recognition model
In the actual environment, conventional string matching and entity annotation cannot be used, and a logical reasoning recognizer needs to be trained. Using the context of the text, for example, how about the clothes of the brand Jordan, here Jordan refers to the brand of clothes, not the star Jordan.
Logical reasoning model
Mem2Seq:Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented DialogSystems
The main challenge of an end-to-end task-based dialogue system is the entire knowledge
Mem2Seq is the first generative network model that combines multi-round attention mechanism and pointer network. Our advantage is faster and more accurate.
Recently, the end-to-end learning method has achieved good results. You can directly match the original QA pair, match and extract the answer, without manual labeling. Later, the attention-based replication mechanism can even handle tokens that do not appear in the dialogue history, and always produce correct and relevant entities.
But these methods have two problems. They strive to integrate external KB knowledge into the hidden layer of the RNN network, but RNN is not stable for long sequences. Processing long sequences is a very time-consuming thing, especially when combined with attention.
end-to-end memory networks (MemNNs). At this time, the end-to-end advantage is reflected, and the circular attention mechanism can be applied to large-scale data. Can store large-scale KB information and code length sequence. The multi-hop mechanism of MemNN has achieved very good performance in reasoning tasks. But MenNN simply selects answers from pre-set candidates, rather than generating answers word by word. At the same time, memory queries require clear practice rather than learning.
In response to these problems, we proposed the Mem2Seq model, which can obtain knowledge directly from the dialogue history or KB by using the multi-hop attention mechanism.
Creativity, difference
Mem2Seq first proposed a model that combines multiple rounds of attention and pointer networks, so that KB information can be integrated efficiently. Mem2Seq will learn how to use dynamic queries to obtain information. Finally, we are fast.
Model description
Mem2Seq consists of two parts
1.MemNN
2. Of course it's a pointer
TorusE: Knowledge Graph Embedding on a LieGroup
TransE is concise and efficient, but there are many problems, such as
Regularization is very important, because filling will produce deviations because of wrong examples.
Torus is an integral part of lie group and can be used to avoid regularity problems
Although the knowledge graph has hundreds of millions of nodes or facts, it is still incomplete and many facts will be missing. This is used to complete the knowledge graph.
TransE performed poorly on HITS@1, but it was very competitive on HITS@10 due to the bilinear model. TransE must require the filled entities to be within a certain range, which affects the accuracy of relationship prediction.
In Torus, we don't require any rules, we just adopted part of TransE's guidelines.
Related work Basic translation model
Bilinear model
Neural network-based model
Here are some things about Li Qun, which means I don’t understand.
Liegroups are real manifolds or complex manifolds with a group structure, and the addition and inverse operations in the group are analytic mappings in the form. Li group plays a very important role in mathematical analysis, physics and geometry. In a paper in the journal "Artificial Intelligence and Pattern Recognition".
A Torus
Knowledge GraphEmbedding with Iterative Guidance from Soft Rules
The representation learning of the knowledge graph in the low-dimensional vector space is the current research hotspot. Recently, the combination of distributed knowledge representation and traditional symbolic logic has attracted more and more attention. However, most previous attempts have adopted the method of injecting logical rules at one time, ignoring the interaction between distributed knowledge representation learning and logical reasoning. In addition, the previous methods only focused on dealing with hard rules, that is, those rules that always hold and cannot be violated. Such rules usually require a lot of manpower to write or verify.
The author of this paper proposes a new knowledge graph distributed representation learning method-rule-guided embedding (rule-guided, abbreviated as RUGE), which uses the iterative guidance of soft rules to complete the knowledge graph representation learning. The so-called soft rules are those rules that are not always valid and have confidence. Such rules can be automatically extracted from the knowledge graph through algorithms.
Specifically, RUGE simultaneously uses three resources of labeled triples, unlabeled triples, and automatically extracted soft rules to perform knowledge graph representation learning in an iterative manner. Each iteration alternates between the two steps of soft label prediction and representation correction. The former uses the currently learned representation and soft rules to predict soft labels for unlabeled triples; the latter further uses labeled triples (hard labels) and unlabeled triples (soft labels) to modify the current representation. Through this iterative process, RUGE can successfully model the interaction between distributed knowledge representation learning and logical reasoning, and the rich knowledge contained in logical rules can also be better transferred to the learned distributed representation.