Algorithm summary---the five most commonly used algorithms (algorithm question ideas)

1. Summary

One sentence summary:

[Clarify what you want: dijkstra is to find the distance from point to point, and the auxiliary array is the array from the source point to the target point]

[The simplest example analysis: for example, thinking about dijkstra: suppose there are only three points first]

1. What is the greedy algorithm?

Currently the best option

local optimum

It is possible to obtain the overall optimal solution or an approximate solution to the optimal solution

Greedy algorithm (also known as greedy algorithm) means that when solving a problem, always make the best choice at present. That is to say, without considering the overall optimality, what he makes is only a local optimal solution in a certain sense. The greedy algorithm can not obtain the overall optimal solution for all problems, but it can generate the overall optimal solution or an approximate solution to the overall optimal solution for many problems with a wide range of problems.

2. Greedy algorithm example?

Prim's algorithm for finding the minimum spanning tree: [select the edges with the smallest weight in turn from the edge set]

Kruskal's algorithm for finding the minimum spanning tree: [It is somewhat similar to finding the shortest path: but here is the distance between two sets]: [The shortest distance from the one-dimensional intermediate array record to the currently selected vertex]: [Two-dimensional table record The shortest distance from each point to each point]

Dijkstra's algorithm for calculating strongly connected subgraphs: [similar to the minimum spanning tree Kruskal] [two-dimensional table records the shortest distance from each point to each point] [clear what is required: dijkstra is to find the distance from point to point, and the auxiliary array is The array from the source point to the target point] [Select the smallest from the auxiliary array each time, and use the selected point to update the auxiliary array] [Analysis of the simplest example: For example, thinking about dijkstra: suppose there are only three points first]

Algorithm for constructing huffman tree: [select two points with small weight each time to synthesize a binary tree]

Brief introduction of Kruskal algorithm

In the weighted connected graph, the smallest edge is continuously found in the edge set, and if the edge satisfies the conditions for obtaining a minimum spanning tree, it is constructed until a minimum spanning tree is finally obtained.

Assuming that WN=(V,{E}) is a connected network containing n vertices, the process of constructing a minimum spanning tree according to the Kruskal algorithm is as follows: first construct a network with only n vertices and an empty edge set Subgraph, if each vertex in the subgraph is regarded as the root node of each tree, then it is a forest containing n trees. Afterwards, select an edge with the smallest weight from the edge set E of the network, if the two vertices of this edge belong to different trees, then add it to the subgraph, that is, add the two vertices to the Two trees are combined into one tree; on the contrary, if the two vertices of the edge have fallen on the same tree, it is not advisable, but the edge with the smallest weight should be taken and tried again. And so on, until there is only one tree in the forest, that is, the subgraph contains n-1 edges.

The core steps of Prim's algorithm are:

In the weighted connected graph, starting from a certain vertex v in the graph, at this time the set U={v}, repeat the following operations: In all the edges (u,w)∈E of u∈U,w∈VU Find an edge with the smallest weight, add the edge (u,w) to the set of found edges, and add point w to the set U. When U=V, the minimum spanning tree is found.

3. The idea of divide and conquer?

A problem of size N is decomposed into K smaller sub-problems

These sub-problems are independent of each other and have the same nature as the original problem

Problem-solving ideas: Decomposition->Solution->Merge

The basic idea of the divide-and-conquer algorithm is to decompose a problem of size N into K smaller sub-problems, which are independent of each other and have the same nature as the original problem. Finding the solution to the subproblem will give you the solution to the original problem.

4. Examples of divide and conquer?

Dichotomy: guess the number game, [you give the number and I will guess]

Quick sort: [Select the reference number (for example, the first one), the one smaller than me moves forward, and the one bigger than me moves back] [Auxiliary variable: sentinel i pointing to the first data, sentinel j pointing to the last data 】【Start "detection" from both ends of the initial sequence "6 1 2 7 9 3 4 5 10 8". First find a number less than 6 from right to left, then find a number greater than 6 from left to right, and then exchange them] [Exchange the value corresponding to the sentinel ij] [Meet the sentinel ij, exchange the position and reference number of the meeting] [ All the locations where sentinel i walked are smaller than the reference number, and all the locations where j walked are larger] [The set reference number is the leftmost number, so it is necessary to let sentry j be dispatched first: because j will first find a number smaller than the reference number , and then ij meets again, and then exchange the reference number and the number that ij meets]

Merge and sort: [Recursive implementation] [Split to a number] [Merge after sorting] [Sorting using complete binary tree characteristics] [Operating array subscripts to realize binary tree] [The process of merging is somewhat similar to the merging of linked lists]

5. What is the basic idea of dynamic programming?

Space for time: If we can save the answers to the sub-problems that have been solved, and find out the answers that have been obtained when needed, we can avoid a lot of repeated calculations and save time.

The difference with the divide and conquer method: Different from the divide and conquer method, it is suitable for problems solved by dynamic programming, [the subproblems obtained by decomposition are often not independent of each other]. If the divide-and-conquer method is used to solve this type of problem, [the number of sub-problems obtained by decomposition is too large], and some [sub-problems are repeatedly calculated many times].

How to write the state transition equation: [See how to decompose the problem into sub-problems: the decomposition process is the state transition equation] [Determining the state is what you want: dijkstra is to find the distance from point to point, and the auxiliary array is the distance from the source point to the target point Array] [The simplest example analysis: such as thinking about dijkstra: suppose there are only three points first]

6. Dynamic programming example?

Floyd's algorithm for finding the shortest path of the full path: [The shortest distance from vertex i to vertex j that only passes through the first k points]

Knapsack problem: [Which ones to take from the knapsack have the best value]

for(k=1;k<=n;k++)
for(i=1;i<=n;i++)
for(j=1;j<=n;j++)
if(e[i][j]>e[i][k]+e[k][j])
e[i][j]=e[i][k]+e[k][j];

7. What is the basic idea of backtracking method?

depth-first algorithm

When searching for a certain step, if it fails, it will return

Pruning: backtracking optimization

The backtracking method (exploration and backtracking method) is a kind of optimal search method, which searches forward according to the optimal conditions to achieve the goal. But when a certain step is explored, if the original choice is not optimal or the goal cannot be reached, then go back one step and make another choice. Called the "backtrack point".

8. Classic example of backtracking method?

Eight Queens:

9. What is the basic idea of the branch and bound method?

Breadth-first search: the node goes out of the queue, and all the children of the node enter the queue

Two common types: queuing and priority queuing

The branch and bound method usually searches the solution space tree of the problem in a breadth-first or minimum-cost (maximum-benefit)-first manner.
In the branch and bound method, each live node has only one chance to become an expanded node. Once a live node becomes an extended node, all its child nodes will be generated at one time. Among these child nodes, the child nodes that lead to infeasible solutions or non-optimal solutions are discarded, and the remaining child nodes are added to the active node table.
Thereafter, the next node is taken from the active node list to become the current expanded node, and the above node expansion process is repeated. This process continues until the desired solution is found or the list of slipnodes is empty.

There are two common branch and bound methods:

(1) Queue (FIFO) branch and bound method
Select the next node as the expansion node according to the queue first-in-first-out (FIFO) principle.
(2) Priority queue branch-and-bound method
According to the priority specified in the priority queue, the node with the highest priority is selected as the current expansion node.

10. What is the difference between the branch and bound method and the backtracking method?

Solution target: backtracking - all solutions, branch and bound method - one solution

Search method: backtracking-deep search, branch and bound method-wide search

(1) Solution goal: the solution goal of the backtracking method is to find all the solutions that satisfy the constraints in the solution space tree, while the solution goal of the branch and bound method is to find a solution that satisfies the constraints, or a solution that satisfies the constraints solution to find the optimal solution in a certain sense.
(2) Different search methods: the backtracking method searches the solution space tree in a depth-first manner, while the branch-and-bound method searches the solution space tree in a breadth-first or minimum-cost-first manner.

Two, the five most commonly used algorithms (transfer)

Reposted from: The five most commonly used algorithms
https://blog.csdn.net/watson2016/article/details/77857824

1. Greedy Algorithm

The characteristic of designing an algorithm with the greedy method is to proceed step by step. It is often based on the current situation to make the optimal choice according to an optimization measure, without considering various possible overall situations. All possible but necessary time-consuming, it uses a top-down, iterative method to make successive greedy choices, each time a greedy choice is made, the problem is simplified into a smaller sub-problem, through each step Greedy selection can obtain an optimal solution to the problem. Although each step must ensure that a local optimal solution can be obtained, the resulting global solution is sometimes not necessarily optimal, so the greedy method does not need backtracking.

Note: For a given problem, there may often be several metrics. At first glance, these metrics seem to be desirable, but in fact, the optimal solution in the sense of the metric obtained by using most of them for greedy processing is not the optimal solution of the problem, but a suboptimal solution . Therefore, choosing the optimal metric that yields the optimal solution to the problem is at the heart of using a greedy algorithm.

The classic Prim algorithm and Kruskal algorithm for finding the minimum spanning tree, the Dijkstra algorithm for computing strongly connected subgraphs, and the algorithm for constructing huffman trees are all beautiful greedy algorithms

The basic idea:

1. Build a mathematical model to describe the problem.
2. Divide the problem to be solved into several sub-problems.
⒊ Solve each sub-problem and obtain the local optimal solution of the sub-problem.
4. Combining the local optimal solutions of the subproblems into a solution of the original solution problem.
The process of realizing the algorithm:
start from an initial solution of the problem;
while can go further towards a given overall goal to find
a solution element of a feasible solution;
combine all solution elements to form a feasible solution of the problem.

example:

Greedy algorithm for horses stepping on a chessboard
[Problem description]
The traversal problem of horses. On an 8×8 square chessboard, start from any specified square, and find a shortest path for the horse to go through every square of the chessboard and only once.

[Greedy Algorithm]
In fact, the problem of horses stepping on a chessboard was raised a long time ago, and as early as 1823, JC Warnsdorff proposed a famous algorithm. When each node selects its child nodes, the smallest 'exit' is preferred for searching. 'Exit' means the number of feasible child nodes among these child nodes, that is, the 'grandchildren' 'The fewer nodes, the more priority to jump, why choose this way, this is an optimal method of local adjustment, if the sub-nodes with more exits are selected first, then there will be more and more sub-nodes with few exits, It is very likely that there will be a 'dead' node (as the name implies, it is a node with no exit and no skip), so that the following search is purely futile, which will waste a lot of useless time. Conversely, if you choose the one with less exit every time If the node jumps, there will be fewer and fewer nodes with fewer exits, so the chance of successful jumping will be greater.

2. Divide and conquer algorithm

Thought:

Application scenarios of divide and conquer:

The problems solved by using the divide and conquer strategy generally have the following characteristics:
1.
The original problem can be decomposed into multiple sub-problems. or similar.
2. During the decomposition process of the original problem, solve the sub-problem recursively
. Since the recursion must have a termination condition, when the scale of the decomposed sub-problem is small enough, it should be able to be solved directly.
3. After solving and obtaining the solutions of each sub-problem,
it should be possible to combine or construct the solution of the original problem in a certain way or method.
It is not difficult to find that in the divide-and-conquer strategy, due to the similarity in structure and solution between the sub-problem and the original problem, most of the problems solved by the divide-and-conquer method adopt the form of recursion. In various sorting methods, such as merge sort, heap sort, quick sort, etc., there is a divide and conquer idea.

The general steps to solve a problem by divide and conquer:

(1) Decomposition, which divides the problem to be solved into several smaller-scale similar problems;
(2) Solving, when the sub-problems are divided into small enough, use a simpler method to solve;
(3) Merge, according to the requirements of the original problem, The solutions to the subproblems are combined layer by layer to form the solution to the original problem.

3. Dynamic programming

Basic idea:

Dynamic programming algorithms are often used to solve problems with some optimal properties. In this type of problem, there may be many feasible solutions. Each solution corresponds to a value, and we want to find the solution with the optimal value. The dynamic programming algorithm is similar to the divide and conquer method, and its basic idea is to decompose the problem to be solved into several sub-problems, first solve the sub-problems, and then obtain the solution of the original problem from the solutions of these sub-problems. Different from the divide and conquer method, it is suitable for the problem solved by dynamic programming, and the sub-problems obtained through decomposition are often not independent of each other. If the divide-and-conquer method is used to solve such problems, the number of sub-problems obtained by decomposition is too large, and some sub-problems are repeatedly calculated many times. If we can save the answers to the sub-problems that have been solved, and find out the answers that have been obtained when we need it, we can avoid a lot of repeated calculations and save time. We can use a table to record the answers to all solved subproblems. Regardless of whether the sub-problem is used in the future, as long as it has been calculated, its result will be filled in the table. This is the basic idea of dynamic programming method. There are various specific dynamic programming algorithms, but they all have the same form of filling the table.

The biggest difference with the divide and conquer method is that it is suitable for problems solved by the dynamic programming method, and the sub-problems obtained after decomposition are often not independent of each other (that is, the solution of the next sub-stage is based on the solution of the previous sub-stage , for further solution)

Application scenario:

The problem applicable to dynamic programming must satisfy the principle of optimization, no aftereffect and overlap.
1. The principle of optimization (property of optimal substructure) The principle of optimization can be stated as follows: an optimization strategy has such a property that no matter what the past state and decision are, for the state formed by the previous decision, the remaining decisions must constitute an optimal policy. In short, subpolicies of an optimal policy are always optimal. A problem that satisfies the optimization principle is also known as having the optimal substructure property.

2. No aftereffect After the stages are arranged in a certain order, for a given stage state, the state of the previous stages cannot directly affect its future decision-making, but can only pass through the current state. In other words, each state is a complete summary of past history. This is no backwardness, also known as no aftereffect.

3. The overlapping dynamic programming of subproblems improves the original search algorithm with exponential time complexity into an algorithm with polynomial time complexity. The key is to solve the redundancy, which is the fundamental purpose of the dynamic programming algorithm. Dynamic programming is essentially a technique of exchanging space for time. During its implementation, it has to store various states in the generation process, so its space complexity is greater than other algorithms.

The Floyd algorithm for finding the shortest path of the full path is a beautiful use of dynamic programming ideas.

The following is a PPT screenshot of a dynamic programming idea I found about the 0-1 knapsack problem:

Problem Description:
Given n items and a backpack. The weight of item i is wi, its value is vi, and the capacity of the knapsack is C. How to choose the items in the knapsack so that the total value of the items in the knapsack is maximized?

For an item, it either goes into the backpack or it doesn't. Therefore, the loading state of an item can be 0 and 1. We set the loading state of item i as xi,xi∈(0,1), and this problem is called the 0-11 knapsack problem.

Data: number of items n=5, item weight w[n]={0, 2, 2, 6, 5, 4}, item value V[n]={0, 6, 3, 5, 4, 6} ,
(bit 0, set to 0, does not participate in the calculation, it is only convenient to unify with the subsequent subscripts, it has no special purpose, and it can not be handled this way.) The total weight c=10. The maximum capacity of the backpack is 10, so when setting the size of the array m, you can set the row and column values to 6 and 11, then, for m(i,j), it means that the optional items are i...n and the backpack capacity is j (total weight) The maximum value of the items in the backpack.

4. Backtracking

Basic idea:

The backtracking method searches the solution space tree from the root node according to the depth-first strategy in the solution space tree of the problem. When the algorithm searches to any point in the solution space tree, it first judges whether the node contains the solution of the problem. If it is definitely not included (pruning process), skip the search of the subtree rooted at this node, and backtrack to its ancestor node layer by layer; otherwise, enter the subtree and continue to search according to the depth-first strategy.

Backtracking is a depth-first search algorithm for implicit graphs

Backtracking method: In order to avoid generating problem states that cannot produce the best solution, the bounding function (bounding function) should be continuously used to kill (prune) those live nodes that are actually impossible to produce the desired solution, so as to reduce the problem. Calculations. Depth-first generative methods with bounded functions are called backtracking methods. (backtracking method = exhaustive + pruning)

General steps:

(1) For the given problem, define the solution space of the problem;
(2) Determine the structure of the solution space that is easy to search;
(3) Search the solution space in a depth-first manner, and use pruning functions to avoid invalid searches during the search process.

Two commonly used pruning functions:

(1) Constraint function: Subtract the number of subtrees that do not satisfy the constraint at the extended node
(2) Boundary function: Subtract the subtrees that cannot obtain the optimal solution

　　A remarkable feature of solving problems by backtracking is that the solution space of the problem is dynamically generated during the search process. At any moment, the algorithm only saves the path from the root node to the currently expanded node. If the length of the longest path from the root node to the leaf node in the solution space tree is h(n), the computational space required by the backtracking method is usually O(h(n)). Explicitly storing the entire solution space requires O(2^h(n)) or O(h(n)!) memory space.

Five, branch and bound method

Basic idea:

The difference between branch and bound method and backtracking method:

There are two common branch and bound methods:

Example: Single-source shortest path problem (refer to http://www.cnblogs.com/chinazhangjie/archive/2010/11/01/1866136.html)

1. Problem description
In the directed graph G given in the figure below, each edge has a non-negative edge weight. The shortest path from source vertex s to target vertex t in graph G is required.

The following figure is the solution space tree generated by solving the single-source shortest path problem of the directed graph G with the priority queue branch and bound method. Among them, the number next to each node indicates the current path length corresponding to the node.

Find a path:

The current shortest path is 8. Once the lower bound of a node is found to be not less than this shortest path, pruning:

Choose the shortest path to reach the same node:

2. Pruning strategy
In the process of expanding nodes in the algorithm, once the lower bound of a node is found to be not less than the currently found shortest path length, the algorithm will cut off the subtree rooted at the node.
In the algorithm, the control relationship between nodes is used for pruning. Starting from source vertex s, two different paths reach the same vertex of graph G. Since the path lengths of the two paths are different, the subtree rooted at the node in the tree corresponding to the long path can be cut off.
3. Algorithm idea
The priority queue branch-and-bound method to solve the single-source shortest path problem uses a minimal heap to store the live node table. Its priority is the current path length corresponding to the node.
The algorithm starts with a source vertex s in graph G and an empty priority queue. After node s is expanded, its son nodes are sequentially inserted into the heap. After that, the algorithm takes the node with the minimum current path length from the heap as the current expansion node, and checks all vertices adjacent to the current expansion node in turn. If there is an edge reachable from the current extended node i to vertex j, and the length of the corresponding path starting from the source, passing through vertex i and then to vertex j is less than the current optimal path length, then insert the vertex as a live node into The active node is in the priority queue. The expansion process of this node continues until the active node priority queue is empty.

Five Algorithmic Structures

Algorithm summary---the five most commonly used algorithms (algorithm question ideas)

1. Summary

One sentence summary:

1. What is the greedy algorithm?

2. Greedy algorithm example?

3. The idea of divide and conquer?

4. Examples of divide and conquer?

5. What is the basic idea of dynamic programming?

6. Dynamic programming example?

7. What is the basic idea of backtracking method?

8. Classic example of backtracking method?

9. What is the basic idea of the branch and bound method?

10. What is the difference between the branch and bound method and the backtracking method?

Two, the five most commonly used algorithms (transfer)

1. Greedy Algorithm

2. Divide and conquer algorithm

3. Dynamic programming

4. Backtracking

Five, branch and bound method

Guess you like

Five Algorithmic Structures

Algorithm summary---the five most commonly used algorithms (algorithm question ideas)

1. Summary

One sentence summary:

1. What is the greedy algorithm?

2. Greedy algorithm example?

3. The idea of ​​divide and conquer?

4. Examples of divide and conquer?

5. What is the basic idea of ​​dynamic programming?

6. Dynamic programming example?

7. What is the basic idea of ​​backtracking method?

8. Classic example of backtracking method?

9. What is the basic idea of ​​the branch and bound method?

10. What is the difference between the branch and bound method and the backtracking method?

Two, the five most commonly used algorithms (transfer)

1. Greedy Algorithm

2. Divide and conquer algorithm

3. Dynamic programming

4. Backtracking

Five, branch and bound method

Guess you like

3. The idea of divide and conquer?

5. What is the basic idea of dynamic programming?

7. What is the basic idea of backtracking method?

9. What is the basic idea of the branch and bound method?