DS-lab3

Part A: Paxos

步骤

1. Add elements to the Paxos struct in paxos.go to hold the state you’ll need, according to the lecture pseudo-code. You’ll need to define a struct to hold information about each agreement instance.
2. Define RPC argument/reply type(s) for Paxos protocol messages, based on the lecture pseudo-code. The RPCs must include the sequence number for the agreement instance to which they refer. Remember the field names in the RPC structures must start with capital letters.
3. Write a proposer function that drives the Paxos protocol for an instance, and RPC handlers that implement acceptors. Start a proposer function in its own thread for each instance, as needed (e.g. in Start()).
4. At this point you should be able to pass the first few tests.
5. Now implement forgetting.

注意点

Hint: more than one Paxos instance may be executing at a given time, and they may be Start()ed and/or decided out of order (e.g. seq 10 may be decided before seq 5).

Hint: in order to pass tests assuming unreliable network, your paxos should call the local acceptor through a function call rather than RPC.

Hint: remember that multiple application peers may call Start() on the same instance, perhaps with different proposed values. An application may even call Start() for an instance that has already been decided.

Hint: think about how your paxos will forget (discard) information about old instances before you start writing code. Each Paxos peer will need to store instance information in some data structure that allows individual instance records to be deleted (so that the Go garbage collector can free / re-use the memory).

Hint: you do not need to write code to handle the situation where a Paxos peer needs to re-start after a crash. If one of your Paxos peers crashes, it will never be re-started.

Hint: have each Paxos peer start a thread per un-decided instance whose job is to eventually drive the instance to agreement, by acting as a proposer.

Hint: a single Paxos peer may be acting simultaneously as acceptor and proposer for the same instance. Keep these two activities as separate as possible.

Hint: a proposer needs a way to choose a higher proposal number than any seen so far. This is a reasonable exception to the rule that proposer and acceptor should be separate. It may also be useful for the propose RPC handler to return the highest known proposal number if it rejects an RPC, to help the caller pick a higher one next time. The px.me value will be different in each Paxos peer, so you can use px.me to help ensure that proposal numbers are unique.

Hint: figure out the minimum number of messages Paxos should use when reaching agreement in non-failure cases and make your implementation use that minimum.

Hint: the tester calls Kill() when it wants your Paxos to shut down; Kill() sets px.dead. You should call px.isdead() in any loops you have that might run for a while, and break out of the loop if px.isdead() is true. It’s particularly important to do this any in any long-running threads you create.

伪代码

proposer(v):
  while not decided:
    choose n, unique and higher than any n seen so far
    send prepare(n) to all servers including self
    if prepare_ok(n, n_a, v_a) from majority:
      v' = v_a with highest n_a; choose own v otherwise
      send accept(n, v') to all
      if accept_ok(n) from majority:
        send decided(v') to all

acceptor's state:
  n_p (highest prepare seen)
  n_a, v_a (highest accept seen)

acceptor's prepare(n) handler:
  if n > n_p
    n_p = n
    reply prepare_ok(n, n_a, v_a)
  else
    reply prepare_reject

acceptor's accept(n, v) handler:
  if n >= n_p
    n_p = n
    n_a = n
    v_a = v
    reply accept_ok(n)
  else
    reply accept_reject

实践注意点

1. 定义好通信的几个步骤， propose， accept， decide
2. 对于done函数的实现，也是需要定义rpc函数的
3. 注意go的mutex的锁，只会作用到子goroutine，对于孙子goroutine，已经管理不到了

Part B: Paxos-based Key/Value Server

步骤

1. Fill in the Op struct in server.go with the “value” information that kvpaxos will use Paxos to agree on, for each client request. Op field names must start with capital letters. You should use Op structs as the agreed-on values – for example, you should pass Op structs to Paxos Start(). Go’s RPC can marshall/unmarshall Op structs; the call to gob.Register() in StartServer() teaches it how.
2. Implement the PutAppend() handler in server.go. It should enter a Put or Append Op in the Paxos log (i.e., use Paxos to allocate a Paxos instance, whose value includes the key and value (so that other kvpaxoses know about the Put() or Append())). An Append Paxos log entry should contain the Append’s arguments, but not the resulting value, since the result might be large.
3. Implement a Get() handler. It should enter a Get Op in the Paxos log, and then “interpret” the the log before that point to make sure its key/value database reflects all recent Put()s.
4. Add code to cope with duplicate client requests, including situations where the client sends a request to one kvpaxos replica, times out waiting for a reply, and re-sends the request to a different replica. The client request should execute just once. Please make sure that your scheme for duplicate detection frees server memory quickly, for example by having the client tell the servers which RPCs it has heard a reply for. It’s OK to piggyback this information on the next client request.

注意点

Hint: your server should try to assign the next available Paxos instance (sequence number) to each incoming client RPC. However, some other kvpaxos replica may also be trying to use that instance for a different client’s operation. So the kvpaxos server has to be prepared to try different instances.

Hint: your kvpaxos servers should not directly communicate; they should only interact with each other through the Paxos log.

Hint: as in Lab 2, you will need to uniquely identify client operations to ensure that they execute just once. Also as in Lab 2, you can assume that each clerk has only one outstanding Put, Get, or Append.

Hint: a kvpaxos server should not complete a Get() RPC if it is not part of a majority (so that it does not serve stale data). This means that each Get() (as well as each Put() and Append()) must involve Paxos agreement.

Hint: don’t forget to call the Paxos Done() method when a kvpaxos has processed an instance and will no longer need it or any previous instance.

Hint: your code will need to wait for Paxos instances to complete agreement. The only way to do this is to periodically call Status(), sleeping between calls. How long to sleep? A good plan is to check quickly at first, and then more slowly:

to := 10 * time.Millisecond
for {
        status, _ := kv.px.Status(seq)
        if status == paxos.Decided{
          ...
          return 
        }
        time.Sleep(to)
        if to < 10 * time.Second {
          to *= 2
        }
  }

Hint: if one of your kvpaxos servers falls behind (i.e. did not participate in the agreement for some instance), it will later need to find out what (if anything) was agree to. A reasonable way to to this is to call Start(), which will either discover the previously agreed-to value, or cause agreement to happen. Think about what value would be reasonable to pass to Start() in this situation.

Hint: When the test fails, check for gob error (e.g. “rpc: writing response: gob: type not registered for interface …”) in the log because go doesn’t consider the error fatal, although it is fatal for the lab.

实践注意点

1. 架构
   
   1. 多轮paxos协议，每一个操作都是一个paxos协议。
   2. 对于多个server同时发起对一个sequence number的paxos协议，最后肯定只有一个server的提案通过，其他的server只能是通过增加sequence number让自己的提案在另外的提案中通过。
2. 去重
   
   1. 这个对于单个client重复向一个server发起请求很好做
   2. 对于client 向一个server发起请求，然后超时又向另外一个server发起请求，就不太好做。
   3. 对于用户的请求，都附带一个id。 然后server端，利用paxos的特性，可以在每次发起paxos协议得知当前sequence number的操作，将该操作的id记录下来，这样就可以知道以前干过什么，这样可以防止同一个操作被执行两次。如下图所示，就是client向多个server发起append操作，会导致操作重复被提交。

References:
1. 6.824 Lab 3: Paxos-based Key/Value Service