[leveldb] 5-Recover数据恢复(3)

数据库恢复到了这, 已经没有什么难点了. 直入主题, 如何从".log"文件恢复memtable/immemtable.

1. 获得当前目录下所有文件名, db/db_impl.cc 324-325,

  std::vector<std::string> filenames;
  s = env_->GetChildren(dbname_, &filenames);

2.由于VersionSet已经恢复了, 可知所有有效的SSTable, 检查一遍, 同时收集".log"文件, http://db_impl.cc 329-346

std::set<uint64_t> expected;
  versions_->AddLiveFiles(&expected);
  uint64_t number;
  FileType type;
  std::vector<uint64_t> logs;
  for (size_t i = 0; i < filenames.size(); i++) {
    if (ParseFileName(filenames[i], &number, &type)) {
      expected.erase(number);
      if (type == kLogFile && ((number >= min_log) || (number == prev_log)))
        logs.push_back(number);
    }
  }
  if (!expected.empty()) {
    char buf[50];
    snprintf(buf, sizeof(buf), "%d missing files; e.g.",
             static_cast<int>(expected.size()));
    return Status::Corruption(buf, TableFileName(dbname_, *(expected.begin())));
  }

3. 将收集到的log文件排序, 回放一遍, 得到memtable, http://db_impl.cc 348-355,

  // Recover in the order in which the logs were generated
  std::sort(logs.begin(), logs.end());
  for (size_t i = 0; i < logs.size(); i++) {
    s = RecoverLogFile(logs[i], (i == logs.size() - 1), save_manifest, edit,
                       &max_sequence);
    if (!s.ok()) {
      return s;
    }

跟入RecoverLogFile阅读源代码. 唯一值得注意的是回放过程中, 有可能memtable满了, 所以要安排compaction并修改VersionEdit, 如果没满, 则可继续复用logfile.

if (mem->ApproximateMemoryUsage() > options_.write_buffer_size) {
      compactions++;
      *save_manifest = true;
      status = WriteLevel0Table(mem, edit, nullptr);
      mem->Unref();
      mem = nullptr;
      if (!status.ok()) {
        // Reflect errors immediately so that conditions like full
        // file-systems cause the DB::Open() to fail.
        break;
      }
    }
  }

  delete file;

  // See if we should keep reusing the last log file.
  if (status.ok() && options_.reuse_logs && last_log && compactions == 0) {
    assert(logfile_ == nullptr);
    assert(log_ == nullptr);
    assert(mem_ == nullptr);
    uint64_t lfile_size;
    if (env_->GetFileSize(fname, &lfile_size).ok() &&
        env_->NewAppendableFile(fname, &logfile_).ok()) {
      Log(options_.info_log, "Reusing old log %s \n", fname.c_str());
      log_ = new log::Writer(logfile_, lfile_size);
      logfile_number_ = log_number;
      if (mem != nullptr) {
        mem_ = mem;
        mem = nullptr;
      } else {
        // mem can be nullptr if lognum exists but was empty.
        mem_ = new MemTable(internal_comparator_);
        mem_->Ref();

梳理下整个脉络,

Open开始 => 检查是否有CURRENT文件 => 没有的话, 新建数据库 => 回放MANIFEST, 归并VersionEdit到一个新Version => 收集.log文件, 回放日志 => 如果memtable满了要compact, 没满可以复用日志 => 重写MANIFEST => 清除无用文件 => Open结束.

一句话总结, 数据库Open的时候就是回放各种日志, 无它尔.

[leveldb] 5-Recover数据恢复(3)

猜你喜欢