Namesrv的功能,就相当于RPC中的注册中心。对于MQ而言,broker启动,将自身创建的topic等信息注册到Namesrv上。consumer和producer需要配置namesrv的地址,启动后,首先和namesrv建立长连接,并获取相应的topic信息(比如,哪些broker有topic路由信息),然后再和broker建立长连接。Namesrv本身无状态,可集群部署。所有的注册信息,都保存在namesrv的类似map内存数据结构中。
2. 启动
Namesrv启动后,首先会加载KVConfig服务,然后监听本地端口(默认9876),等待客户端连接,并定时清理非活跃broker和打印KVConfig值信息。代码清晰易懂。
public boolean initialize() { this.kvConfigManager.load(); this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService); this.remotingExecutor = Executors.newFixedThreadPool(nettyServerConfig.getServerWorkerThreads(), new ThreadFactoryImpl("RemotingExecutorThread_")); this.registerProcessor(); this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { NamesrvController.this.routeInfoManager.scanNotActiveBroker(); } }, 5, 10, TimeUnit.SECONDS); this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() { @Override public void run() { NamesrvController.this.kvConfigManager.printAllPeriodically(); } }, 1, 10, TimeUnit.MINUTES); return true; }
3. 请求处理
Namesrv能处理的所有请求都在DefaultRequestProcessor#processRequest方法:
public RemotingCommand processRequest(ChannelHandlerContext ctx, RemotingCommand request) throws RemotingCommandException { if (log.isDebugEnabled()) { log.debug("receive request, {} {} {}",// request.getCode(), // RemotingHelper.parseChannelRemoteAddr(ctx.channel()), // request); } switch (request.getCode()) { case RequestCode.PUT_KV_CONFIG: return this.putKVConfig(ctx, request); case RequestCode.GET_KV_CONFIG: return this.getKVConfig(ctx, request); case RequestCode.DELETE_KV_CONFIG: return this.deleteKVConfig(ctx, request); case RequestCode.REGISTER_BROKER: Version brokerVersion = MQVersion.value2Version(request.getVersion()); if (brokerVersion.ordinal() >= MQVersion.Version.V3_0_11.ordinal()) { return this.registerBrokerWithFilterServer(ctx, request); } else { return this.registerBroker(ctx, request); } case RequestCode.UNREGISTER_BROKER: return this.unregisterBroker(ctx, request); case RequestCode.GET_ROUTEINTO_BY_TOPIC: return this.getRouteInfoByTopic(ctx, request); case RequestCode.GET_BROKER_CLUSTER_INFO: return this.getBrokerClusterInfo(ctx, request); case RequestCode.WIPE_WRITE_PERM_OF_BROKER: return this.wipeWritePermOfBroker(ctx, request); case RequestCode.GET_ALL_TOPIC_LIST_FROM_NAMESERVER: return getAllTopicListFromNameserver(ctx, request); case RequestCode.DELETE_TOPIC_IN_NAMESRV: return deleteTopicInNamesrv(ctx, request); case RequestCode.GET_KVLIST_BY_NAMESPACE: return this.getKVListByNamespace(ctx, request); case RequestCode.GET_TOPICS_BY_CLUSTER: return this.getTopicsByCluster(ctx, request); case RequestCode.GET_SYSTEM_TOPIC_LIST_FROM_NS: return this.getSystemTopicListFromNs(ctx, request); case RequestCode.GET_UNIT_TOPIC_LIST: return this.getUnitTopicList(ctx, request); case RequestCode.GET_HAS_UNIT_SUB_TOPIC_LIST: return this.getHasUnitSubTopicList(ctx, request); case RequestCode.GET_HAS_UNIT_SUB_UNUNIT_TOPIC_LIST: return this.getHasUnitSubUnUnitTopicList(ctx, request); default: break; } return null; }
broker启动后,会调用REGISTER_BROKER注册topic等信息,其心跳也是直接调用REGISTER_BROKER的,broker下线会调用UNREGISTER_BROKER方法。
4. 数据结构
Namesrv的数据都保存在RouteInfoManager类中:
private final static long BrokerChannelExpiredTime = 1000 * 60 * 2; private final ReadWriteLock lock = new ReentrantReadWriteLock(); private final HashMap<String/* topic */, List<QueueData>> topicQueueTable; private final HashMap<String/* brokerName */, BrokerData> brokerAddrTable; private final HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable; private final HashMap<String/* brokerAddr */, BrokerLiveInfo> brokerLiveTable; private final HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;
可以看到,broker非活跃时间为2分钟。
5. KVConfig作用
猜测MQ作者,想通过KVConfig做类似configserver的简易功能,将信息配置在namesrv后,客户端获取后进行业务处理。其数据结构为:
private final HashMap<String/* Namespace */, HashMap<String/* Key */, String/* Value */>> configTable = new HashMap<String, HashMap<String, String>>();
KVConfig的数据最终会保存在kvConfig.json文件中,每次的put,delete操作都会直接修改kvConfig.json文件,并在namesrv启动时,加载数据。
6. 控制台
MQ的mqadmin脚本可以直接获取namesrv存储的信息,在MQ的bin目录下:
这里面,最重要的一个命令为wipeWritePerm,该命令可清除指定broker的写权限。当某台broker需要重启,将会导致向这台broker发消息失败,消息消费失败。通过该命令清除broker写权限后,由于producer只会向带有写权限的master broker发消息,当producer从namesrv更新(定时30s更新topic路由信息)到该broker无写权限后,将会将消息发往其他broker,然后再根据业务情况(譬如30分钟后),让consumer消费完消息再停掉该broker。