First, the idea of self-realization
-
Parse the configuration file
-
netty communication
-
serialize, deserialize
-
The service address is registered to the registry
Second, how to parse the spring configuration file
We generally put the service information in the spring configuration file for dubbo to parse and call. So how do these configuration files work?
2.1, first define the custom tags that spring can recognize
-
Spring provides
NamespaceHandlerSupportandBeanDefinitionParserthese two interfaces for us to extend and implement custom tags -
.xsdDeclare custom tags and attributes via file
2.2, dubbo to parse custom tags
Next, let's take a look at how dubbo parses its custom tags:
//DubboNamespaceHandler.java
public class DubboNamespaceHandler extends NamespaceHandlerSupport {
static {
Version.checkDuplicate(DubboNamespaceHandler.class);
}
@Override
public void init() {
registerBeanDefinitionParser("application", new DubboBeanDefinitionParser(ApplicationConfig.class, true));
registerBeanDefinitionParser("module", new DubboBeanDefinitionParser(ModuleConfig.class, true));
registerBeanDefinitionParser("registry", new DubboBeanDefinitionParser(RegistryConfig.class, true));
registerBeanDefinitionParser("monitor", new DubboBeanDefinitionParser(MonitorConfig.class, true));
registerBeanDefinitionParser("provider", new DubboBeanDefinitionParser(ProviderConfig.class, true));
registerBeanDefinitionParser("consumer", new DubboBeanDefinitionParser(ConsumerConfig.class, true));
registerBeanDefinitionParser("protocol", new DubboBeanDefinitionParser(ProtocolConfig.class, true));
registerBeanDefinitionParser("service", new DubboBeanDefinitionParser(ServiceBean.class, true));
registerBeanDefinitionParser("reference", new DubboBeanDefinitionParser(ReferenceBean.class, false));
registerBeanDefinitionParser("annotation", new AnnotationBeanDefinitionParser());
}
}
It can be seen that this class inherits from spring, NamespaceHandlerSupportand each tag is parsed through the parsing class of dubbo, and then the parsed data is put into its own configuration class.
Careful students should have discovered that other tags are Config, but service and reference are Bean, because after parsing the specific configuration, they need to call their corresponding methods to initialize
If the tag "service"will be DubboBeanDefinitionParserparsed, then the data will be filled into ServiceBeanit.
3. Release process
In ServiceBean, there are many implementation classes, including the ApplicationListener interface, which triggers service release according to the spring's listener. as follows:
//ServiceBean.java
//监听 spring上下文刷新或者加载的时候
@Override
public void onApplicationEvent(ContextRefreshedEvent event) {
if (isDelay() && !isExported() && !isUnexported()) {
if (logger.isInfoEnabled()) {
logger.info("The service ready on spring started. service: " + getInterface());
}
//服务发布
export();
}
}
Then follow up:
//ServiceBean.java
@Override
public void export() {
//调用父类方法
super.export();
// Publish ServiceBeanExportedEvent
//发布监听事件
publishExportEvent();
}
//ServiceConfig.java
public synchronized void export() {
if (provider != null) {
if (export == null) {
export = provider.getExport();
}
if (delay == null) {
delay = provider.getDelay();
}
}
//是否发布
if (export != null && !export) {
return;
}
//是否延迟
if (delay != null && delay > 0) {
delayExportExecutor.schedule(new Runnable() {
@Override
public void run() {
doExport();
}
}, delay, TimeUnit.MILLISECONDS);
} else {
doExport();
}
}
continue to see doExport()how
//ServiceConfig.java
protected synchronized void doExport() {
...
checkApplication();
checkRegistry();
checkProtocol();
appendProperties(this);
checkStubAndMock(interfaceClass);
if (path == null || path.length() == 0) {
path = interfaceName;
}
doExportUrls();
}
continue to see doExportUrls()how
//ServiceConfig.java
private void doExportUrls() {
//加载注册中心，并且生成URL地址
//例如：regitry://192.168.1.1:2080/org.apache.dubbo.registry.RegistryService/...
List<URL> registryURLs = loadRegistries(true);
for (ProtocolConfig protocolConfig : protocols) {
//服务发布
doExportUrlsFor1Protocol(protocolConfig, registryURLs);
}
}
then follow updoExportUrlsFor1Protocol()
private void doExportUrlsFor1Protocol(ProtocolConfig protocolConfig, List<URL> registryURLs) {
String name = protocolConfig.getName();
if (name == null || name.length() == 0) {
name = "dubbo";
}
//1、生成map，填充参数
Map<String, String> map = new HashMap<String, String>();
map.put(Constants.SIDE_KEY, Constants.PROVIDER_SIDE);
map.put(Constants.DUBBO_VERSION_KEY, Version.getProtocolVersion());
map.put(Constants.TIMESTAMP_KEY, String.valueOf(System.currentTimeMillis()));
if (ConfigUtils.getPid() > 0) {
map.put(Constants.PID_KEY, String.valueOf(ConfigUtils.getPid()));
}
appendParameters(map, application);
appendParameters(map, module);
appendParameters(map, provider, Constants.DEFAULT_KEY);
appendParameters(map, protocolConfig);
appendParameters(map, this);
...省略
if (Constants.LOCAL_PROTOCOL.equals(protocolConfig.getName())) {
protocolConfig.setRegister(false);
map.put("notify", "false");
}
String contextPath = protocolConfig.getContextpath();
if ((contextPath == null || contextPath.length() == 0) && provider != null) {
contextPath = provider.getContextpath();
}
//2、从protocol标签中获取服务的host
String host = this.findConfigedHosts(protocolConfig, registryURLs, map);
//3、从protocol标签中获取服务的端口
Integer port = this.findConfigedPorts(protocolConfig, name, map);
//4、把map转为URL
URL url = new URL(name, host, port, (contextPath == null || contextPath.length() == 0 ? "" : contextPath + "/") + path, map);
if (ExtensionLoader.getExtensionLoader(ConfiguratorFactory.class)
.hasExtension(url.getProtocol())) {
url = ExtensionLoader.getExtensionLoader(ConfiguratorFactory.class)
.getExtension(url.getProtocol()).getConfigurator(url).configure(url);
}
//具体服务发布
String scope = url.getParameter(Constants.SCOPE_KEY);
// don't export when none is configured
if (!Constants.SCOPE_NONE.toString().equalsIgnoreCase(scope)) {
// export to local if the config is not remote (export to remote only when config is remote)
if (!Constants.SCOPE_REMOTE.toString().equalsIgnoreCase(scope)) {
exportLocal(url);
}
// export to remote if the config is not local (export to local only when config is local)
if (!Constants.SCOPE_LOCAL.toString().equalsIgnoreCase(scope)) {
if (logger.isInfoEnabled()) {
logger.info("Export dubbo service " + interfaceClass.getName() + " to url " + url);
}
if (registryURLs != null && !registryURLs.isEmpty()) {
for (URL registryURL : registryURLs) {
url = url.addParameterIfAbsent(Constants.DYNAMIC_KEY, registryURL.getParameter(Constants.DYNAMIC_KEY));
URL monitorUrl = loadMonitor(registryURL);
if (monitorUrl != null) {
//将监控中心的 URL 作为 "monitor" 参数添加到服务提供者的 URL 中
url = url.addParameterAndEncoded(Constants.MONITOR_KEY, monitorUrl.toFullString());
}
if (logger.isInfoEnabled()) {
logger.info("Register dubbo service " + interfaceClass.getName() + " url " + url + " to registry " + registryURL);
}
// For providers, this is used to enable custom proxy to generate invoker
String proxy = url.getParameter(Constants.PROXY_KEY);
if (StringUtils.isNotEmpty(proxy)) {
registryURL = registryURL.addParameter(Constants.PROXY_KEY, proxy);
}
//invoker -> 代理类
//将服务提供者的 URL 作为 "export" 参数添加到注册中心的 URL 中
Invoker<?> invoker = proxyFactory.getInvoker(ref, (Class) interfaceClass, registryURL.addParameterAndEncoded(Constants.EXPORT_KEY, url.toFullString()));
DelegateProviderMetaDataInvoker wrapperInvoker = new DelegateProviderMetaDataInvoker(invoker, this);
Exporter<?> exporter = protocol.export(wrapperInvoker);
exporters.add(exporter);
}
} else {
Invoker<?> invoker = proxyFactory.getInvoker(ref, (Class) interfaceClass, url);
DelegateProviderMetaDataInvoker wrapperInvoker = new DelegateProviderMetaDataInvoker(invoker, this);
Exporter<?> exporter = protocol.export(wrapperInvoker);
exporters.add(exporter);
}
}
}
this.urls.add(url);
}
This code is very long, don't panic, let's look at the key content first, and roughly know the function of each piece of code. The above code can be roughly divided into the following steps:
- Generate map, fill parameters
- Get the host of the service from the protocol tag
- Get the port of the service from the protocol tag
- Generate the URL of the service
- Add the service provider's URL as an "export" parameter to the registry's URL
- call
Protocol#export(invoker)method
3.1. The first step is to fill all the parameters related to service publishing into the map, as shown below:
3.2, the second step, get the host and port, and finally convert the map to a URL, as shown below:
3.3. Step 3. Add the URL of the service provider as the "export" parameter to the URL of the registry.
3.4, Step 4, call the Protocol#export(invoker)method.
At this point, the benefits of the Dubbo SPI self-adaptive feature come out, and the corresponding extension implementation can be obtained automatically according to the URL parameters. For example, after the invoker is passed in, the corresponding Protocol extension is automatically obtained according to the invoker.url DubboProtocol.
So how exactly is it achieved?
First, through Protocol$Adapterthe export method in the class, as shown below:
The url we pass in here is registry://, so we will get an RegistryProtocolobject here.
Then, RegistryProtocolcall the export method, let's look at the code:
//RegistryProtocol.java
public <T> Exporter<T> export(final Invoker<T> originInvoker) throws RpcException {
//export invoker
//1、暴露服务，本质上就是启动一个neety服务
final ExporterChangeableWrapper<T> exporter = doLocalExport(originInvoker);
//获取注册中心url registry:// -> zookeeper://
URL registryUrl = getRegistryUrl(originInvoker);
//registry provider
//创建注册中心对象，与注册中心创建TCP连接。
final Registry registry = getRegistry(originInvoker);
// 获得服务提供者 URL dubbo://
final URL registeredProviderUrl = getRegisteredProviderUrl(originInvoker);
//to judge to delay publish whether or not
boolean register = registeredProviderUrl.getParameter("register", true);
// 向本地注册表，注册服务提供者
ProviderConsumerRegTable.registerProvider(originInvoker, registryUrl, registeredProviderUrl);
//2、 向注册中心注册服务提供者（自己）
if (register) {
register(registryUrl, registeredProviderUrl);
ProviderConsumerRegTable.getProviderWrapper(originInvoker).setReg(true);
}
// 订阅configurators节点，监听服务动态属性变更事件。
// Subscribe the override data
// FIXME When the provider subscribes, it will affect the scene : a certain JVM exposes the service and call the same service. Because the subscribed is cached key with the name of the service, it causes the subscription information to cover.
final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registeredProviderUrl);
final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);
overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);
registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);
//Ensure that a new exporter instance is returned every time export
return new DestroyableExporter<T>(exporter, originInvoker, overrideSubscribeUrl, registeredProviderUrl);
}
The code here also mainly looks at the approximate function, which can be divided into two parts:
- Expose the service, start netty
- Register the service provider with the registry
3.4.1. Let's first look at the first part, expose services, doLocalExportclick to see:
//RegistryProtocol.java
@SuppressWarnings("unchecked")
private <T> ExporterChangeableWrapper<T> doLocalExport(final Invoker<T> originInvoker) {
// 获得在 `bounds` 中的缓存 Key
String key = getCacheKey(originInvoker);
//从 `bounds` 获取exporter
//这里的`bounds`是一个map缓存，避免暴露过的服务重复暴露。
ExporterChangeableWrapper<T> exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
if (exporter == null) {
synchronized (bounds) {
exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
// 没有暴露过，则进行暴露服务
if (exporter == null) {
// 创建 Invoker Delegate 对象，继续对invoker进行包装
final Invoker<?> invokerDelegete = new InvokerDelegete<T>(originInvoker, getProviderUrl(originInvoker));
//Protocol$Adapter.export() -> DubbpProtocol.export()
exporter = new ExporterChangeableWrapper<T>((Exporter<T>) protocol.export(invokerDelegete), originInvoker);
bounds.put(key, exporter);
}
}
}
return exporter;
}
The logic here is very simple. First, determine whether it has been exposed. If not, it will be exposed. The protocol here is still Protocol$Adaptercalled. The Protocol$Adapter.export()code is as follows: The extension here is wrapped in many layers, and finally it is
ProtocolFilterWrapper中called DubboProtocol.export(), as shown in the following figure:
Why is this so? Because the originInvoker is the url of the service provider, we are here the dubbo protocol, so it will be adaptive
DubboProtocol.
Next look at the most important DubboProtocol.export()code:
//DubboProtocol.java
/**
* 服务器集合
*
* key: 服务器地址。格式为：host:port
*/
private final Map<String, ExchangeServer> serverMap = new ConcurrentHashMap<String, ExchangeServer>();
public <T> Exporter<T> export(Invoker<T> invoker) throws RpcException {
URL url = invoker.getUrl();
// export service.
//从url中组装服务器地址 例如 120.0.0.1:8080
String key = serviceKey(url);
DubboExporter<T> exporter = new DubboExporter<T>(invoker, key, exporterMap);
//存入集合
exporterMap.put(key, exporter);
//export an stub service for dispatching event
Boolean isStubSupportEvent = url.getParameter(Constants.STUB_EVENT_KEY, Constants.DEFAULT_STUB_EVENT);
Boolean isCallbackservice = url.getParameter(Constants.IS_CALLBACK_SERVICE, false);
if (isStubSupportEvent && !isCallbackservice) {
String stubServiceMethods = url.getParameter(Constants.STUB_EVENT_METHODS_KEY);
if (stubServiceMethods == null || stubServiceMethods.length() == 0) {
if (logger.isWarnEnabled()) {
logger.warn(new IllegalStateException("consumer [" + url.getParameter(Constants.INTERFACE_KEY) +
"], has set stubproxy support event ,but no stub methods founded."));
}
} else {
stubServiceMethodsMap.put(url.getServiceKey(), stubServiceMethods);
}
}
//开启一个服务，暴露20880端口
openServer(url);
//优化序列化
optimizeSerialization(url);
return exporter;
}
Next, look at the code to start the service openServer(url):
//DubboProtocol.java
private void openServer(URL url) {
// find server.
String key = url.getAddress();
//client can export a service which's only for server to invoke
boolean isServer = url.getParameter(Constants.IS_SERVER_KEY, true);
//判断是否是服务端
if (isServer) {
ExchangeServer server = serverMap.get(key);
if (server == null) {
//启动服务
serverMap.put(key, createServer(url));
} else {
// server supports reset, use together with override
server.reset(url);
}
}
}
Keep following createServer(url):
//DubboProtocol.java
private ExchangeServer createServer(URL url) {
// send readonly event when server closes, it's enabled by default
url = url.addParameterIfAbsent(Constants.CHANNEL_READONLYEVENT_SENT_KEY, Boolean.TRUE.toString());
// enable heartbeat by default
//默认开启心跳检测
url = url.addParameterIfAbsent(Constants.HEARTBEAT_KEY, String.valueOf(Constants.DEFAULT_HEARTBEAT));
//选择使用什么方式启动服务，netty3? netty4? mina? 默认是netty4
String str = url.getParameter(Constants.SERVER_KEY, Constants.DEFAULT_REMOTING_SERVER);
if (str != null && str.length() > 0 && !ExtensionLoader.getExtensionLoader(Transporter.class).hasExtension(str))
throw new RpcException("Unsupported server type: " + str + ", url: " + url);
// 设置编解码器为 `"Dubbo"`
url = url.addParameter(Constants.CODEC_KEY, DubboCodec.NAME);
ExchangeServer server;
try {
// 启动服务
server = Exchangers.bind(url, requestHandler);
} catch (RemotingException e) {
throw new RpcException("Fail to start server(url: " + url + ") " + e.getMessage(), e);
}
str = url.getParameter(Constants.CLIENT_KEY);
if (str != null && str.length() > 0) {
Set<String> supportedTypes = ExtensionLoader.getExtensionLoader(Transporter.class).getSupportedExtensions();
if (!supportedTypes.contains(str)) {
throw new RpcException("Unsupported client type: " + str);
}
}
return server;
}
Exchanger is the facade class of the data exchange layer, which abstracts two methods:
@SPI(HeaderExchanger.NAME)
public interface Exchanger {
/**
* bind.
* 绑定一个服务器
* @param url
* @param handler
* @return message server
*/
@Adaptive({Constants.EXCHANGER_KEY})
ExchangeServer bind(URL url, ExchangeHandler handler) throws RemotingException;
/**
* connect.
* 连接一个服务器
* @param url
* @param handler
* @return message channel
*/
@Adaptive({Constants.EXCHANGER_KEY})
ExchangeClient connect(URL url, ExchangeHandler handler) throws RemotingException;
server = Exchangers.bind(url, requestHandler);This method will eventually call the netty method.
//NettyServer.java
protected void doOpen() throws Throwable {
bootstrap = new ServerBootstrap();
bossGroup = new NioEventLoopGroup(1, new DefaultThreadFactory("NettyServerBoss", true));
workerGroup = new NioEventLoopGroup(getUrl().getPositiveParameter(Constants.IO_THREADS_KEY, Constants.DEFAULT_IO_THREADS),
new DefaultThreadFactory("NettyServerWorker", true));
final NettyServerHandler nettyServerHandler = new NettyServerHandler(getUrl(), this);
channels = nettyServerHandler.getChannels();
bootstrap.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childOption(ChannelOption.TCP_NODELAY, Boolean.TRUE)
.childOption(ChannelOption.SO_REUSEADDR, Boolean.TRUE)
.childOption(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT)
.childHandler(new ChannelInitializer<NioSocketChannel>() {
@Override
protected void initChannel(NioSocketChannel ch) throws Exception {
NettyCodecAdapter adapter = new NettyCodecAdapter(getCodec(), getUrl(), NettyServer.this);
ch.pipeline()//.addLast("logging",new LoggingHandler(LogLevel.INFO))//for debug
.addLast("decoder", adapter.getDecoder())
.addLast("encoder", adapter.getEncoder())
.addLast("handler", nettyServerHandler);
}
});
// 启动netty服务
ChannelFuture channelFuture = bootstrap.bind(getBindAddress());
channelFuture.syncUninterruptibly();
channel = channelFuture.channel();
As you can see, the netty method is finally called to build a Socket connection.
总结一下dubbo服务发布的主要流程：
- Parse the spring configuration file
- assembly url
- Build an Invoker
- RegistryProtocol.export()
- DubboProtpcol.export()
- Start a nettyServer
- Register the service provider with the registry
{{o.name}}
{{m.name}}