我们继续进入到org.apache.hadoop.hdfs.NameNodeProxies的函数
public static <T> ProxyAndInfo<T> createProxy(Configuration conf,URI nameNodeUri, Class<T> xface, AtomicBoolean fallbackToSimpleAuth)中，开始分析HA代理部分代码，代码如下：
// HA case
Conf config = new Conf(conf);
T proxy = (T) RetryProxy.create(xface, failoverProxyProvider,
RetryPolicies.failoverOnNetworkException(
RetryPolicies.TRY_ONCE_THEN_FAIL, config.maxFailoverAttempts,
config.maxRetryAttempts, config.failoverSleepBaseMillis,
config.failoverSleepMaxMillis));
Text dtService;
if (failoverProxyProvider.useLogicalURI()) {
dtService = HAUtil.buildTokenServiceForLogicalUri(nameNodeUri,
HdfsConstants.HDFS_URI_SCHEME);
} else {
dtService = SecurityUtil.buildTokenService(
NameNode.getAddress(nameNodeUri));
}
return new ProxyAndInfo<T>(proxy, dtService,
NameNode.getAddress(nameNodeUri));我们在继续讲解上面的代码前，我们先看看failoverProxyProvider的创建过程，这个变量为dfs.client.failover.proxy.provider加上nameNodeUri.host()类对象，代码如下：
/** Creates the Failover proxy provider instance*/
//这个函数用来创建一个故障转移代理类实例
//conf为配置类对象
//nameNodeUri对象中包含了NameNode服务器url信息
//xface为ClientProtocol.class
//boolean为false
@VisibleForTesting
public static <T> AbstractNNFailoverProxyProvider<T> createFailoverProxyProvider(
Configuration conf, URI nameNodeUri, Class<T> xface, boolean checkPort,
AtomicBoolean fallbackToSimpleAuth) throws IOException {
Class<FailoverProxyProvider<T>> failoverProxyProviderClass = null;
AbstractNNFailoverProxyProvider<T> providerNN;
//如果xface与NamenodeProtocols不是同一个类或者同一个接口，且xface不是NamenodeProtocols的
//父类或父接口，那么就抛出异常。
Preconditions.checkArgument(
xface.isAssignableFrom(NamenodeProtocols.class),
"Interface %s is not a NameNode protocol", xface);
try {
// Obtain the class of the proxy provider
//返回dfs.client.failover.proxy.provider加上nameNodeUri.host()对应类的Class类对象
failoverProxyProviderClass = getFailoverProxyProviderClass(conf,
nameNodeUri);
if (failoverProxyProviderClass == null) {
return null;
}
// Create a proxy provider instance.
//获取对应的构造函数,构造函数有三个参数
Constructor<FailoverProxyProvider<T>> ctor = failoverProxyProviderClass
.getConstructor(Configuration.class, URI.class, Class.class);
//根据构造函数创建类对象
FailoverProxyProvider<T> provider = ctor.newInstance(conf, nameNodeUri,
xface);
// If the proxy provider is of an old implementation, wrap it.
//provider是否是AbstractNNFailoverProxyProvider类的实例或者是
//AbstractNNFailoverProxyProvider子类实例,如果不是,那么采用
//WrappedFailoverProxyProvider类封装,否则直接返回
if (!(provider instanceof AbstractNNFailoverProxyProvider)) {
providerNN = new WrappedFailoverProxyProvider<T>(provider);
} else {
providerNN = (AbstractNNFailoverProxyProvider<T>)provider;
}
} catch (Exception e) {
String message = "Couldn't create proxy provider " + failoverProxyProviderClass;
if (LOG.isDebugEnabled()) {
LOG.debug(message, e);
}
if (e.getCause() instanceof IOException) {
throw (IOException) e.getCause();
} else {
throw new IOException(message, e);
}
}
// Check the port in the URI, if it is logical.
//检查端口号
if (checkPort && providerNN.useLogicalURI()) {
int port = nameNodeUri.getPort();
//如果端口号不为8020,那么就抛异常
if (port > 0 && port != NameNode.DEFAULT_PORT) {
// Throwing here without any cleanup is fine since we have not
// actually created the underlying proxies yet.
throw new IOException("Port " + port + " specified in URI "
+ nameNodeUri + " but host '" + nameNodeUri.getHost()
+ "' is a logical (HA) namenode"
+ " and does not use port information.");
}
}
providerNN.setFallbackToSimpleAuth(fallbackToSimpleAuth);
return providerNN;
}我们分析failoverProxyProviderClass = getFailoverProxyProviderClass(conf,nameNodeUri);getFailoverProxyProviderClass函数代码如下：
/** Gets the configured Failover proxy provider's class */
@VisibleForTesting
public static <T> Class<FailoverProxyProvider<T>> getFailoverProxyProviderClass(
Configuration conf, URI nameNodeUri) throws IOException {
if (nameNodeUri == null) {
return null;
}
//获取域名
String host = nameNodeUri.getHost();
//DFS_CLIENT_FAILOVER_PROXY_PROVIDER_KEY_PREFIX值为dfs.client.failover.proxy.provider
String configKey = DFS_CLIENT_FAILOVER_PROXY_PROVIDER_KEY_PREFIX + "."
+ host;
try {
@SuppressWarnings("unchecked")
//返回类名称为configKey对应的Class类对象,同时这个类要与FailoverProxyProvider是同一个类
//或者同一个接口，或者这个类是FailoverProxyProvider的子类或子接口
Class<FailoverProxyProvider<T>> ret = (Class<FailoverProxyProvider<T>>) conf
.getClass(configKey, null, FailoverProxyProvider.class);
return ret;
} catch (RuntimeException e) {
if (e.getCause() instanceof ClassNotFoundException) {
throw new IOException("Could not load failover proxy provider class "
+ conf.get(configKey) + " which is configured for authority "
+ nameNodeUri, e);
} else {
throw e;
}
}
}我们分析getClass函数，待如下：
/**
* Get the value of the <code>name</code> property as a <code>Class</code>
* implementing the interface specified by <code>xface</code>.
*
* If no such property is specified, then <code>defaultValue</code> is
* returned.
*
* An exception is thrown if the returned class does not implement the named
* interface.
*
* @param name the class name.
* @param defaultValue default value.
* @param xface the interface implemented by the named class.
* @return property value as a <code>Class</code>,
* or <code>defaultValue</code>.
*/
public <U> Class<? extends U> getClass(String name,
Class<? extends U> defaultValue,
Class<U> xface) {
try {
//获取类名称为name对应的Class类对象,如果没有那么采用默认值defaultValue
Class<?> theClass = getClass(name, defaultValue);
//如果Class类对象不为null且xface与theClass不是相同的类或接口，且xface不是theClass的父类
//或父接口,那么就抛出异常。
if (theClass != null && !xface.isAssignableFrom(theClass))
throw new RuntimeException(theClass+" not "+xface.getName());
else if (theClass != null)
return theClass.asSubclass(xface);
else
return null;
} catch (Exception e) {
throw new RuntimeException(e);
}
}我们分析getClass函数，代码如下：
/**
* Get the value of the <code>name</code> property as a <code>Class</code>.
* If no such property is specified, then <code>defaultValue</code> is
* returned.
*
* @param name the class name.
* @param defaultValue default value.
* @return property value as a <code>Class</code>,
* or <code>defaultValue</code>.
*/
public Class<?> getClass(String name, Class<?> defaultValue) {
//将字符串前后的空格符去掉
String valueString = getTrimmed(name);
//如果字符串为null,那么就返回defaultValue
if (valueString == null)
return defaultValue;
try {
//通过valueString获取对应的Class对象
return getClassByName(valueString);
} catch (ClassNotFoundException e) {
throw new RuntimeException(e);
}
}getClassByName函数最终会调用函数public Class<?> getClassByNameOrNull(String name)，代码如下：
在org.apache.hadoop.conf.Configuration类中
/**
* Load a class by name, returning null rather than throwing an exception
* if it couldn't be loaded. This is to avoid the overhead of creating
* an exception.
*
* @param name the class name
* @return the class object, or null if it could not be found.
*/
public Class<?> getClassByNameOrNull(String name) {
Map<String, WeakReference<Class<?>>> map;
//采用同步的方式
synchronized (CACHE_CLASSES) {
//CACHE_CLASSES的创建：private static final Map<ClassLoader, Map<String,
//WeakReference<Class<?>>>>
//CACHE_CLASSES = new WeakHashMap<ClassLoader, Map<String, WeakReference<Class<?
//>>>>();
//可以看出CACHE_CLASSES是一个map，key为加载类对象,value也为一个map，key为名称，value为
//WeakReference类对象，这个对象中包含了Class类对象
map = CACHE_CLASSES.get(classLoader);
if (map == null) {
//如果map为null,说明没有找到key为相应加载器类对象的元素,那么此时就创建一个
map = Collections.synchronizedMap(
new WeakHashMap<String, WeakReference<Class<?>>>());
CACHE_CLASSES.put(classLoader, map);
}
}
Class<?> clazz = null;
WeakReference<Class<?>> ref = map.get(name);
if (ref != null) {
//如果找到了对应name的WeekReference类对象，那么就获取相应的Class类对象
clazz = ref.get();
}
//如果为null
if (clazz == null) {
try {
//使用加载类对象classLoader加载名称为name的Class类对象，true表示加载的时候会执行类名称
//为name的类里面的静态区代码
clazz = Class.forName(name, true, classLoader);
} catch (ClassNotFoundException e) {
// Leave a marker that the class isn't found
//如果加载失败，那么就存储一个标识，表示该类不存在
map.put(name, new WeakReference<Class<?>>(NEGATIVE_CACHE_SENTINEL));
return null;
}
// two putters can race here, but they'll put the same class
//将Class类对象保存到map中，供下次再次使用,并返回Class类对象。
map.put(name, new WeakReference<Class<?>>(clazz));
return clazz;
} else if (clazz == NEGATIVE_CACHE_SENTINEL) {
return null; // not found
} else {
// cache hit
return clazz;
}
}到此，failoverProxyProvider就讲解完了，该类中包含了一个NameNode的代理类接下来我们继续往下分析。
我们进入到static <T> Object create(Class<T> iface,FailoverProxyProvider<T> proxyProvider, RetryPolicy retryPolicy)函数中，代码如下：
/**
* Create a proxy for an interface of implementations of that interface using
* the given {@link FailoverProxyProvider} and the same retry policy for each
* method in the interface.
*
* @param iface the interface that the retry will implement
* @param proxyProvider provides implementation instances whose methods should be retried
* @param retryPolicy the policy for retrying or failing over method call failures
* @return the retry proxy
*/
public static <T> Object create(Class<T> iface,
FailoverProxyProvider<T> proxyProvider, RetryPolicy retryPolicy) {
//直接调用Java动态代理构造方法，返回ClientProtocol的代理对象,RetryInvocationHandler实现了
//InvocationHandler接口,里面将proxyProvider和retryPolicy传入到对象中
return Proxy.newProxyInstance(
proxyProvider.getInterface().getClassLoader(),
new Class<?>[] { iface },
new RetryInvocationHandler<T>(proxyProvider, retryPolicy)
);
}上面代码中的proxyProvider其实是org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider类对象。回到org.apache.hadoop.hdfs.NameNodeProxies的函数createProxy中，将对象返回，进入到DFSClient类的DFSClient(URI nameNodeUri, ClientProtocol rpcNamenode,Configuration conf, FileSystem.Statistics stats)函数中，剩下的代码在此就忽略，到此整个DFSClient构造函数就结束了，当然里面还涉及到很多知识点，后面我会不断更新文章，把遗漏的知识点补上，敬请期待。