Flume (四) Sink Processors

Flume Sink Processors (Flume Sink处理器)

接收组（Sink groups）允许用户将多个接收器分组到一个实体中。接收器处理器（Sink processors）可用于在组内的所有接收器上提供负载均衡功能，或在临时故障（temporal failure）的情况下实现从一个接收器到另一个接收器的故障转移。

必需属性以粗体显示。

Property Name	Default	Description
sinks	–	以空格分隔的参与组的接收器列表
processor.type	default	组件类型名称需要是default，failover或load_balance

Example for agent named a1:

a1.sinkgroups = g1
a1.sinkgroups.g1.sinks = k1 k2
a1.sinkgroups.g1.processor.type = load_balance

Default Sink Processor

默认接收器只接受一个接收器。用户不必为单个接收器创建处理器（接收器组）。相反，用户可以遵循本用户指南中上面解释的源 - 通道 - 接收器模式。

Failover Sink Processor

故障转移接收器维护一个优先级的接收器列表，保证只要有一个可用的事件将被处理（传递）。

The failover mechanism works by relegating failed sinks to a pool where they are assigned a cool down period, increasing with sequential failures before they are retried. Once a sink successfully sends an event, it is restored to the live pool. The Sinks have a priority associated with them, larger the number, higher the priority. If a Sink fails while sending a Event the next Sink with highest priority shall be tried next for sending Events. For example, a sink with priority 100 is activated before the Sink with priority 80. If no priority is specified, thr priority is determined based on the order in which the Sinks are specified in configuration.
故障转移机制的工作原理是将故障接收器降级到池中，在池中为它们分配一个冷却期，在重试之前随顺序故障而增加。一旦接收器成功发送事件后，它将恢复到实时池。接收器优先级与之相关，数量越大，优先级越高。如果在发送事件时接收器发生故障，则应尝试下一个具有最高优先级的接收器以发送事件。例如，在优先级为80的接收器之前激活优先级为100的接收器。如果未指定优先级，则根据配置中指定接收器的顺序确定thr优先级。

要进行配置，请将接收器组处理器设置为故障转移failover并为所有单个接收器设置优先级。所有指定的优先级必须是唯一的此外，可以使用maxpenalty属性设置故障转移的时间上限（以毫秒为单位）。

必需属性以粗体显示。

Property Name	Default	Description
sinks	–	以空格分隔的参与组的接收器列表
processor.type	default	The component type name, needs to be `failover`
processor.priority.`<sinkName>`	-	优先值。 `<sinkName>`必须是与当前接收器组关联的接收器实例之一。较高优先级值Sink较早被激活。绝对值越大表示优先级越高
processor.maxpenalty	30000	组件类型名称需要是default，failover或load_balance

Example for agent named a1:

a1.sinkgroups = g1
a1.sinkgroups.g1.sinks = k1 k2
a1.sinkgroups.g1.processor.type = failover
a1.sinkgroups.g1.processor.priority.k1 = 5
a1.sinkgroups.g1.processor.priority.k2 = 10
a1.sinkgroups.g1.processor.maxpenalty = 10000

Load balancing Sink Processor

Load balancing sink processor provides the ability to load-balance flow over multiple sinks. It maintains an indexed list of active sinks on which the load must be distributed. Implementation supports distributing load using either via round_robin or random selection mechanisms. The choice of selection mechanism defaults to round_robin type, but can be overridden via configuration. Custom selection mechanisms are supported via custom classes that inherits from AbstractSinkSelector.
负载均衡接收处理器提供了在多个接收器上进行负载均衡流量的功能。它维护一个索引的活动接收器列表，必须在其上分配负载。实现支持使用round_robin或随机选择机制（random selection）分配负载。默认round_robin类型，但可以通过配置覆盖。通过从继承AbstractSinkSelector的实现自定义选择机制。

调用时，选择器使用其配置的选择机制选择下一个接收器并调用它。对于round_robin和random如果所选接收器无法传递事件，则处理器通过其配置的选择机制选择下一个可用接收器。此实现不会将失败的接收器列入黑名单，而是继续乐观地尝试每个可用的接收器。如果所有接收器调用都导致失败，则选择器将故障传播到接收器运行器。

If backoff is enabled, the sink processor will blacklist sinks that fail, removing them for selection for a given timeout. When the timeout ends, if the sink is still unresponsive timeout is increased exponentially to avoid potentially getting stuck in long waits on unresponsive sinks. With this disabled, in round-robin all the failed sinks load will be passed to the next sink in line and thus not evenly balanced
如果启用了backoff，则接收器处理器会将失败的接收器列入黑名单，将其删除以供给定超时的选择。当超时结束时，如果接收器仍然没有响应，则超时会呈指数级增加，以避免在无响应的接收器上长时间等待时卡住。在禁用此功能的情况下，在循环中，所有失败的接收器负载将被传递到下一个接收器中，因此不均衡

必需属性以粗体显示。

Property Name	Default	Description
sinks	–	以空格分隔的参与组的接收器列表
processor.type	default	组件类型名称需要为`load_balance`
processor.backoff	false	失败的接收器是否会以指数方式退回。
processor.selector	`round_robin`	选择机制。必须是round_robin，random或自定义类的FQCN，它继承自AbstractSinkSelector
processor.selector.maxTimeOut	30000	退避选择器用于限制指数退避（以毫秒为单位）

Example for agent named a1:

a1.sinkgroups = g1
a1.sinkgroups.g1.sinks = k1 k2
a1.sinkgroups.g1.processor.type = load_balance
a1.sinkgroups.g1.processor.backoff = true
a1.sinkgroups.g1.processor.selector = random

Custom Sink Processor

目前不支持自定义接收器处理器。