[Translation] akka in action of akka-stream (4 mediate between producers and consumers)

Mediate between producers and consumers

The next example involves broadcasting the event to the consumer services. So far, we have written to disk log events - log file containing all events, as well as warnings, errors and critical errors of several files.

 

In the final version of the log stream processors, the event will be sent to the archive services, notification services and measurement services.

 

Log stream processors must balance supply and demand, to ensure that when one service application back pressure, it does not slow down the producer of the event log. In the next section, we will discuss how to use a buffer to achieve this.

 

Integration services
in the examples in this section, we hope that all services are provided Sink. Akka-stream There are several options not available Sources or Sinks integration of external services used. For example, mapAsync method with a Future, and will issue further results Future downstream, when you already have a client using Future service code, which could be very useful.
Implementation and other Reactive Streams can be used Source.fromPublisher and Sink.fromSubscriber integrate convert any Reactive Streams Publisher as Source and any Subscriber converted to Sink. You can also use ActorPublisher and ActorSubscriber qualities integrated Actor, in certain specific cases, be useful.
The best and easiest option is to use the Source and Sink based akka-stream library.

 

4.1 buffer

Let's look at the process of FIG event, and now it is adapted to send data to the three services Sink.

figure 1

figure 2

Broadcast添加到每个过滤流上。一个输出像先前写入到日志文件Sink,另一个用于向下游服务发送数据。(这里的期望是日志文件Sinks 非常快,所以它们没有缓冲。)MergePreferred 阶段将所有通知摘要合并到通知服务的Sink, 关键事件摘要比错误和警告摘要优先级高。关键事件摘要总是包含一个关键事件。

 

OK事件也使用广播进行拆分,并将它们发送到度量服务。

 

该图还展示了插入缓冲区的位置。缓冲区允许下游消费者在消费数据的速度上有所不同。但是当缓冲区满了,就必须做出决定。

 

Flow上的buffer方法需要两个参数:一个是缓冲大小和一个OverflowStrategy,这个策略决定了缓冲区即将溢出时会发生什么。OverflowStrategy 可以设置为dropHead, dropTail, dropBuffer, drop-New, backpressure, or fail之一,分别从缓冲中放弃第一个元素,缓冲中最后一个元素,整个缓冲,或者最新的元素了,或者当缓冲满的时候应用背压;或者让整个流失败。选择哪一个有赖于应用的需求和具体的案例什么最重要。

 

在这个例子中,已经做出决定,即使在高负载下,所有事件都必须存档,这意味着当不能将事件写入归档服务Sink时,日志流处理器流应该失败。生产者可以稍后再试。设置一个大的缓冲区,如果归档Sink在一段时间内缓慢响应,则进行缓冲。

 

下面的代码展示了如何在图中设置缓冲。

缓冲

 

如果通知服务Sink接收慢,则在高负载下可以丢弃最旧的警告。不能删除错误摘要。关键错误没有缓冲,所以默认情况下流会使用背压。

 

val bcast = builder.add(Broadcast[Event](5))
val wbcast = builder.add(Broadcast[Event](2))
val ebcast = builder.add(Broadcast[Event](2))
val cbcast = builder.add(Broadcast[Event](2))
val okcast = builder.add(Broadcast[Event](2))

val mergeNotify = builder.add(MergePreferred[Summary](2))
val archive = builder.add(jsFlow)

 

MergePreferred 始终有一个首选端口和多个次要端口, 在本例中为2个。

 

首先, 下面的代码展示了所有图形节点是如何连接的。

连接图形节点

 

在下一节中,我们将研究如何在流中以不同的速率处理元素。将使用一种特殊的流操作来分离流的一方与另一方的速率。

 

发布了6 篇原创文章 · 获赞 43 · 访问量 57万+

Guess you like

Origin blog.csdn.net/hany3000/article/details/96481263