目录
这篇博客先整体整理下CNStream流处理多路并发Pipeline框架，后面笔记会详细整理内部代码流程以及代码细节。
1 CNStream概述
1.1 CNStream简介
CNStream 是⾯向寒武纪开发平台的数据流处理 SDK，基于模块化和流⽔线的思想，提供了⼀套基于C++11 的类和接⼝来⽀持流处理多路并发的 Pipeline 框架。⽤⼾可以根据 CNStream 提供的接⼝，开发⾃定义模块，并通过模块之间相互连接，实现⾃⼰的业务处理流程。CNStream 能够⼤⼤简化寒武纪⼈⼯智能平台提供的推理和其他处理，如视频解码、图像前处理的集成。也能够在兼顾灵活性的同时，充分发挥寒武纪硬件解码和⼈⼯智能算法的运算性能。
1.2 CNStream 数据流动
CNStream 模块间流动的数据实体为 std::shared_ptr<CNFrameInfo>，随着计算流的推进不断丰富结构体中的内容，CNFrameInfo 包含了⼀张图像所有的结构化信息。
流⽔线模块之间是⼀种⽣产者消费者的关系，通过模块的输⼊队列串联。上游模块⽣产数据后塞⼊到下游模块的输⼊队列，下游模块则从⾃⾝的输⼊队列中获取数据进⾏消费，如此迭代往后流动。当某个模块的输⼊数据队列被塞满，则会对上游模块造成阻塞，阻⽌内存占⽤⽆限上涨的同时阻⽌上游模块对计算资源的⽆意义占⽤。
2 框架介绍
2.1 核心框架
CNStream SDK 基于管道（Pipeline）和事件总线（EventBus）实现了模块式数据处理流程。
Pipeline 类似⼀个流⽔线，把复杂问题的解决⽅案分解成⼀个个处理阶段，然后依次处理。⼀个处理阶段的结果是下⼀个处理阶段的输⼊。Pipeline 模式的类模型由三部分组成：
- Pipeline：代表执⾏流。
- Module：代表执⾏流中的⼀个阶段。
- Context：是 Module 执⾏时的上下⽂信息。
EventBus 模式主要⽤来处理事件，包括三个部分：
- 事件源（Event Source）：将消息发布到事件总线上。
- 事件监听器（Observer/Listener）：监听器订阅事件。
- 事件总线（EventBus）：事件发布到总线上时被监听器接收。
Pipeline 和 EventBus 模式实现了 CNStream 框架。相关组成以及在 CNStream SDK 实现中对应关系如下：
- Pipeline：对应 cnstream::Pipeline 类。
- Module：Pipeline 的每个处理阶段是⼀个组件，对应 cnstream::Module 类。每⼀个具体的module都是 cnstream::Module 的派⽣类。
- FrameInfo：Pipeline 模式的 Context，对应 cnstream::CNFrameInfo 类。
- Event-bus 和 Event：分别对应 cnstream::EventBus 类和 cnstream::Event 类。
2.2 cnstream::Pipeline 类
cnstream::Pipeline 类实现了 pipeline 的搭建、module 管理、以及 module 的调度执⾏。在 module⾃⾝不传递数据时，负责 module 之间的数据传递。此外，该类集成事件总线，提供注册事件监听器的机制，使⽤⼾能够接收事件。例如 stream EOS（End of Stream）等。Pipeline 通过隐含的深度可控的队列来连接 module，使⽤ module 的输⼊队列连接上游的 module。CNStream 也提供了根据 JSON 配置⽂件来搭建 pipeline 的接⼝。在不重新编译源码的情况下，通过修改配置⽂件搭建不同的 pipeline。当⽤⼾程序想要获取 Pipeline 处理结果作进⼀步的处理时，可以从 cnstream::IModuleObserver 接⼝类继承实现⼀个观察者，并通过 RegisterFrameDoneCallBack 注册到 Pipeline 中，处理完的帧数据会通过 Notify 回调函数传出。
cnstream::Pipeline 类在 cnstream_pipeline.hpp ⽂件内定义，cnstream_pipeline.hpp ⽂件存放于framework/core/include ⽬录下.
/**
* @class Pipeline
*
* @brief Pipeline is the manager of the modules, which manages data transmission between modules and controls messages
* delivery.
*/
class Pipeline : private NonCopyable {
public:
/**
* @brief A constructor to construct one pipeline.
*
* @param[in] name The name of the pipeline.
*
* @return No return value.
*/
explicit Pipeline(const std::string& name);
/**
* @brief A destructor to destruct one pipeline.
*
* @param[in] name The name of the pipeline.
*
* @return No return value.
*/
virtual ~Pipeline();
/**
* @brief Gets the pipeline's name.
*
* @return Returns the pipeline's name.
*/
const std::string& GetName() const;
/**
* @brief Builds a pipeline by module configurations.
*
* @param[in] module_configs The configurations of a module.
* @param[in] profiler_config The configuration of a profiler.
*
* @return Returns true if this function has run successfully. Otherwise, returns false.
*/
bool BuildPipeline(const std::vector<CNModuleConfig>& module_configs,
const ProfilerConfig& profiler_config = ProfilerConfig());
/**
* @brief Builds a pipeline by graph configuration.
*
* @param[in] graph_config The configuration of a graph.
*
* @return Returns true if this function has run successfully. Otherwise, returns false.
*/
bool BuildPipeline(const CNGraphConfig& graph_config);
/**
* @brief Builds a pipeline from a JSON file.
* You can learn to write a configuration file by looking at the description of CNGraphConfig.
*
* @see CNGraphConfig
*
* @param[in] config_file The configuration file in JSON format.
*
* @return Returns true if this function has run successfully. Otherwise, returns false.
*
*/
bool BuildPipelineByJSONFile(const std::string& config_file);
/**
* @brief Starts a pipeline.
* Starts data transmission in a pipeline.
* Calls the ``Open`` function for all modules. See Module::Open.
*
* @return Returns true if this function has run successfully. Returns false if the ``Open``
* function did not run successfully in one of the modules, or the link modules failed.
*/
bool Start();
/**
* @brief Stops data transmissions in a pipeline.
*
* @return Returns true if this function has run successfully. Otherwise, returns false.
*/
bool Stop();
/**
* @brief The running status of a pipeline.
*
* @return Returns true if the pipeline is running. Returns false if the pipeline is not running.
*/
bool IsRunning() const;
/**
* @brief Gets a module in current pipeline by name.
*
* @param[in] module_name The module name specified in the module configuration.
* If you specify a module name written in the module configuration, the first module with the same name as
* the specified module name in the order of DFS will be returned.
* When there are modules with the same name as other graphs in the subgraph, you can also find the
* module by adding the graph name prefix divided by slash. eg. pipeline_name/subgraph1/module1.
*
* @return Returns the module pointer if the module has been added to
* the current pipeline. Otherwise, returns nullptr.
*/
Module* GetModule(const std::string& module_name) const;
/**
* @brief Gets the module configuration by the module name.
*
* @param[in] module_name The module name specified in module configuration.
* The module name can be specified by two ways, see Pipeline::GetModule for detail.
*
* @return Returns module configuration if this function has run successfully.
* Returns NULL if the module specified by ``module_name`` has not been
* added to the current pipeline.
*/
CNModuleConfig GetModuleConfig(const std::string& module_name) const;
/**
* @brief Checks if profiling is enabled.
*
* @return Returns true if profiling is enabled.
**/
bool IsProfilingEnabled() const;
/**
* @brief Checks if tracing is enabled.
*
* @return Returns true if tracing is enabled.
**/
bool IsTracingEnabled() const;
/**
* @brief Provides data for the pipeline that is used in source module or the module transmitted by itself.
*
* @param[in] module The module that provides data.
* @param[in] data The data that is transmitted to the pipeline.
*
* @return Returns true if this function has run successfully. Returns false if the module
* is not added in the pipeline or the pipeline has been stopped.
*
* @note ProvideData can be only called by the head modules in pipeline. A head module means the module
* has no parent modules.
*
* @see Module::Process.
*/
bool ProvideData(const Module* module, std::shared_ptr<CNFrameInfo> data);
/**
* @brief Gets the event bus in the pipeline.
*
* @return Returns the event bus.
*/
EventBus* GetEventBus() const;
/**
* @brief Binds the stream message observer with a pipeline to receive stream message from this pipeline.
*
* @param[in] observer The stream message observer.
*
* @return No return value.
*
* @see StreamMsgObserver.
*/
void SetStreamMsgObserver(StreamMsgObserver* observer);
/**
* @brief Gets the stream message observer that has been bound with this pipeline.
*
* @return Returns the stream message observer that has been bound with this pipeline.
*
* @see Pipeline::SetStreamMsgObserver.
*/
StreamMsgObserver* GetStreamMsgObserver() const;
/**
* @brief Gets this pipeline's profiler.
*
* @return Returns profiler.
*/
PipelineProfiler* GetProfiler() const;
/**
* @brief Gets this pipeline's tracer.
*
* @return Returns tracer.
*/
PipelineTracer* GetTracer() const;
/**
* @brief Checks if module is root node of pipeline or not.
* The module name can be specified by two ways, see Pipeline::GetModule for detail.
*
* @param[in] module_name module name.
*
* @return Returns true if it's root node, otherwise returns false.
**/
bool IsRootNode(const std::string& module_name) const;
/**
* @brief Checks if module is leaf node of pipeline.
* The module name can be specified by two ways, see Pipeline::GetModule for detail.
*
* @param[in] module_name module name.
*
* @return Returns true if it's leaf node, otherwise returns false.
**/
bool IsLeafNode(const std::string& module_name) const;
/**
* @brief Registers a callback to be called after the frame process is done.
* The callback will be invalid when Pipeline::Stop is called.
*
* @param[in] callback The call back function.
*
* @return No return value.
*
*/
void RegisterFrameDoneCallBack(const std::function<void(std::shared_ptr<CNFrameInfo>)>& callback);
private:
/** called by BuildPipeline **/
bool CreateModules(std::vector<std::shared_ptr<Module>>* modules);
void GenerateModulesMask();
bool CreateConnectors();
/* ------Internal methods------ */
bool PassedByAllModules(uint64_t mask) const;
void OnProcessStart(NodeContext* context, const std::shared_ptr<CNFrameInfo>& data);
void OnProcessEnd(NodeContext* context, const std::shared_ptr<CNFrameInfo>& data);
void OnProcessFailed(NodeContext* context, const std::shared_ptr<CNFrameInfo>& data, int ret);
void OnDataInvalid(NodeContext* context, const std::shared_ptr<CNFrameInfo>& data);
void OnEos(NodeContext* context, const std::shared_ptr<CNFrameInfo>& data);
void OnPassThrough(const std::shared_ptr<CNFrameInfo>& data);
void TransmitData(NodeContext* context, const std::shared_ptr<CNFrameInfo>& data);
void TaskLoop(NodeContext* context, uint32_t conveyor_idx);
EventHandleFlag DefaultBusWatch(const Event& event);
void UpdateByStreamMsg(const StreamMsg& msg);
void StreamMsgHandleFunc();
std::vector<std::string> GetSortedModuleNames();
std::unique_ptr<CNGraph<NodeContext>> graph_;
std::vector<std::string> sorted_module_names_;
std::string name_;
std::atomic<bool> running_{false};
std::unique_ptr<EventBus> event_bus_ = nullptr;
std::unique_ptr<IdxManager> idxManager_ = nullptr;
std::vector<std::thread> threads_;
// message observer members
ThreadSafeQueue<StreamMsg> msgq_;
std::thread smsg_thread_;
StreamMsgObserver* smsg_observer_ = nullptr;
std::atomic<bool> exit_msg_loop_{false};
uint64_t all_modules_mask_ = 0;
std::unique_ptr<PipelineProfiler> profiler_;
std::function<void(std::shared_ptr<CNFrameInfo>)> frame_done_cb_ = NULL;
/**
* StreamIdx helpers for SourceModule instances.
* ModuleIdx helpers for Module instances
*/
friend class Module;
friend class SourceModule;
uint32_t GetStreamIndex(const std::string& stream_id) {
if (idxManager_) {
return idxManager_->GetStreamIndex(stream_id);
}
return kInvalidStreamIdx;
}
void ReturnStreamIndex(const std::string& stream_id) {
if (idxManager_) {
idxManager_->ReturnStreamIndex(stream_id);
}
}
size_t GetModuleIdx() {
if (idxManager_) {
return idxManager_->GetModuleIdx();
}
return kInvalidModuleId;
}
void ReturnModuleIdx(size_t idx) {
if (idxManager_) {
idxManager_->ReturnModuleIdx(idx);
}
}
}; // class Pipeline2.3 cnstream::Module类
CNStream SDK 要求所有的 Module 类使⽤统⼀接⼝和数据结构 cnstream::CNFrameInfo 。从框架上要求了 module 的通⽤性，并简化了 module 的编写。实现具体 module 的⽅式如下：
- 从 cnstream::Module 派⽣：适合功能单⼀，内部不需要并发处理的场景。Module 实现只需要关注对 CNFrameInfo 的处理，由框架传递（transmit）CNFrameInfo。
- 从 cnstream::ModuleEx 派⽣：Module 除了处理 CNFrameInfo 之外，还负责 CNFrameInfo 的传递，以及保证数据顺序带来的灵活性，从⽽可以实现内部并发。
配置搭建 pipeline 的基础是实现根据 module 类名字创建 module 实例，因此具体 module 类还需要继承 cnstream::ModuleCreator 。
⼀个 module 的实例，会使⽤⼀个或者多个线程对多路数据流进⾏处理，每⼀路数据流使⽤ pipeline 范围内唯⼀的 stream_id 进⾏标识。此外从 cnstream::IModuleObserver 接⼝类继承实现⼀个观察者，并通过 SetObserver 注册到 module 中，应⽤程序就可以观察每个 module 处理结果。
cnstream::Module 类在 cnstream_module.hpp ⽂件定义，主要接⼝如下。cnstream_module.hpp ⽂件存放在 framework/core/include ⽂件夹下
/**
* @class Module.
*
* @brief Module is the parent class of all modules. A module could have configurable
* number of upstream links and downstream links.
* Some modules are already constructed with a framework,
* such as source, inferencer, and so on. You can also design your own modules.
*/
class Module : private NonCopyable {
public:
/**
* @brief Constructor. A constructor to construct module object.
*
* @param[in] name The name of a module. Modules defined in a pipeline must have different names.
*
* @return No return value.
*/
explicit Module(const std::string &name) : name_(name) {}
/**
* @brief Destructor. A destructor to destruct module instance.
*
* @return No return value.
*/
virtual ~Module();
/**
* @brief Registers an observer to the module.
*
* @param[in] observer An observer you defined.
*
* @return No return value.
*/
void SetObserver(IModuleObserver *observer) {
RwLockWriteGuard guard(observer_lock_);
observer_ = observer;
}
/**
* @brief Opens resources for a module.
*
* @param[in] param_set A set of parameters for this module.
*
* @return Returns true if this function has run successfully. Otherwise, returns false.
*
* @note You do not need to call this function by yourself. This function is called
* by pipeline automatically when the pipeline is started. The pipeline calls the ``Process`` function
* of this module automatically after the ``Open`` function is done.
*/
virtual bool Open(ModuleParamSet param_set) = 0;
/**
* @brief Closes resources for a module.
*
* @return No return value.
*
* @note You do not need to call this function by yourself. This function is called
* by pipeline automatically when the pipeline is stopped. The pipeline calls the ``Close`` function
* of this module automatically after the ``Open`` and ``Process`` functions are done.
*/
virtual void Close() = 0;
/**
* @brief Processes data.
*
* @param[in] data The data to be processed by the module.
*
* @retval >=0: The data is processed successfully.
* @retval <0: Pipeline will post an event with the EVENT_ERROR event type and the return number.
*/
virtual int Process(std::shared_ptr<CNFrameInfo> data) = 0;
/**
* @brief Notifies flow-EOS arriving, the module should reset internal status if needed.
*
* @param[in] stream_id The stream identification.
*
* @note This function will be invoked when flow-EOS is forwarded by the framework.
*/
virtual void OnEos(const std::string &stream_id) {}
/**
* @brief Gets the name of this module.
*
* @return Returns the name of this module.
*/
inline std::string GetName() const { return name_; }
/**
* @brief Posts an event to the pipeline.
*
* @param[in] type The type of an event.
* @param[in] msg The event message string.
* @param[in] msg_id The event message id.
*
* @return Returns true if this function has run successfully. Returns false if this
* module has not been added to the pipeline.
*/
bool PostEvent(EventType type, const std::string &msg, const std::string &msg_id = "");
/**
* @brief Posts an event to the pipeline.
*
* @param[in] Event with event type, stream_id, message, module name and thread_id.
*
* @return Returns true if this function has run successfully. Returns false if this
* module has not been added to the pipeline.
*/
bool PostEvent(Event e);
/**
* @brief Transmits data to the following stages.
*
* Valid when the module has permission to transmit data by itself.
*
* @param[in] data A pointer to the information of the frame.
*
* @return Returns true if the data has been transmitted successfully. Otherwise, returns false.
*/
bool TransmitData(std::shared_ptr<CNFrameInfo> data);
/**
* @brief Checks parameters for a module, including parameter name, type, value, validity, and so on.
*
* @param[in] param_set Parameters for this module.
*
* @return Returns true if this function has run successfully. Otherwise, returns false.
*/
virtual bool CheckParamSet(const ModuleParamSet ¶m_set) const { return true; }
/**
* @brief Gets the pipeline this module belongs to.
*
* @return Returns the pointer to pipeline instance.
*/
Pipeline *GetContainer() const { return container_; }
/**
* @brief Gets module profiler.
*
* @return Returns a pointer to the module's profiler.
*/
ModuleProfiler *GetProfiler();
/**
* @brief Checks if this module has permission to transmit data by itself.
*
* @return Returns true if this module has permission to transmit data by itself. Otherwise, returns false.
*
* @see Process
*/
bool HasTransmit() const { return hasTransmit_.load(); }
/**
* Each module registers its own parameters and descriptions.
* CNStream Inspect tool uses this class to detect parameters of each module.
*/
ParamRegister param_register_;
#ifdef UNIT_TEST
public: // NOLINT
#else
protected: // NOLINT
#endif
friend class Pipeline;
friend class CNFrameInfo;
/**
* @brief Sets a container to this module and identifies which pipeline the module is added to.
*
* @param[in] container A pipeline pointer to the container of this module.
*
* @note This function is called automatically by the pipeline after this module
* is added into the pipeline. You do not need to call this function by yourself.
*/
void SetContainer(Pipeline *container);
/**
* @brief Processes the data. This function is called by a pipeline.
*
* @param[in] data A pointer to the information of the frame.
*
* @retval 0: The process has been run successfully. The data should be transmitted by framework then.
* @retval >0: The process has been run successfully. The data has been handled by this module. The ``hasTransmit_``
* must be set. The Pipeline::ProvideData should be called by Module to transmit data to the next modules in the
* pipeline.
* @retval <0: Pipeline posts an event with the EVENT_ERROR event type and return number.
*/
int DoProcess(std::shared_ptr<CNFrameInfo> data);
Pipeline *container_ = nullptr; ///< The container.
RwLock container_lock_;
std::string name_; ///< The name of the module.
std::atomic<bool> hasTransmit_{false}; ///< Whether it has permission to transmit data.
void SetPriority(int priority) {
if (priority < 1 || priority > 99) return;
priority_ = priority;
}
int GetPriority() { return priority_; }
#ifdef UNIT_TEST
public: // NOLINT
#else
private: // NOLINT
#endif
IModuleObserver *observer_ = nullptr;
RwLock observer_lock_;
void NotifyObserver(std::shared_ptr<CNFrameInfo> data) {
RwLockReadGuard guard(observer_lock_);
if (observer_) {
observer_->Notify(data);
}
}
int DoTransmitData(std::shared_ptr<CNFrameInfo> data);
size_t GetId();
size_t id_ = kInvalidModuleId;
NodeContext *context_ = nullptr; // used by pipeline
int priority_ = -1;
};2.4 cnstream::CNFrameInfo 类
cnstream::CNFrameInfo 类 是 module 之 间 传 递 的 数 据 结 构， 即 pipeline 的 Context。 该 类 在cnstream_frame.hpp ⽂件中定义。cnstream_frame.hpp ⽂件存放在 framework/core/include ⽂件夹下。
/**
* @class CNFrameInfo
*
* @brief CNFrameInfo is a class holding the information of a frame.
*
*/
class CNFrameInfo : private NonCopyable {
public:
/**
* @brief Creates a CNFrameInfo instance.
*
* @param[in] stream_id The data stream alias. Identifies which data stream the frame data comes from.
* @param[in] eos Whether this is the end of the stream. This parameter is set to false by default to
* create a CNFrameInfo instance. If you set this parameter to true,
* CNDataFrame::flags will be set to ``CN_FRAME_FLAG_EOS``. Then, the modules
* do not have permission to process this frame. This frame should be handed over to
* the pipeline for processing.
*
* @return Returns ``shared_ptr`` of ``CNFrameInfo`` if this function has run successfully. Otherwise, returns NULL.
*/
static std::shared_ptr<CNFrameInfo> Create(const std::string& stream_id, bool eos = false,
std::shared_ptr<CNFrameInfo> payload = nullptr);
CNS_IGNORE_DEPRECATED_PUSH
private:
CNFrameInfo() = default;
public:
/**
* @brief Destructs CNFrameInfo object.
*
* @return No return value.
*/
~CNFrameInfo();
CNS_IGNORE_DEPRECATED_POP
/**
* @brief Checks whether DataFrame is end of stream (EOS) or not.
*
* @return Returns true if the frame is EOS. Returns false if the frame is not EOS.
*/
bool IsEos() { return (flags & static_cast<size_t>(cnstream::CNFrameFlag::CN_FRAME_FLAG_EOS)) ? true : false; }
/**
* @brief Checks whether DataFrame is removed or not.
*
* @return Returns true if the frame is EOS. Returns false if the frame is not EOS.
*/
bool IsRemoved() {
return (flags & static_cast<size_t>(cnstream::CNFrameFlag::CN_FRAME_FLAG_REMOVED)) ? true : false;
}
/**
* @brief Checks if DataFrame is valid or not.
*
*
*
* @return Returns true if frame is invalid, otherwise returns false.
*/
bool IsInvalid() {
return (flags & static_cast<size_t>(cnstream::CNFrameFlag::CN_FRAME_FLAG_INVALID)) ? true : false;
}
/**
* @brief Sets index (usually the index is a number) to identify stream.
*
* @param[in] index Number to identify stream.
*
* @return No return value.
*
* @note This is only used for distributing each stream data to the appropriate thread.
* We do not recommend SDK users to use this API because it will be removed later.
*/
void SetStreamIndex(uint32_t index) { channel_idx = index; }
/**
* @brief Gets index number which identifies stream.
*
*
*
* @return Index number.
*
* @note This is only used for distributing each stream data to the appropriate thread.
* We do not recommend SDK users to use this API because it will be removed later.
*/
uint32_t GetStreamIndex() const { return channel_idx; }
std::string stream_id; /*!< The data stream aliases where this frame is located to. */
int64_t timestamp = -1; /*!< The time stamp of this frame. */
std::atomic<size_t> flags{0}; /*!< The mask for this frame, ``CNFrameFlag``. */
Collection collection; /*!< Stored structured data. */
std::shared_ptr<cnstream::CNFrameInfo> payload = nullptr; /*!< CNFrameInfo instance of parent pipeline. */
private:
/**
* The below methods and members are used by the framework.
*/
friend class Pipeline;
mutable uint32_t channel_idx = kInvalidStreamIdx; ///< The index of the channel, stream_index
void SetModulesMask(uint64_t mask);
uint64_t GetModulesMask();
uint64_t MarkPassed(Module* current); // return changed mask
RwLock mask_lock_;
/* Identifies which modules have processed this data */
uint64_t modules_mask_ = 0;
};
/*!
* Defines an alias for the std::shared_ptr<CNFrameInfo>. CNFrameInfoPtr now denotes a shared pointer of frame
* information.
*/
using CNFrameInfoPtr = std::shared_ptr<CNFrameInfo>;2.5 cnstream::EventBus 类
cnstream::EventBus 类是各个模块与 pipeline 通信的事件总线。各模块发布事件到总线上，由总线监听器接收。⼀条事件总线可以拥有多个监听器。
每条 pipeline 有⼀条事件总线及对应的⼀个默认事件监听器。pipeline 会对事件总线进⾏轮询，收到事件后分发给监听器。
cnstream::EventBus 类 在 cnstream_eventbus.hpp ⽂ 件 中 定 义， 主 要 接 ⼝ 如 下。cnstream_eventbus.hpp ⽂件存放在 framework/core/include ⽂件夹下
/**
* @class EventBus
*
* @brief EventBus is a class that transmits events from modules to a pipeline.
*/
class EventBus : private NonCopyable {
public:
friend class Pipeline;
/**
* @brief Destructor. A destructor to destruct event bus.
*
* @return No return value.
*/
~EventBus();
/**
* @brief Starts an event bus thread.
*
* @return Returns true if start successfully, otherwise false.
*/
bool Start();
/**
* @brief Stops an event bus thread.
*
* @return No return values.
*/
void Stop();
/**
* @brief Adds a watcher to the event bus.
*
* @param[in] func The bus watcher to be added.
*
* @return The number of bus watchers that has been added to this event bus.
*/
uint32_t AddBusWatch(BusWatcher func);
/**
* @brief Posts an event to a bus.
*
* @param[in] event The event to be posted.
*
* @return Returns true if this function run successfully. Otherwise, returns false.
*/
bool PostEvent(Event event);
#ifndef UNIT_TEST
private: // NOLINT
#else
Event PollEventToTest();
#endif
EventBus() = default;
/**
* @brief Polls an event from a bus.
*
* @return Returns the event.
*
* @note This function is blocked until an event availabe or the bus stopped.
*/
Event PollEvent();
/**
* @brief Gets all bus watchers from the event bus.
*
* @return A list with pairs of bus watcher and module.
*/
const std::list<BusWatcher> &GetBusWatchers() const;
/**
* @brief Removes all bus watchers.
*
* @return No return value.
*/
void ClearAllWatchers();
/**
* @brief Checks if the event bus is running.
*
* @return Returns true if the event bus is running. Otherwise, returns false.
*/
bool IsRunning();
void EventLoop();
private:
mutable std::mutex watcher_mtx_;
ThreadSafeQueue<Event> queue_;
#ifdef UNIT_TEST
ThreadSafeQueue<Event> test_eventq_;
bool unit_test = true;
#endif
std::list<BusWatcher> bus_watchers_;
std::thread event_thread_;
std::atomic<bool> running_{false};
}; // class EventBus
2.6 cnstream::Event类
cnstream::Event 类是模块和 pipepline 之间通信的基本单元，即事件。事件由四个部分组成：事件
类型、消息、发布事件的模块、发布事件的线程号。消息类型包括：⽆效、错误、警告、EOS(End of
Stream)、停⽌，数据流错误，以及⼀个预留类型。cnstream::Event 类 在 cnstream_eventbus.hpp ⽂ 件 定 义，cnstream_eventbus.hpp ⽂ 件 存 放 在framework/core/include ⽂件夹下。
2.7 cnstream::Collection类
cnstream::Collection 是 CNStream ⽤于存放任意类型数据的类, 通过该类添加与读取数据的操作都是线程安全的。⽅便⽤⼾使⽤⾃定义的类型存储推理结果。该类在 cnstream_collection.hpp ⽂件中定义。cnstream_collection.hpp ⽂件存放在 framework/core/include ⽂件夹下
/**
* @class Collection
*
* @brief Collection is a class storing structured data of variable types.
*
* @note This class is thread safe.
*/
class Collection : public NonCopyable {
public:
/*!
* @brief Constructs an instance with empty value.
*
* @return No return value.
*/
Collection() = default;
/*!
* @brief Destructs an instance.
*
* @return No return value.
*/
~Collection() = default;
/**
* @brief Gets the reference to the object of typename ValueT if it exists, otherwise crashes.
*
* @param[in] tag The unique identifier of the data.
*
* @return Returns the reference to the object of typename ValueT which is tagged by `tag`.
*/
template <typename ValueT>
ValueT& Get(const std::string& tag);
/**
* @brief Adds data tagged by `tag`. Crashes when there is already a piece of data tagged by `tag`.
*
* @param[in] tag The unique identifier of the data.
* @param[in] value Value to be add.
*
* @return Returns the reference to the object of typename ValueT which is tagged by `tag`.
*/
template <typename ValueT>
ValueT& Add(const std::string& tag, const ValueT& value);
/**
* @brief Adds data tagged by `tag` using move semantics. Crashes when there is already a piece of data tagged by
* `tag`.
*
* @param[in] tag The unique identifier of the data.
* @param[in] value Value to be add.
*
* @return Returns the reference to the object of typename ValueT which is tagged by `tag`.
*/
template <typename ValueT>
ValueT& Add(const std::string& tag, ValueT&& value);
/**
* @brief Adds data tagged by `tag`, only if there is no piece of data tagged by `tag`.
*
* @param[in] tag The unique identifier of the data.
* @param[in] value Value to be add.
*
* @return Returns true if the data is added successfully, otherwise returns false.
*/
template <typename ValueT>
bool AddIfNotExists(const std::string& tag, const ValueT& value);
/**
* @brief Adds data tagged by `tag` using move semantics, only if there is no piece of data tagged by
* `tag`.
*
* @param[in] tag The unique identifier of the data.
* @param[in] value Value to be add.
*
* @return Returns true if the data is added successfully, otherwise returns false.
*/
template <typename ValueT>
bool AddIfNotExists(const std::string& tag, ValueT&& value);
/**
* @brief Checks whether there is the data tagged by `tag`.
*
* @param[in] tag The unique identifier of the data.
*
* @return Returns true if there is already a piece of data tagged by `tag`, otherwise returns false.
*/
bool HasValue(const std::string& tag);
#if !defined(_LIBCPP_NO_RTTI)
/**
* @brief Gets type information for data tagged by `tag`.
*
* @param[in] tag The unique identifier of the data.
*
* @return Returns type information of the data tagged by `tag`.
*/
const std::type_info& Type(const std::string& tag);
/**
* @brief Checks if the type of data tagged by `tag` is `ValueT` or not.
*
* @param tag The unique identifier of the data.
*
* @return Returns true if the type of data tagged by `tag` is ``ValueT``, otherwise returns false.
*/
template <typename ValueT>
bool TaggedIsOfType(const std::string& tag);
#endif
private:
void Add(const std::string& tag, std::unique_ptr<cnstream::any>&& value);
bool AddIfNotExists(const std::string& tag, std::unique_ptr<cnstream::any>&& value);
private:
std::map<std::string, std::unique_ptr<cnstream::any>> data_;
RwLock rw_lock_;
}; // class Collection数据最终以键值对形式存储于如下所⽰的 data_ 中
std::map<std::string, std::unique_ptr<cnstream::any>> data_;⽤⼾通过 Add 接⼝添加数据, 通过 Get 接⼝读取数据，如下代码所⽰，
//⽤⼾⾃定义类型数据
struct UserDefinedData {
int id = 0;
}
//由⽤⼾⾃定义索引 UserDefinedData 类型的键值
constexpr std::string kTag = "Tag";
//创建 CNFrameInfo 和 UserDefinedData 数据
CNFrameInfoPtr data = CNFrameInfo::Create("stream_0"/*stream id*/, false/*eos*/);
std::shared_ptr<UserDefinedData> my_data = std::make_shared<UserDefinedData>();
//将 example_data 添加到 collection 中
data->collection.Add(kTag,my_data);
//在其他地⽅获取数据并对数据进⾏处理
auto ptr = data->collection.Get<std::shared_ptr<UserDefinedData>>(kTag)；
ptr->id++;参考文献：
https://www.cambricon.com/docs/cnstream/user_guide_html/index.html