文章目录
环境
- flume使用
| 产品 | 版本 |
|---|---|
| Centos 7 | 3.10.0-1062.9.1.el7.x86_64 |
| java | jdk-8u211-linux-x64 |
| flume | apache-flume-1.9.0-bin |
flume安装与配置
下载地址
- 清华镜像
https://mirrors.tuna.tsinghua.edu.cn/apache/flume/
解压
tar -zxvf apache-flume-1.9.0-bin.tar.gz
创建软链接
ln -s apache-flume-1.8.0-bin/ flume
配置环境变量
vi ~/.bashrc
#FLUME
export FLUME_HOME=/home/hadoop/apache-flume-1.9.0-bin
export PATH=$PATH:$FLUME_HOME/bin
- 生效
source ~/.bashrc
查看版本
flume-ng version
telnet安装
- 检查是否安装 telnet-server和xinetd
rpm -qa telnet-server
rpm -qa xinetd
- 如果没有安装过就安装
yum -y install telnet-server.x86_64 telnet.x86_64 xinetd.x86_64
- 设置开机自启：
systemctl enable xinetd.service
systemctl enable telnet.socket
- 开启service：
systemctl start telnet.socket
systemctl start xinetd
测试flume
创建example.properties
cd conf
vim example.properties
# example.conf: A single-node Flume configuration
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
在这个配置文件中，·配置了source:a1 sink:k1 channel:c1。a1作为信息源监听本机44444端口，并将监听到的信息转换成event，channel缓存这些收集到的事件并发送给sink，sink将这些event传输到另一个终端上。
运行example.properties
bin/flume-ng agent --conf conf --conf-file conf/example.conf --name a1 -Dflume.root.logger=INFO,console
使用telnet
启动另一个终端，模拟远端：
telnet localhost 44444
有如下反馈。
使用后，关闭进程
netstat -lnpt|grep 44444
kill -9 [进程号]