报错:
用命令启动:bin/flume-ng agent --conf ./conf/ -f conf/ flume-conf.conf -n agent1
一切正常,而且flume.log日志也正常。
用bin/flume-ng avro-client -H kf-app1 -p 44444 -F /chunk1/demo/flume/test2.txt发送文件,flume.log如下:
[demoe3base@kf-app1 logs]$ tail -f flume.log
08 五月 2013 14:34:31,370 INFO [lifecycleSupervisor-1-3] (org.apache.flume.instrumentation.MonitoredCounterGroup.start:82) - Component type: SOURCE, name: source1 started
08 五月 2013 14:34:31,370 INFO [lifecycleSupervisor-1-3] (org.apache.flume.source.AvroSource.start:155) - Avro source source1 started.
08 五月 2013 14:34:45,932 INFO [pool-6-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x34bf1d3b, /172.21.3.61:39262 => /172.21.3.60:44444] OPEN
08 五月 2013 14:34:45,938 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x34bf1d3b, /172.21.3.61:39262 => /172.21.3.60:44444] BOUND: /172.21.3.60:44444
08 五月 2013 14:34:45,938 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x34bf1d3b, /172.21.3.61:39262 => /172.21.3.60:44444] CONNECTED: /172.21.3.61:39262
08 五月 2013 14:34:46,267 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x34bf1d3b, /172.21.3.61:39262 :> /172.21.3.60:44444] DISCONNECTED
08 五月 2013 14:34:46,267 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x34bf1d3b, /172.21.3.61:39262 :> /172.21.3.60:44444] UNBOUND
08 五月 2013 14:34:46,268 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x34bf1d3b, /172.21.3.61:39262 :> /172.21.3.60:44444] CLOSED
08 五月 2013 14:34:46,268 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.channelClosed:209) - Connection to /172.21.3.61:39262 disconnected.
08 五月 2013 14:34:46,922 INFO [hdfs-hdfssink1-call-runner-0] (org.apache.flume.sink.hdfs.BucketWriter.doOpen:189) - Creating hdfs://kf-app1:8020//FlumeData.1367994886244.tmp
问题来了:1、为什么是“FlumeData.1367994886244.tmp”临时文件,而不能将文件关闭呢?当我把代理强行kill掉或者关掉后,日志才打印出“08 五月 2013 14:21:17,556 INFO [hdfs-hdfssink1-call-runner-5] (org.apache.flume.sink.hdfs.BucketWriter.renameBucket:379) - Renaming hdfs://kf-app1:8020/flume/FlumeData.1367993804350.tmp to hdfs://kf-app1:8020/flume/FlumeData.1367993804350”,难道说代理不能够自动关闭?
2、而且在发送第二个文件后发现日志报错UNBOUND,难道说一个通道直接接收一个文件?
08 五月 2013 14:30:47,202 INFO [pool-6-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x5a9b8ff9, /172.21.3.61:38652 => /172.21.3.60:44444] OPEN
08 五月 2013 14:30:47,203 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x5a9b8ff9, /172.21.3.61:38652 => /172.21.3.60:44444] BOUND: /172.21.3.60:44444
08 五月 2013 14:30:47,203 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x5a9b8ff9, /172.21.3.61:38652 => /172.21.3.60:44444] CONNECTED: /172.21.3.61:38652
08 五月 2013 14:30:47,913 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x5a9b8ff9, /172.21.3.61:38652 :> /172.21.3.60:44444] DISCONNECTED
08 五月 2013 14:30:47,913 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x5a9b8ff9, /172.21.3.61:38652 :> /172.21.3.60:44444] UNBOUND
08 五月 2013 14:30:47,913 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream:171) - [id: 0x5a9b8ff9, /172.21.3.61:38652 :> /172.21.3.60:44444] CLOSED
08 五月 2013 14:30:47,914 INFO [pool-7-thread-1] (org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.channelClosed:209) - Connection to /172.21.3.61:38652 disconnected
解决方法:
將8082改爲9000
.