gem sources -a https://ruby.taobao.org/这个步骤可能会报错:ERROR: While executing gem … (Gem::Exception)
Unable to require openssl, install OpenSSL and rebuild ruby (preferred) or use non-HTTPS sources
[root@Source ~]# gem sources -u ; gem source -l
ERROR: While executing gem … (Gem::Exception)
Unable to require openssl, install OpenSSL and rebuild ruby (preferred) or use non-HTTPS sources
解决这个问题需要
把/root/bi/ruby/lib/ruby/gems/2.4.0/gems/fluentd-0.14.13/fluent.conf拷贝到/et c/fluent目录
配置source和match
source配置:
match配置
这步可能会报/usr/local/lib/ruby/gems/2.2.0/gems/activerecord- 4.2.6/lib/active_record/connection_adapters/connection_specification.rb:177:in gem ‘mysql’` to your Gemfile (and ensure its version is at the minimum required by ActiveRecord). (Gem::LoadError)
这个问题是mysql插件需要用到mysql adapter适配器,需要安装mysql adapter适配器,执行以下命令:
注意是运行gem install mysql2 而不是gem install mysql,同时在source配置里面也要把adapter 设为mysql2。在这一步也可能遇到mysql包冲突的问题,所以建议在一个没有安装mysql的服务器上安装。
因为阿里云datahub是大小写敏感的,所以要把所有字段设为小写,不然字段会变为Null或空白。那么这需要怎么解决呢,查找fluent的官方文档,有record_transformer Filter Plugin这个东西,对于有大写字母的字段我们需配置一个filter,同时注意filter必须配置在match前面。
filter配置