版权声明：如若转载，请联系作者。 https://blog.csdn.net/liu16659/article/details/82468225
openTSDB中存在冗余数据怎么办？
[root@server4 build]# ./tsdb fsck --full-scan --threads=1 --fix --resolve-duplicates --compact
2018-09-02 23:33:45,967 INFO [main] Config: Successfully loaded configuration file: opentsdb.conf
2018-09-02 23:33:45,985 INFO [main] Config: Successfully loaded configuration file: opentsdb.conf
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:host.name=server4
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:java.version=1.8.0_171
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:java.vendor=Oracle Corporation
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.el7_5.x86_64/jre
····
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:java.io.tmpdir=/tmp
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:java.compiler=<NA>
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:os.name=Linux
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:os.arch=amd64
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:os.version=3.10.0-862.9.1.el7.x86_64
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:user.name=root
2018-09-02 23:33:46,177 INFO [main] ZooKeeper: Client environment:user.home=/root
2018-09-02 23:33:46,178 INFO [main] ZooKeeper: Client environment:user.dir=/usr/local/opentsdb-2.3.0/build
2018-09-02 23:33:46,178 INFO [main] ZooKeeper: Initiating client connection, connectString=server4,server5,server6 sessionTimeout=5000 watcher=org.hbase.async.HBaseClient$ZKClient@7494e528
2018-09-02 23:33:46,192 INFO [main-SendThread(server4:2181)] ClientCnxn: Opening socket connection to server server4/192.168.211.4:2181. Will not attempt to authenticate using SASL (unknown error)
2018-09-02 23:33:46,201 INFO [main] HBaseClient: Need to find the -ROOT- region
2018-09-02 23:33:46,272 INFO [main-SendThread(server4:2181)] ClientCnxn: Socket connection established to server4/192.168.211.4:2181, initiating session
2018-09-02 23:33:46,279 INFO [main-SendThread(server4:2181)] ClientCnxn: Session establishment complete on server server4/192.168.211.4:2181, sessionid = 0x400405a780b003f, negotiated timeout = 5000
2018-09-02 23:33:46,297 INFO [main-EventThread] HBaseClient: Connecting to .META. region @ 192.168.211.5:16020
2018-09-02 23:33:46,375 INFO [main-EventThread] ZooKeeper: Session: 0x400405a780b003f closed
2018-09-02 23:33:46,375 INFO [main-EventThread] ClientCnxn: EventThread shut down
2018-09-02 23:33:46,400 INFO [AsyncHBase I/O Worker #1] HBaseClient: Added client for region RegionInfo(table="tsdb", region_name="tsdb,,1531477215287.9cb3eb86a739464fc2381ed6ebbac5f4.", stop_key=""), which was added to the regions cache. Now we know that RegionClient@1775360065(chan=[id: 0xe024b32c, /192.168.211.4:43608 => /192.168.211.5:16020], #pending_rpcs=0, #batched=0, #rpcs_inflight=1) is hosting 1 region.
2018-09-02 23:33:46,401 INFO [AsyncHBase I/O Worker #1] HBaseClient: Added client for region RegionInfo(table="tsdb-uid", region_name="tsdb-uid,,1531477213838.960cd2d7815bda8bfb755401a732128f.", stop_key=""), which was added to the regions cache. Now we know that RegionClient@1003400516(chan=null, #pending_rpcs=0, #batched=0, #rpcs_inflight=0) is hosting 1 region.
2018-09-02 23:33:46,404 INFO [main] Fsck: Starting full table scan
2018-09-02 23:33:46,411 INFO [main] Fsck: Spooling up [1] worker threads
2018-09-02 23:33:46,529 INFO [Fsck #1] Fsck: More than one column had a value for the same timestamp: (1535899800000 - Sun Sep 02 22:50:00 CST 2018)
row key: (0000025B8BECE0000002000004)
write time: (1535901288128) compacted: (true) qualifier: [56, 65, 60, 1, -121, 1, -69, -127] <--- keep oldest
write time: (1535901348988 - Sun Sep 02 23:15:48 CST 2018) compacted: (false) qualifier: [-69, -127] value: 460.0
2018-09-02 23:33:49,343 INFO [main] Fsck: Thread [Thread[Fsck #1,5,]] Finished
2018-09-02 23:33:49,344 INFO [main] Fsck: Key Values Processed: 14
2018-09-02 23:33:49,344 INFO [main] Fsck: Rows Processed: 11
2018-09-02 23:33:49,344 INFO [main] Fsck: Valid Datapoints: 27
2018-09-02 23:33:49,344 INFO [main] Fsck: Annotations: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Invalid Row Keys Found: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Invalid Rows Deleted: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Duplicate Datapoints: 1
2018-09-02 23:33:49,344 INFO [main] Fsck: Duplicate Datapoints Resolved: 1
2018-09-02 23:33:49,344 INFO [main] Fsck: Orphaned UID Rows: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Orphaned UID Rows Deleted: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Possible Future Objects: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Unknown Objects: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Unknown Objects Deleted: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Unparseable Datapoint Values: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Unparseable Datapoint Values Deleted: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Improperly Encoded Floating Point Values: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Improperly Encoded Floating Point Values Fixed: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Unparseable Compacted Columns: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Unparseable Compacted Columns Deleted: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Datapoints Qualified for VLE : 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Datapoints Compressed with VLE: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Bytes Saved with VLE: 0
2018-09-02 23:33:49,344 INFO [main] Fsck: Total Errors: 1
2018-09-02 23:33:49,344 INFO [main] Fsck: Total Correctable Errors: 1
2018-09-02 23:33:49,344 INFO [main] Fsck: Total Errors Fixed: 1
2018-09-02 23:33:49,344 INFO [main] Fsck: Completed fsck in [3] seconds
2018-09-02 23:33:49,344 INFO [main] TSDB: Flushing compaction queue
2018-09-02 23:33:49,345 INFO [main] HBaseClient: Channel [id: 0xf490be09, /192.168.211.4:53620 => /192.168.211.6:16020] is disconnecting: [id: 0xf490be09, /192.168.211.4:53620 => /192.168.211.6:16020] DISCONNECT
2018-09-02 23:33:49,346 INFO [main] HBaseClient: Channel [id: 0xe024b32c, /192.168.211.4:43608 => /192.168.211.5:16020] is disconnecting: [id: 0xe024b32c, /192.168.211.4:43608 => /192.168.211.5:16020] DISCONNECT
2018-09-02 23:33:49,346 INFO [main] HBaseClient: Lost connection with the .META. region
2018-09-02 23:33:49,351 INFO [main] TSDB: Completed shutting down the TSDB