1. Convert the TUM dataset to klg format and run ElasticFusion
1. Download the TUM dataset
Download address: https://cvg.cit.tum.de/data/datasets/rgbd-dataset/download
TUM dataset 1 shares three robots, recorded as fr1, fr2, fr3. The parameters of these three cameras are here: http://vision.in.tum.de/data/datasets/rgbd-dataset/file_formats#intrinsic_camera_calibration_of_the_kinect
I downloaded 'freiburg3_teddy', the included files are as follows:
Description:
(1 ) The rgb/ and depth/ directories store the collected image files in png format. The color image is an 8-bit three-channel image, and the depth image is a 16-bit single-channel image. Images are named with timestamps.
(2) rgb.txt and depth.txt record the collection time and corresponding file name of each file.
(3) groundtruth.txt is the camera pose collected by the external motion capture system, the format is (time, tx , ty , tz , qx , qy , qz , qw ), each data is: time (time), position ( x,y,z), attitude quaternion (qx, qy, qz, qw).
You can draw a picture to see what's inside:
create a draw_groundtruth.py file, and copy the following code into it.
#!/usr/bin/env python
# coding=utf-8
import numpy as np
import matplotlib.pyplot as plt
import mpl_toolkits.mplot3d
f = open("./groundtruth.txt")
x = []
y = []
z = []
for line in f:
if line[0] == '#':
continue
data = line.split()
x.append( float(data[1] ) )
y.append( float(data[2] ) )
z.append( float(data[3] ) )
ax = plt.subplot( 111, projection='3d')
ax.plot(x,y,z)
plt.show()
Run in terminal:
python3 draw_groundtruth.py
See the trajectory map:
2. The code address for converting the TUM dataset to klg format: https://github.com/HTLife/png_to_klg
Download the code:
git clone https://github.com/HTLife/png_to_klg.git
3. The collection of color map, depth map and standard trajectory are all independent, and the collection frequency of trajectory is much higher than that of image. Before using the data, it is necessary to align the data in time according to the acquisition time, so as to pair the color map and the depth map. In principle, we can regard the data whose acquisition time is close to a threshold as a pair of images. And the pose at a similar time is regarded as the real acquisition position of the image.
Copy the associate.py file in png_to_klg to the TUM dataset, and then execute the code:
./associate.py depth.txt rgb.txt > associations.txt
The one-to-one correspondence between the depth map and the rgb image is completed.
Note: If you are python3, you will report such an error because of the high version of python: AttributeError: 'dict_keys' object has no attribute 'remove' The solution is to change the following two lines of code
:
first_keys = first_list.keys()
second_keys = second_list.keys()
change into:
first_keys = list(first_list)
second_keys = list(second_list)
4. Compile the png_to_klg file
cd png_to_klg
mkdir build && cd build
cmake ..
make
5. Convert the TUM dataset to .klg format
After compiling, enter the build of png_to_klg, and execute the following code on the terminal:
./pngtoklg -w '/数据集存储路径/' -o '/生成的.klg文件存储路径/xxx.klg' -t #生成.klg文件
The .klg file can be generated.
6. Run ElasticFusion
Enter the build folder of ElasticFusion and run the following code:
./ElasticFusion -l (此处填写生成的.klg文件的路径)
Display the result:
After the execution, the xxx.klg.freiburg file will be generated in the folder where the .klg is located, which is the camera pose estimated by ElasticFusion, and then you can use the tools provided by TUM, such as evaluate_ate.py.
Others: Because the recording frequency of the external motion capture device is relatively high, the obtained trajectory points are much denser than the images, so to find the real position of each image, you can also use the alignment method to match the associations.txt and groundtruth.txt Time information:
python associate.py associations.txt groundtruth.txt > associate_with_groundtruth.txt
At this point, our new file associations_with_groundtruth.txt contains the pose information for each frame.
2. Error evaluation
(to be completed)