Hive in TimeStamp Accuracy Analysis

---

Hive when using TimeStamp, timestamp default is accurate to the second, how to deal with a time stamp that require accurate to the millisecond in the Hive? How will 13 of various formats into timestamps accurate to the millisecond?

Hive in TimeStamp acquisition and conversion

Hive using CURRENT_TIMESTAMP () function to get the current time to the millisecond

select current_timestamp();
+--------------------------+--+
|           _c0            |
+--------------------------+--+
| 2019-06-02 15:31:33.355  |
+--------------------------+--+
1 row selected (0.316 seconds)

Hive acquires the current timestamp, default UNIX_TIMESTAMP () function, accurate to the second

select unix_timestamp();
+-------------+--+
|     _c0     |
+-------------+--+
| 1559460765  |
+-------------+--+
1 row selected (0.194 seconds)

Hive will be converted to a date stamp type, default FROM_UNIXTIME ()

desc function extended from_unixtime;

from_unixtime(unix_time, format) - returns unix_time in the specified format
Example:                                         
  > SELECT from_unixtime(0, 'yyyy-MM-dd HH:mm:ss') FROM src LIMIT 1;
  '1970-01-01 00:00:00'       

The first argument is only of type int time stamp, the second time is a string format, the second parameter may be null, the default time format: yyyy-MM-dd HH: mm: ss

select from_unixtime(unix_timestamp());
+----------------------+--+
|         _c0          |
+----------------------+--+
| 2019-06-02 15:34:10  |
+----------------------+--+
1 row selected (0.227 seconds)

select from_unixtime(unix_timestamp(), 'yyyyMMddHHmmss');
+-----------------+--+
|       _c0       |
+-----------------+--+
| 20190602153447  |
+-----------------+--+
1 row selected (0.191 seconds)

select from_unixtime(unix_timestamp(), 'yyyy/MM/dd HH:mm:ss');
+----------------------+--+
|         _c0          |
+----------------------+--+
| 2019/06/02 15:35:13  |
+----------------------+--+
1 row selected (0.207 seconds)

Hive acquired millisecond level timestamp

select current_timestamp() as current_time, cast(current_timestamp() as double) * 1000 as timestamp;

2019-06-02 15:44:23.324	1559461463324

The results can be seen from the above two images, a digital display Beeline scientific computing, on the normal display hue

Hive processing milliseconds timestamp

desc function extended to_utc_timestamp;

to_utc_timestamp(timestamp, string timezone) - Assumes given timestamp is in given timezone and converts to UTC (as of Hive 0.8.0)


select to_utc_timestamp(1559461463324, 'GMT');
+--------------------------+--+
|           _c0            |
+--------------------------+--+
| 2019-06-02 15:44:23.324  |
+--------------------------+--+
1 row selected (2.745 seconds)

Converting milliseconds timestamp format specified milliseconds, SSS Representative ms

select date_format(to_utc_timestamp(1559461463324, 'GMT'), 'yyyyMMddHHmmssSSS');
+--------------------+--+
|        _c0         |
+--------------------+--+
| 20190602154423324  |
+--------------------+--+
1 row selected (0.323 seconds)


select date_format(to_utc_timestamp(1559461463324, 'GMT'), 'yyyy/MM/dd HH:mm:ss.SSS');
+--------------------------+--+
|           _c0            |
+--------------------------+--+
| 2019/06/02 15:44:23.324  |
+--------------------------+--+
1 row selected (0.183 seconds)

to sum up

1. Hive acquired timestamp way UNIX_TIMESTAMP () function, which is only accurate to the level of the second time, the exact time for the demanding applications are not suitable for the function.

2. You need to use the cast function to get the current time when Hive millisecond level timestamp current_timestamp () into a timestamp and a double by 1000, then get millisecond level.

3. For timestamp millisecond accuracy Hive stored in the library, in order to ensure no loss of precision time you need to use to_utc_timestamp () function, which supports millisecond level time wrong, but you need to specify the current time zone.