JSON SerDe: date column in any time zone - for at least Hive-3.1+

[randallwhitman]

Date-time tests pass with Hive-3.1 profile.
Hive-3.0 and before may exhibit HIVE-12192.
Complete handling of absent/invalid date-time as SQL NULL would be broader scope (separate).
#182 @tpolong

[tpolong]

This issue seems to have been forgotten

[randallwhitman]

@tpolong Have you tested with this branch?

[tpolong]

您好，您的邮件已收到，谢谢！

[tpolong]

[ERROR] \work\code\spatial-framework-for-hadoop\hive\src\main\java\com\esri\hadoop\hive\serde\BaseJsonSerDe.java:[327,24] 错误: 无法将类 HiveShims中的方法 getPrimitiveEpoch应用到给定类型;
[ERROR] \work\code\spatial-framework-for-hadoop\hive\src\main\java\com\esri\hadoop\hive\serde\BaseJsonSerDe.java:[431,59] 错误: 不兼容的类型: long无法转换为Date

[tpolong]

change
case DATE: // DateWritable stores days not milliseconds. See also HIVE-12192.
HiveShims.setDateWritable(row.get(fieldIndex), parseDate(parser));
to
case DATE: // DateWritable stores days not milliseconds. See also HIVE-12192.
HiveShims.setDateWritable(row.get(fieldIndex), new java.sql.Date(parseDate(parser)) );

[randallwhitman]

Using new java.sql.Date(epoch) was the prior approach that no longer worked with v3.1 of Hive - #174.

[randallwhitman]

Question raised: is it in fact possible to interpret a day-only (no time-of-day) date in a foolproof manner, worldwide (for-all/independent-of) time zones.
One idea is to interprete date as noon UTC, which should be the same Gregorian/ISO calendar date worldwide, notwithstanding time of day ranging from 00:00.000 to 23:59.999 . This would break with seasonal shift ("daylight") of time in the zones immediately adjacent to the international date line - would it work for all other cases?
But then what if data is collected by observers in UTC-8 and analyzed on computers in UTC+8, or vice-versa?
What if we have a SerDe parameter constructor for the observation timezone, and interpret dates as noon in that time zone? And if that is helpful, how to use that from HQL/SQL?

[tpolong]

您好，您的邮件已收到，谢谢！

[tpolong]

I have tested the latest code and it works fine