ios doesn't fill in metadata timezone information for rw documents

[shankari]

While investigating documents created directly from javascript today, I discovered that iOS doesn't fill in metadata timezone information, at least for for rw documents. And since the metadata is consistent, it probably doesn't fill in the information for any documents.

We do workaround this by setting the timezone to America/Los_Angeles if it is not present, but that's incorrect for locations other than the west coast of the US. We should either stop using the metadata timezone, or fill it in correctly.

[shankari]

Example sensor_config objects without metadata timezone.

[{u'_id': ObjectId('57966efb115514afcd793360'),
  u'data': {u'accuracy': -1,
   u'accuracy_threshold': 200,
   u'android_geofence_responsiveness': -1,
   u'filter_distance': 50,
   u'filter_time': -1,
   u'geofence_radius': 100,
   u'ios_use_remote_push_for_sync': 1,
   u'ios_use_visit_notifications_for_detection': 1,
   u'is_duty_cycling': 0,
   u'simulate_user_interaction': 1,
   u'trip_end_stationary_mins': 10},
  u'metadata': {u'key': u'config/sensor_config',
   u'type': u'document',
   u'write_ts': 1469476603.373646},
  u'user_id': UUID('1c4e043f-cffe-3429-9ad8-09fadc96ff2f')},
 {u'_id': ObjectId('57964463115514afcd79334e'),
  u'data': {u'accuracy': -1,
   u'accuracy_threshold': 200,
   u'android_geofence_responsiveness': -1,
   u'filter_distance': 50,
   u'filter_time': -1,
   u'geofence_radius': 100,
   u'ios_use_remote_push_for_sync': 1,
   u'ios_use_visit_notifications_for_detection': 1,
   u'is_duty_cycling': 0,
   u'simulate_user_interaction': 1,
   u'trip_end_stationary_mins': 10},
  u'metadata': {u'key': u'config/sensor_config',
   u'type': u'document',
   u'write_ts': 1469473156.203633},
  u'user_id': UUID('3594ff76-d3e2-36c1-9118-45a6a698286e')},
 {u'_id': ObjectId('579671de115514afcd79336e'),
  u'data': {u'accuracy': -1,
   u'accuracy_threshold': 200,
   u'android_geofence_responsiveness': -1,
   u'filter_distance': 50,
   u'filter_time': -1,
   u'geofence_radius': 100,
   u'ios_use_remote_push_for_sync': 1,
   u'ios_use_visit_notifications_for_detection': 1,
   u'is_duty_cycling': 0,
   u'simulate_user_interaction': 0,
   u'trip_end_stationary_mins': 10},
  u'metadata': {u'key': u'config/sensor_config',
   u'type': u'document',
   u'write_ts': 1469478208.197791},
  u'user_id': UUID('0763de67-f61e-3f5d-90e7-518e69793954')}]

[shankari]

android is fine:

2016-07-22 10:00:09,501:DEBUG:Inserting entry AttrDict({'data': AttrDict({'curr_state': 2, 'transition': 2, 'local_dt': AttrDict({'hour': 10, 'month': 7, 'second': 55, 'weekday': 4, 'year': 2016, 'timezone': u'Europe/Copenhagen', 'day': 22, 'minute': 27}), 'ts': 1469176075.087, 'fmt_time': '2016-07-22T10:27:55.087000+02:00'}), '_id': ObjectId('5791e753abbf384cadf544d4'), 'user_id': UUID(<redacted>), 'metadata': AttrDict({u'platform': u'android', 'write_local_dt': LocalDate({'hour': 10, 'month': 7, 'second': 55, 'weekday': 4, 'year': 2016, 'timezone': u'Europe/Copenhagen', 'day': 22, 'minute': 27}), u'key': u'statemachine/transition', u'write_ts': 1469176075.087, u'read_ts': 0, u'type': u'message', u'time_zone': u'Europe/Copenhagen', 'write_fmt_time': '2016-07-22T10:27:55.087000+02:00'})}) into timeseries

but it looks like it works for iOS as well.

2016-07-25 16:00:17,144:DEBUG:Inserting entry AttrDict({'data': AttrDict({u'fmt_time': '2016-07-24T20:23:18.000018-03:00', <redacted>, u'filter': u'distance', 'local_dt': LocalDate({'hour': 20, 'month': 7, 'second': 18, 'weekday': 6, 'year': 2016, 'timezone': u'America/Sao_Paulo', 'day': 24, 'minute': 23}), u'latitude': -23.49404459352755, 'loc': <redacted>, u'sensed_speed': 31.05999946594238, u'accuracy': 10}), '_id': ObjectId('5796353dabbf384cadf9e0c2'), 'user_id': UUID(<redacted>), 'metadata': AttrDict({u'platform': u'ios', 'write_local_dt': LocalDate({'hour': 20, 'month': 7, 'second': 18, 'weekday': 6, 'year': 2016, 'timezone': u'America/Sao_Paulo', 'day': 24, 'minute': 23}), u'read_ts': 0, u'key': u'background/filtered_location', u'plugin': u'none', u'write_ts': 1469402598.315054, u'type': u'sensor-data', u'time_zone': u'America/Sao_Paulo', 'write_fmt_time': '2016-07-24T20:23:18.315054-03:00'})}) into timeseries

[shankari]

Actually, it looks like the metadata time_zone is missing only for documents, not read-write documents. So then why didn't we hit the errors fixed by e-mission/e-mission-server@a31ba77 earlier?

[{u'_id': ObjectId('5796b698115514afcd79339b'),
  u'data': {u'accuracy': -1,
   u'accuracy_threshold': 200,
   u'android_geofence_responsiveness': -1,
   u'filter_distance': 50,
   u'filter_time': -1,
   u'geofence_radius': 100,
   u'ios_use_remote_push_for_sync': 1,
   u'ios_use_visit_notifications_for_detection': 1,
   u'is_duty_cycling': 0,
   u'simulate_user_interaction': 0,
   u'trip_end_stationary_mins': 10},
  u'metadata': {u'key': u'config/sensor_config',
   u'platform': u'ios',
   u'plugin': u'none',
   u'read_ts': 0,
   u'time_zone': u'America/Los_Angeles',
   u'type': u'rw-document',
   u'write_ts': 1469494639.076389},
  u'user_id': UUID('0763de67-f61e-3f5d-90e7-518e69793954')},
 {u'_id': ObjectId('57969b14115514afcd79339a'),
  u'data': {u'accuracy': -1,
   u'accuracy_threshold': 200,
   u'android_geofence_responsiveness': -1,
   u'filter_distance': 30,
   u'filter_time': -1,
   u'geofence_radius': 100,
   u'ios_use_remote_push_for_sync': 1,
   u'ios_use_visit_notifications_for_detection': 1,
   u'is_duty_cycling': 0,
   u'simulate_user_interaction': 0,
   u'trip_end_stationary_mins': 10},
  u'metadata': {u'key': u'config/sensor_config',
   u'platform': u'ios',
   u'plugin': u'none',
   u'read_ts': 0,
   u'time_zone': u'America/Los_Angeles',
   u'type': u'rw-document',
   u'write_ts': 1469482757.695187},
  u'user_id': UUID('0763de67-f61e-3f5d-90e7-518e69793954')}]