Indexing fails for objects from member node RW

[taojing2002]

RW reported some objects missing solr indexes today.
We took a look and found the metadata objects in those packages don't have solr indexes but the data objects have. In the index_queue table, one metadata object has multiple index tasks with the status of In Process. There is no tasks with the status of New for the metadata object. In the index_processor log, we found the statement of starting index process for the metadata object, but could not find the statement of ending index process.
It is obvious that some silent failures happened and the index tasks were not reset the status to failure.
I also noticed the objects have series ids.

[taojing2002]

I talked with Dave about this bug a little bit. He suggested we can decrease the index processor to single thread and the batch size to one on cn-sandbox for testing. I am going to talk with Chris Turner as well to see if they have somethings we overlooked.