Change FTS3 options that ingestd uses to be able to configurably configure an FTS3 activity and to add checksum checking to the FTS3 transfer

[StevenCTimm]

Recent debugging shows that Ingestd is submitting FTS3 transfers without requesting checking the checksum and with no FTS3 activity specified. We need option to have a activity specified in the ingestd config file which is then added to the FTS3 call,
and also to include the checksum checking in the FTS3 options that are submitted. Same was recently added to DeclaD.

[marcmengel]

Did some research, discoveries so far:

• I'm not sure where we're getting checksums -- the code does try to grab checksums from the job_status once we have submitted the job, but that I think just returns ones we passed in when submitting it; I don't see an initial fetch of a checksum...
• ingestd uses the REST interface to FTS3, and they have an example here of setting the checksum and activity in the REST call, so we need to include those bits around here when creating the request.

[marcmengel]

@StevenCTimm -- do we need one singular activity on an ingestd instance, or do we need some sort of fillename-match to activity mapping?

[marcmengel]

Okay, so we do pull down a metadata file and try to extract checksum and file size
https://github.com/fermitools/ingestd/blob/main/Mover.py#L92
(looks like this needs update for old/new metadata flexibility)
(looks like this leaks a file handle on the metadata file? sigh.)
So then at https://github.com/fermitools/ingestd/blob/main/Mover.py#L117 we could pass the checksum we just sucked out of the metadata into the transfer request call.

[StevenCTimm]

Any one ingestd can be considered to be doing only one Activity in fts3 terms.
If we need different activities we can always stand up a second or third ingestd.

With respect to checksum matching, expect to find some vestiges in the code from the previous FTS-Light era which was doing the xrdcp on its own and grabbing the checksum from that.. but that code isn't active anymore.. the part you found is the good part and it seems you are on the right track.

[marcmengel]

@StevenCTimm -- do you have a test/integration instance where you could try out the #4 changes? It is a fairly small tweak, but I don't have an FTS3 instance for integration testing yet, and it will probably take a little bit to get one setup.

[StevenCTimm]

I had on my list that I should be making a test/integration instance so this is a good time to do it.
IT may take a couple days, there are a couple other higher priority forest fires burning at the moment.