[Tiny] Fix scrabble test

[alexhernandezgarcia]

One of the Scrabble tests was disabled in a previous PR because it was failing. This PR fixes it. The reason for the fail was that the state had to be copied because the method step() of the Scrabble env does not copy the state unlike other envs.

[josephdviviano]

LGTM - can you add a comment explaining why copy is required here?

[josephdviviano]

looks great but would appreciate a comment as to why copy is needed for our future selves.

[alexhernandezgarcia]

@josephdviviano copy is needed because of the following:

• The state of the Scrabble is a list of ints.
• This particular test stores a list of states in a trajectory as it samples actions. So a list of lists, called states_trajectory_fw in the code.
• Because the step() method of the environment (which applies the actions) does not copy the state, if the test does not copy the state either, then the states stored in states_trajectory_fw would be the same object. That is, they would be updated as actions are made and we will not have all the states in the trajectory but just the last state repeated multiple times.
• This was not a problem in other environments because step() does copy the state, so we're good.

Does that make sense?

[josephdviviano]

@alexhernandezgarcia it does make sense indeed - I just meant a comment in the test code so people in the future can also understand why there need be a copy there.

[alexhernandezgarcia]

@alexhernandezgarcia it does make sense indeed - I just meant a comment in the test code so people in the future can also understand why there need be a copy there.

Oh, but there are multiple places where copy is used and the explanation is kind of long :/ Should I add a general comment at the beginning of the script?

[alexhernandezgarcia]

Comment about copies added here.

[josephdviviano]

I added some inline comments as well. Thanks!