Two trains of thought and I am caught in the middle.
Picture your table of three or four:
In Australia it is rude to remove the plate of someone who has finished while the others are still eating, or even one person is still eating. This means the rest sit there and stare at the person who is still eating, making them uncomfortable and rushed.
In America it is rude to leave a finished plate in front of a person who is obv done. Americans do not like dirty plates left on a table, and they might like to move their coffee cup or paper in front of them now.
Just as in Latin countries they will never offer the bill to you or deliver it to you, it is considered presumptious and forward, you are supposed to ask for the bill. In America they get fed up when the bill is not delivered when they feel it should be, when they are ready to go and have to ask for it.
So all that to say - go for your life, whatever you feel is right with your people.
Personally I remove the plates of the parties who are together, at the same time. So you know our layout one big table. I remove the couple over here when they are both done, and leave the other couple over there to continue eating. This may also be a situation where one couple comes down later and eats later, the others should not have to wait for them, they are strangers anyway. Some times our guests are too polite, they will not start or finish without the others, even though they don't know them...I have to have feelers out for this too..