I think you have it there. The TV is the driving force that keeps the sway going. And the reason that a 5th wheel or trailers using a Hensley or Pullrite hitch pretty much eliminate the problem.
Suppose you pulled a badly loaded but small trailer behind a semi tractor. If the trailer deflected due to wind, the truck would not move and would pull the trailer back into position. But when pulled by a small vehicle, the rear of the TV would be pulled out of position. As it tries to pull the trailer back into position and straighten itself out it could put energy into the system causing the trailer to swing even further in the opposite direction. The rear of the TV also swings even further setting up an oscillation.
I talked to a couple that had a new 30 foot trailer and new tow vehicle and were using a Hensley hitch. The previous year they started off on vacation with a similar set up except using a standard platform hitch. They got about 30 miles from home when the trailer began to sway and before they could react it rolled.