It's probably one of the few things I disagree with him on, although in his defense when he said this we didnt have the extreme issues with voter fraud that exist today. He probably believed that when voters were fed up and had enough of one party not living up to their promises, they'd reward someone with fresh ideas. That would NOT happen today.
It was also one of the few things the Founding Fathers missed on, because they never envisioned people becoming "career politicians" or even wanting to in the first place.