Trickster Rabbit agrees with Foolish Fox to double Fox's money every time Fox crosses the bridge by Rabbit's house, as long as Fox pays $32$ coins in toll to Rabbit after each crossing. The payment is made after the doubling. Fox is excited about his good fortune until he discovers that all his money is gone after crossing the bridge four times. How many coins did Fox have at the beginning?