A painter needs $4$ gallons of a specific color of paint to finish a room. He only has a small measuring cup that holds $\frac{1}{3}$ gallon. How many times must he fill this measuring cup to get the required amount of paint?