Jim starts with a positive integer $n$ and creates a sequence of numbers. Each successive number is obtained by subtracting the largest possible integer square less than or equal to the current number until zero is reached. For example, if Jim starts with $n=55$, then his sequence contains $5$ numbers:

Let $N$ be the smallest number for which Jim’s sequence has $7$ numbers. What is the units digit of $N$?