Function $f$ is defined by $f(x)=-a^x+b$, where $a$ and $b$ are constants. In the $xy$-plane, the graph of $y=f(x)+5$ passes through the point $(0,-\frac{1}{3})$. The sum of $a$ and $b$ is $\frac{10}{3}$. What is the value of $a$?