There is definitely a mathy aspect in programming.

Take regular expressions as an example: they provide a way to interact with finite state machines.
Or the lambda calculus-it was a purely mathematical model and is now the second most important programming model
in the world (after the turing machine).

So there is definitely a mathematical background, however, this background is not needed to understand programming.
You can totally learn regular expression without knowing about the theory, or use haskell without really understanding
the theoretical foundation of the lambda calculus.