First, for those who haven't read it yet, you better read the official blog regarding this issue.
Now to answer your questions:
Why do suspended users have a reputation of one?
The reputation is not only reduced to 1, it's also locked at 1 point. This is done, in addition to "auto block" most privileges, as some sort of intimidation - most people care very much about their reputation, almost like money, so taking it away is almost as hurting as banning from asking questions or posting answers. Even if someone will upvote answers such a person made in the past, the reputation won't be gained until the suspension is over.
Even though they have one reputation point, they cannot post questions.
That's right, suspended user is not just user with 1 rep.
there is no use of making "Ask Question" visible. This is an unnecessary server hit. Am I right?
The server get millions of hits every day. There are maybe several dozens of suspended user at any given time - the server can survive this extra hit.
As suspended users do not have the privilege of answering questions, there is no use of making "Answer Question" visible
Might be true, but probably not worth developing - the developers have more important things to do, that affect the whole community. If you want feel free to start Feature Request asking to hide the Answer section (and Ask a Question button) for suspended users.
Also, since "Why not 0?" is a common question, it's explained nicely in answers to this question: Why does reputation have a lower bound of 1?