If no one has upvoted my answer for a long period of time, say a month, and the OP didn't accept it, should I delete it if I am sure my solution is working (usually I test the posted code) and I put an effort in it? I regard my profile as a little portfolio, so I don't want to have bad answers in it.
No, you shouldn't. Not receiving any upvotes does not necessarily indicate a bad answer. I personally have some answers which received little attention, but seem to gather some upvotes even a year after the fact. If your answer is correct, all you would be doing is destroy valuable content.
And even if you receive downvotes, the best thing to do is find out where your answer is lacking and improve it. Deleting content, whether it be a question or an answer, should only ever be done in extreme cases. (Note: If you do so too often, it might even contribute to an automatic question or answer ban)
This is my personal experience. I did not register to Stack Overflow for more than a year. I googled my problems and got references to Stack Overflow questions. 30 to 40 % of the answers to those questions, I got my solutions from unaccepted answers. So don't delete your valuable answers. They may (I think must) help others in the present and in the future.
If there are several other upvoted answers, and they essentially contain your solution, you might consider removing yours to reduce the noise. The question is answered anyway.
If your answer is the only one, or contains a different approach, you should probably leave it there. It sometimes just takes a long time for a post to get the attention it deserves.
The answers you give to somenone's question is not just for the person, who asked the question.
The accepted answer might have fitted the questioner's requirements. But there are many other person googling around, whose problem might be slightly different from the questioner's problem. The SO community is for all the seekers who are looking for the answers to certain problems. Who knows your answer might fit that person's problem.
So one must not delete their unaccepted or less upvoted questions. Keep them in your account, by the time they might get some attention. All answers are valuable.