As the question implies, I am talking about the name of this website. It has evidently been formed from 'Stack' & 'Exchange'. But what does it mean and why was this name chosen ?
Stack Exchange is the short form of "Stack Overflow Knowledge Exchange Platform", which was the original name of the platform.
This is officially confirmed by Joel Spolsky in this comment:
that was a typo on the Fog Creek site; It was actually Stack Overflow Knowledge Exchange shortened to "StackExchange"