Counterpoint comes from two Latin words—contra, meaning "against" or "opposite", and punctus, meaning "point". A punctus was the early ancestor of what we'd call a "note". Placing notes against or in opposition to each other was, in some ways, a visual expression of the idea of melodies that play off each other. Probably the earliest known example we have of counterpoint in this sense was a style that originated around the 11th and 12th century known as organum, where a chant melody formed a static baseline for several more florid melodies that were placed "opposite" each other.
A good example of this is Viderunt Omnes by Perotin. When listening to this, the human ear naturally tends to want to hear the top notes being sung. The rest is secondary. What doesn't come through in recorded performances is the fact that these melodies are constantly switching places, intertwining, popping on top of and dipping below each other. You can see a copy of the score of Viderunt Omnes here. If you follow along with the music, you can see this happening. But you still can't hear it happening. In a recording, like the YouTube link above, it simply sounds like one guy is singing the entire melody, when it's really it's a motley affair, the contribution of several different singers.
In a live performance, things are different. Due to the physical presence of the singers and their distance from each other, you can hear different melodies coming from different places in the room. The result is that you hear X melody coming first from your left, and then, suddenly, the same melody X from your right, while the guy singing on your left is now singing Y melody. And then it switches again. The texture is an enormous—perhaps a fundamental—part of understanding the music. It's something you won't ever get in recorded performance.
This isn't just true of Perotin. It's true of many pre- and post-Reformation composers, including Dufay, Josquin, Palestrina and Byrd, perhaps to a lesser extent, and the Lutherans through Bach. The physical placement of the musicians is totally essential to understanding counterpoint. In many ways, without live performance, we understand the punctus part of counterpoint, but not the contra part.
Contra gives us words in English like "contrarian", "counterclockwise", and "counteract". But contra has some more delicate connotations than simply "against" or "opposite". In Vergil's Aeneid, for instance, he constantly uses the word in his conversations. Sic Venus et Veneris contra sic filius orsus is literally, "Thus spoke Venus, and the son of Venus began opposite." Another denotation is "in turn". The word contra adds a back-and-forth dimension to the conversation. They're speaking contra in the same way you might throw the baseball contra somebody. You're throwing the ball back and forth, opposite each other.
Applying that back to music, the contra of counterpoint is the playful, conversational element of it that can't be expressed through Bose speakers. Your boom box can only give a slight impression of distances between singers and the delineation of lines. A CD can't really express how the singers are interacting and jumping on top of each other and creating notes that are in response to other notes. That's why counterpoint really necessitates live performance. Being present while different melodies come from different parts of the room and hit your ear at different places will only really give you the idea of a conversation between parts.