What you want is fans. Let's dispel a newsletter myth. The myth is that you want to keep as many subscribers as you can. That's simply not true. You pay for subscribers. If someone is not reading or forwarding your newsletter, why do you want to send it to them? And you don't really want to annoy people.
Have a simple escape route:
Make it easy to unsubscribe.
Tell people it's easy to unsubscribe.
Tell them how to do it.
It's not personal! Just because someone unsubscribes from your newsletter, it doesn't mean they have unsubscribed from you. And, if people know it's easy to unsubscribe, maybe they'll stick around a little longer... ?