You don't have to be a spammer to get reported for spamming. Even totally clean, double opt-in lists will get one or two abuse reports per 50,000 recipients. Sometimes it's just a simple mistake, but getting reported for abuse is pretty serious. If a major ISP receives even a small handful of complaints about your e-mails, it will start blocking all e-mail from your server.