Many people who use Twitter check their follower numbers very frequently, and respond with displeasure if they see it going down. The fact is, anyone can choose to unfollow you at any time, and they will have their own reasons for doing so. 

Among the major reasons people have for ceasing to follow a Twitter feed is the crime of posting too much. You may have just had a really nice sandwich for lunch, but think before posting about it. Just how good was that sandwich? Sometimes people will only follow those whose tweets are genuinely informative. 

Other people will unfollow you if all you ever do is re-tweet what other people have said. If they wanted to know what someone else was saying, they'd follow them. Excessive re-tweeting looks like spamming to many. 

The same applies to the at-reply. If every one of your tweets begins with "@", then although these will only appear to people who have both parties in their feed, they will begin to feel like they are intruding in a two-way conversation - and the only way to get out of it is unfollowing you. 

People will also have limited patience with Tweets about your social life. If you are constantly tweeting things like "I'm in a club, drinking cocktails! Having a great time!" they will simply wonder why, if you are having such a great time, you keep busting into Twitter rather than going with the flow. Especially if you use Twitpic to post pictures of the cocktails. 

Some of the above, in moderation, is always acceptable and even fun. But if it is all you do, you will get unfollowed.