Theodore Casser has been a loyal Bank of America credit card customer for about 10 years. But the prospect that the bank might start charging him an annual fee because he pays off his balance monthly has the Baltimore software developer ready to sever that relationship.

I take it almost as an insult, said Casser, who hasnt heard yet if he will be among the small percentage of unprofitable Bank of America customers to be charged a $29 to $99 fee starting next year. Im happy to take the hit to my credit rating to cancel the card.

Many consumers are mad these days at card issuers for imposing fees, boosting interest rates or raising minimum payments and penalties. And like Casser, some customers are upset enough that they threaten to cancel their cards in retaliation. But can getting even backfire by damaging your credit score?

The short answer is yes, but there are ways to minimize the effect.

A credit score, used by creditors to decide when to extend credit and at what rate, is derived from data on credit reports.

A small part of the score is based on the length of credit history. Canceling a card wont affect that, as many mistakenly believe.

Even if you close a card, the good payment on it remains in your report for 10 or more years, until the credit bureaus remove the information because its deemed too out of date to be relevant, says Craig Watts, a spokesman with FICO, the most widely used credit score.

Instead, the most significant part of canceling a card is the negative effect on your so-called utilization rate. Thats the balance on all your cards compared with the total credit limit. Say, you have three cards with $10,000 credit limit on each and you carry a $10,000 balance on one of them. You are using one-third of your credit limit. But close one of the cards, and suddenly your balance jumps to half your credit limit.

Going from a low utilization rate to a high one can slash 50 to 100 points off your score, says John Ulzheimer, president of consumer education for Credit.com.

There are ways to build that back. Pay off your balance by the same amount as the credit limit on the card youre canceling, so the utilization rate doesnt change. Thats not realistic for many people, he says.

Another option is to open another credit card with the same credit limit as the card being canceled, Ulzheimer says.

This, too, has a negative consequence on your score, but only minor and temporary, he says. When you shop for a new card, the inquiries into your report by creditors may lower your score for a period. Plus, adding a new card will lower the overall age of your credit account history, which also may ding your score, Ulzheimer says.

Opening a new card now may not be a realistic option, either, if you have a poor credit history. Card issuers remain stingy about extending credit except to those with healthy credit scores of 720 and up, Ulzheimer says.

Ulzheimer suggests consumers have seven to 10 credit cards, using each periodically and paying off the balance monthly. This way if a card issuer does something you dont like, you can cancel the card without consequence, he says.

Bank of America, the latest card issuer to stir up consumer ire, says its testing an annual fee on less than 1 percent of its U.S. customers. But Casser, the Baltimore software developer, says he has other credit cards he can use.

But what will he do if all card issuers start charging annual fees? Then its time for me to dust off my checkbook again and pay cash and pay with checks, he says.