No, you should not refund. You are in the right.
You stated the shipping costs up front and he essentially agreed to them when he bid. Contract made.
I had the same problem some time ago and found that a polite e-mail clarified the buyer's concerns. I can't remember exactly, but this is roughly what I say in these situations:
"While we try to keep postage costs to a minimum, many buyers are unaware that this includes time to pack the item, the cost of packing material and the time taken to get the package to the post office.
In addition, it can take as long to work out how much to charge for postage as to actualy pack the item- is it air mail? land mail? over a certain weight? over a certain size? does it need insurace? is it fragile? does it comply with import regulations? the list is endless.
Regularly our slight margin (profit) is eaten up by unforseen extra postage costs which we have to absorb since no customer would accept additional postage costs not detailed on the item description page.
The postage cost quoted on the item description is not what Royal Mail charged me to ship the item, but a combination of this and other factors as detailed above. It is the cost
I charge to the customer and is clearly stated on the item description.
To this end I have a sliding scale of average prices which quickly estimates the postage costs. There will inevitably be some packages that I have overestimated the costs on. Likewise there are (many more) that I have underestimated on. We do not make a profit on postage- the costs average out as "break even" over the year. This is why the postage is clearly detailed on the item description and is a true representation of our average costs incurred."
You are being polite but firm. If the postage company charged you 30 dollars instead of the 5 you quoted would your customer happily pay the extra 25? I think not!
They usually give up after that
