I posted by email an image whose filename contained a single quote. The filename was, as expected, used as the alt text for the image, but the single quote broke the markup because single quotes were used within the IMG tag:
[Error: Irreparable invalid markup ('<img history-bat_26jul04.jpg'>') in entry. Owner must fix manually. Raw contents below.]
<img src='http://pics.livejournal.com/cmshaw/pic/00009xgq/s320x240' alt='nat'l history-bat_26Jul04.jpg' border='0'>
ETA: "nat'l history-bat_26Jul04.jpg" is a perfectly valid filename in Palm OS 5, which is what I'm using to test post by email imanges. My suggestion would be to strip single quotes out of the filenames before using them as alt text.