We've seen the apostrophe bug, where some quotes and apostrophes display as URLencoded HTML. It seems that the way to fix them is to edit them and then click "Add" again. Although occasionally after clicking add, the caption field just truncates the whole caption after the first URLencoded character, which sucks.
Also, the photo caption cancel button doesn't work on both IE7 and FF on XP. Anyone else seeing this?