Casual Male profit falls 45% on higher tax rate