Is there any problem with stuffing unfaced fiberglass insulation within the air gap formed between a brick fireplace and its external brick chimney (I.e., where the firplace goes through the wall)?

Thanks in advance.