this can happen, if a app on the phone plays a short notification sound (playtime 1 or 2 seconds) , then you will hear nothing, because the timeframe for establishing the bluetooth connection last longer ~4 sec. than the played sound. And you just hear the drop and cutting out of the environmental sounds (from the microphone input) during switchoveer to bluetooth streaming and then switching back to normal HA mode.
So you have to find out which phone app is sending the notification-alert sounds and disable this notification sounds to stop the happening of this ocasionaly “cutting out”.
see also