According to the BOK, it looks like fuse 101 is the blower relay feed.

https://www.crownvic.net/ubbthreads/ubbthreads.php?ubb=showflat&Number=1920429#Post1920429

That would be the first place I would check. I've also found that when blower motors are going bad if you give it a whack it'll start back up and run sometimes, just make sure you do it with it turned on. I'd say the motor itself or the fuse are more likely than the relay.

If you can switch the relay with another one of the same size and (looks like the rear defroster one is the same from that diagram, but I may be wrong), you can rule that out as being your problem.