I do fair amount of streaming off Spotify and Amazon and got lot of music stored on my FAT32 USB drive.Keep in mind that FAT32 is not a robust file system. For example, although it is unlikely you could lose all of the data on the drive if power were to drop out at the wrong instant while the drive is in use, and cause the file table to be corrupted.
Which makes it doubly important to have your music files well backed up, preferably on two additional drives, and with one of them preferably stored at a different location.
Also, if whatever the drive is connected to can work with a more robust file system, such as NTFS which can be used by Windows or an Ext3 or Ext4 file system which can be used by Linux, I would consider reformatting the drive to the more robust file system and then restoring the data to it from backups.
Regards,
-- Al