You could try reducing the fan speed. That might be okay, if the hardware doesn’t actually need the cooling. If the BIOS has fan curves, go fiddle with that. If it doesn’t, dedicated fan controllers do exist.
If the server is a standard ATX motherboard and if your rack has vertical space, you can also probably get a new ATX case of whatever sort you want, preferably taller, and get something with larger, slower fans and transplant the hardware. A lot of rack servers are vertically-cramped to let a datacenter put as many in a rack as possible, so you get stuff like 1U machines with those dinky 30mm fans. In general, the larger the fan, the less noise per airflow.
searches
https://www.amazon.com/RackChoice-Mini-ITX-Rackmount-Chassis-Standard/dp/B0D296DVD8
I’ve never used that, but it’s a 3U and has three 120mm fans.
If you don’t care about cost, there are also sound-isolated racks. These have some sort of sound-blocking material like plywood on the outside and sound-absorbing foam on the inside. I have been interested in these in the past, because I would like one, but everything I’ve seen has been absolutely obscenely-priced, probably because datacenters don’t care about noise, and few people are running racks in homes or offices. I doubt that the people that sell them get much volume.
EDIT: Example sound-isolated rack:
https://tripplite.eaton.com/smartrack-quiet-server-rack-18u-sound-suppression~SRQ18U








I use a USB DAS JBOD enclosure with fans, which is also an option if you have a mini-PC and just want a bunch of drive space.
https://www.amazon.com/dp/B0DCDDGHMJ
This particular enclosure has physical buttons that lock in place so that the power state is restored on power loss, something that (to my surprise) a number of USB DAS enclosures apparently don’t do.