(I made a thread asking about fan noise, but I actually stopped each case, CPU and GPU fan, and the noisy fan didn't stop (in fact they're all very quiet). So it must be the PSU all along. It's coming from that area.) My PC gets a bit noisy under load now. That or I only just started noticing it. But understandable as games need more as time goes on. The PSU is a 750W Seasonic EVO 80, and I got it 4 years ago. I made sure to get a good PSU, and new, so at the time it was tier 2. (If it wasn't, then it was tier 1!) I'm also using a Zotac 780 AMP! and an overclocked 2500K, so 750W is plenty. So do I just need a new PSU? Or have I only just noticed it being a bit noisy now?
The fan is most likely starting to age more so than the PSU. Clean it out, as VT says, and consider replacing the fan if necessary.
I'm not saying a failing fan isn't possible but it would be highly unlikely in a four year old PSU. Most fans are rated for over 5 years of constant (24x7) use so if it is failing something may have caused it to fail (damage to the blades, clogged with dust, higher/lower than specified temperatures). If i suspected the PSU fan, and I'm in now way recommending anyone does this, I'd use something with insulative properties like some stiff plastic or a thin bit of wood (not a pencil as the 'lead' in the middle is conductive) and stick it through the mesh just far enough to prevent the fan from spinning when i started the PC.
PSU fans can get noisier with age due to mechanical problems. Dust build up is a potential issue too, although if you still get noise after cleaning I'd consider it a mechanical age issue. May also be worth checking there isn't any loose debris rattling around inside it.
TBH I've taken quite a few of my PSUs apart and the fans are nothing special. Just cheap magnetic bearing things. Never had a PSU fan get noisy but have had several fans (Corsair AF being one) start to groan within a year. A PSU to the human ear should be silent. Unless it is whining, in which case I would replace it as the noise is annoying. If the fan is noisy? replace it. You may need to do some soldering though, as most come with a 12v 2 pin fan cable. So you may need to cut off the connector and solder it but YMMV.