No, it matters when using displays in bright areas too. If you want to use outside in the garden on these lovely days higher brightness needed.
One thing I've noticed having to use my 10yr old TV as a monitor, is how difficult it is to see stuff when the sun comes out. Crank the brightness up to compensate and black isn't very black any more and it's still difficult to see, especially sat 6'-7' away.
Will depend on the TV and whether brightness is increasing backlighting or just adjusts the colour tone, you need to up the backlighting for more legibility rather than brightness, these two things whilst linked are often separate menus, and unfortunately strengthening the backlight on the LCD can depending on panel tech make black more grey. Try your TV in demo mode that they use in store with high lighting, might give you the pump up required.
Both of those are firmly in the realms of SDR. Neither will deliver an acceptable HDR image unless they are OLED (and even then it will be marginal) and any marketed 'HDR' support should be considered merely as box-art rather than actual capability. The max brightness will affect the apparent/perceived contrast for the same actual measured contrast. In a perfectly dark room (of sufficient size that wall reflections of the displayed image are attenuated) then the lower brightness panel will give a higher apparent contrast. In a more realistic environment where there is some ambient light (where that ambient light causes reflected ambient light from the panel to be above the brightness level of the dimmest possible grey displayed on the panel), the brighter panel will have the higher apparent contrast. If the two displays are absolutely identical other then brightness then pick the brighter one (as even if you acquire the perfectly dark room, you can just drop the panel light level), otherwise go for the one with the features you need even if it's not as bright.
400 is basically standard, you want much higher for hdr, my oled screen does something like 800 measured for HDR and really that isn't very good compared to my old LCD and in normal content sits at around 300-400, it is just acceptable, great picture even though it is not that bright. It does not burn retinas at 400 don't worry, better to have a screen capable of more and turn it down for most use than not have ability to go any higher.
HDR can be great for games too (as long as they have put some effort into supporting it usefully rather than just stretching an SDR range). The main draw of HDR is not peak brightness (though that is a necessity for decent HDR) nor large colour gamut, it is contrast. Your typical monitor will get ~ 1000:1 for a nice IPS, and 2000:1 or 3000:1 for a VA panel. HDR aims for 1,000,000:1 , with 20,000:1 being the minimum acceptable. To achieve that, you either need a display technology that can simultaneously display two parts of an image with a 10,000x different in brightness (OLED is the only commercial technology that can come close so far), or to augment LCD with a backlight that can output different brightnesses at different portions of the panel. This is 'FALD': Full Array Local Dimming, where the backlight is a grid of a few hundred individually addressable zones that can vary brightness independently (there are else 'edge lit' versions that split the panel into 4/8 chunks, but these are basically worthless in practice). If you backlight can vary in intensity ~1000x, and your LCD pixels ~1000x, you can achieve that 10,000:1 contrast. Gotchas include needing to update that FALD once per display refresh at the same time the LCD updates (or bright objects will have a 'halo lag' as they move around), and having to modify what value the each LCD pixel is set up based on what the backlight bvrightness behind it is set up, even if two pixels are ultimately emitting the exact same value. In a dark room, OLED wins out for visual appeal: even with a lack of peak brightness it's ability to show a massive contrast range for every pixel independently is very visually impressive. As soon as you add some ambient light though, you get a mostly washed out view if the panel is set up correctly (most HDR content will be skewed towards the 'dim' range of the available brightness range, mirroring reality). FALD backlit LCD panels win out here, as while they are unable to display 'pure black' like OLED even a little ambient light obscures that and the higher maximum brightness means a perceptually higher contrast range. In practice, no hardware available today comes close to achieving that 1,000,000:1 target in practice, and is unlikely to in the near term for any acceptable price (stacking a watercooled inorganic-LED panel behind multiple LCD panels may come closest, at the cost of being a massive ballache to use). tl;dr: if it's not OLED and it doesn't have a FALD backlight, it's not HDR, period. Any claims to the contrary on the box are simply straight-up lies.