The IDE way would be to just show the underline by default and show the diagnostics in a floating window, when the mouse (in vim the cursor) hovers over the diagnostic. This behavior could also be a config option!

it’s handy to just hide the inline ones via a mapping

What do you mean by the inline ones?

I really like the format of the diagnostics, however it shifts around all of the lines everytime a new diagnostic is shown. also when scrolling through a window and jumping over a line that is full of diagnostics, the window jumps around because it skips a whole bunch of lines.

I get, that having all diagnostics shown all of the time in a floating window overlaps with the code that you're trying to edit. That being said, do you see an option of making the diagnostics float, so that the visual changes are not shown on their own lines?