Steps:
Expected: You then get a dialog to choose its install folder Actual: ... nothing whatsoever
ThesaurusRex reported something similar on Windows 10, although that is the only report elsewhere so far.
Found the fix, will be in 1.51. The request to open the file dialog was coming from the main thread, not the Swing thread. For whatever reason that never caused an issue for 13 years, but is causing an issue on recent versions of OS X (and maybe elsewhere). Credit to https://stackoverflow.com/a/33601489/3534605 for pointing me in the right direction.
Oddly it didn't happen when launched from NetBeans, and had no logs/error messages... not sure why it had Heisenbug characteristics but it's fixed now.