See https://lobste.rs/s/6bphbw/fennel_programming_language_rationale . We have some of this but it's mostly in the form of loose notes in a couple different places. The basic idea is "be the C to Rust's C++", but "how" is going to need a lot of work.
This can probably be put off until we can actually add features again though.
https://www.youtube.com/watch?v=bSkpMdDe4g4 5:10
Above all, do no harm.
Okay, looking at the various links, there's a structure that tends to get followed. It goes:
- Introduction
- Why X?
- Why not Y or Z?
- Talk about various talking points in following sections
Here's an incomplete attempt:
Rust is a revolutionary language because it combines the speed of low-level native code with almost complete memory safety, in a modern and well-engineered package. It allows low-level memory manipulation with no runtime assistance, and enforces compile-time safety on allocation, deallocation and mutation of memory. Unlike previous semi-experimental languages like Cyclone, it also manages to make it relatively pleasant to do this as well, without the type system and borrow checker imposing incredibly restrictive constraints. In addition to this Rust makes a large number of extremely good design decisions that toss away a lot of the cruft of older languages and carries on the best ideas from a variety of sources, and the Rust community puts a lot of work into programmer convenience for both the language and the tools around it.
Rust has one glaring weakness: It is big. It is a large, complicated language with many moving parts, the sort where you can use it and study it for years and still not really know all the details. This has several follow-on effects: It has a steep learning curve, with many small bits of finesse that are nice once you learn them, but need to be learned. It is difficult to implement, with only one other compiler besides the official one, which does not implement the borrow checker or keep up to date with the latest language. It is still evolving, albeit at a fairly controlled pace as of 2021, which makes it difficult for documentation and specifications to stay up to date. And, of course, the compiler runs relatively slowly, because it has a lot of work to do.
So, what happens if we make a language using similar techniques to pursue safety, concurrency, and speed, but try to limit its size to no more than what is required? Garnet intends to pursue a different set of trade-offs from Rust, to make a language that is safe, fast and low-level, but also lightweight, simple to use and learn, and easy to implement. To do this, Garnet is a language designed with some key differences in priorities:
- Speed of compilation. Rust's design is always to throw better compiler technology at problems, and if that makes the compiler do a lot of extra work that's okay. What if we tried to reduce the amount of work the compiler had to do to generate fast code?
- Size. Rust is a big language, and is continually (albeit slowly) getting bigger. It would be nice to have a small language which is mostly finished after a certain point, and where an individual or small team could feasibly implement their own basic compiler from scratch.
- Interoperability. Rust does not use a stable ABI, so writing libraries in Rust that can be used from other languages is tricky. It is entirely possible, by writing a wrapper using the C ABI, but it takes extra work and you can't use a number of nice features. This makes it a bit less appealing to write fundamental libraries in Rust than in other languages, since it makes it harder for other languages such as Python or Go to interface with them.
To do this, tradeoffs must be made. Design is the art of choosing a good set of tradeoffs after all. Tradeoffs that Garnet is willing to make which Rust is not are:
- Runtime performance. A large part of Rust's selling point is "zero cost abstractions", ie, writing things in the most natural way should also result in the best code possible. Garnet definitely aims to be the same "incredibly fast by default" type of language as Rust and C, but sacrificing a smidge of runtime performance the common cases is acceptable. (For now.)
- Ergonomics. Rust does a lot of stuff under the hood, and still makes it easy for a user to deal with in everyday life. By not adding layers of niceness on top of the complexity, Garnet may end up more verbose or "clunkier" than Rust. It should still be a nice, easy language to program in, but it may take a bit more work to tell it how to do something nuanced and you may not be able to pack quite as much functionality into so few lines/steps.
Why not...
- Go -- Go is not a system programming language, it is an application programming language. It is GC'd, it has a runtime, and is made for efficient high-level programming rather than low-level programming. Go is not designed for writing things like operating systems or thread executors.
- D -- I first met D in the early 2000's, so I've seen it off and on for a long time, and the results are just one of those historical tragedies. D is a perfectly respectable language, but in terms of design, feature set and marketing it did the wrong things at the wrong time to gain popularity, and now probably never will. It was always a good evolution of C++, but never quite managed to be a great enough evolution of C++ to convince large numbers of people to leave C++ behind.
- C -- By 2021 we can make a language far, far better than C, and there really shouldn't be much reason not to. I have a lot of respect for C, but it is covered with warts and misfeatures. I would rather write entirely unsafe Rust than C, all day every day.
- Zig -- Zig is a very nice language that has a lot of the same goals as Garnet; it just doesn't have a borrow checker. To misquote the creator(?), this means it can prevent space-related memory safety problems such as buffer overruns, but not all time-related memory safety problems such as dangling pointers. Need to investigate more, but I'm kind of holding off on it on purpose to see how far I can go with the Rust-centric concept before trying to incorporate new ideas.
- Swift -- Need to investigate more. Looks kinda "C#, but made by Apple", though I know of a few really fancy things it does that are worth consideration.
How to accomplish these goals:
- No (less aggressive?) monomorphization
- Known ABI
- More orthogonal -- TODO, what does this mean and how does it help?
Other small languages to contemplate: C, Lua, Scheme. What do these have in common? There's many implementations of them with different characteristics. They can show up tucked away in odd places, like microcontrollers and embedded languages. No matter what system you are using, it will have a C compiler, and if it has a C compiler then it will almost certainly also be able to run Lua and at least one Scheme. (Forth is similar, but it doesn't play with other languages as well.)
Notes on porting and multiple compilers: See things like https://people.gnome.org/~federico/blog/librsvg-rust-and-non-mainstream-architectures.html . Portability, multiple implementations etc make this problem easier.
To quote This Week In Rust, "Rust is a systems language pursuing the trifecta: safety, concurrency, and speed." Garnet is a systems language pursuing safety, simplicity and portability/interoperability.
Also note that C is smallish, but definitely not simple or easy, either to implement or to write. Well, maybe easy to implement, 'cause you have so much wiggle room about how to do things.
Other reference points: https://drewdevault.com/2021/05/24/io_uring-finger-server.html
V and Jai are also worth at least thinking about; I don't consider them valid but other people are inevitably going to ask about them.
Stuff I wrote that just needs to be incorporated here:
One of the wisest things that anyone has ever said to me about software engineering is "you should use the right tool for the right job". So if I'm creating a new tool, what kind of job am I making it for, and how will it be better than existing tools in that area? Currently for me, this translates to "what does Rust not do well?" The main list that I have come up with is:
- Portable. Rust can replace C++ and be a strict improvement. Rust will probably not replace C itself any time soon because it is difficult to implement, difficult to retarget, and difficult for other languages to interoperate with.
- Quick to compile. This is not a priority at the cost of everything else, but it should avoid making design and implementation decisions that require the compiler to do huge amounts of work.
- Finished. Garnet as a language should reach an good point and then be essentially complete, apart from minor fixes and improvements. Avoid open-ended research problems.
You get nothing for nothing. Just as important as goals are anti-goals. These are the negative space in your problem domain that you are not going to try filling in. Knowing what problems are not important for you has a huge influence on your design: Python is not designed for high performance, Rust is not designed for runtime type reflection, and APL is not designed for text processing. Because software is so flexible, of course you can turn these tools to these purposes, the same way you can use a shovel to break down a wall, but you're going to have a harder time of it. Garnet's explicit anti-goals are:
Runtime performance uber alles. Garnet should be able to compile to fast, efficient code that comes close to the performance of the fastest Rust/C/C++ out there, but sacrificing a smidge of runtime performance to make life much simpler is a tradeoff that is worth making as long as it doesn't go too far.
Ergonomics. A lot of Rust's complexity and opacity comes from the fact that it does a lot of inference and special casing of how things behave; my classic example are the auto-deref rules, which are very convenient but also produce lots of hidden behavior such as
&*foobeing a cast. I would rather have a slightly more verbose or superficially complicated system, if I can make it clearer.Small and easy to implement. Favor a small number of abstractions that fit together well over mandating completeness in every situation.
Defined ABI. This is a bit of a tricky goal rather than something where you can point at an existing solution and say "we should use that everywhere", but we can do better than the C model of ABI by now and it would be useful to have a language that tried to do so.
(Yes, I am aware of Zig. I want to try for something different.)
I don't want algebraic effects, dependent types, higher-kinded types, or anything else that are essentially unsolved problems. Useful subsets of these things may be okay.
I really kinda want Garnet to be the Lua of low level languages.
Notes on C's problems: https://thephd.dev/your-c-compiler-and-standard-library-will-not-help-you
An interesting "high-level" discussion about Rust and C++ with Steve Klabnik and Herb Sutter, including some "what if we had a time machine" pondering as well:
https://softwareengineeringdaily.com/2024/10/23/rust-vs-c-with-steve-klabnik-herb-sutter/ (via)
a transcript (with some small issues): http://softwareengineeringdaily.com/wp-content/uploads/2024/10/SED1756-Rust-vs-Cpp.txt (via)
Oooh, neat. Thanks!