A lot of my operating system ideas aren’t focused around the design of the kernel itself, but rather the design of the userspace and user interface.
One particular idea I’ve been thinking about is how we handle command-line interfaces. Currently, commandline interfaces live in the terminal, an imitation of the ancient VT100 dumbterminals that would plug into a mainframe. To control a VT100, you would use escape codes to move around the cursor and change colours and formatting. This design is frankly awful. First of all, it’s bound to the idea of characters, and is thusly very alphabetcentric - not every language has a distinct idea of characters. The terminal, to support every language, first must force those languages into a monospace, separated form, and then it has to be hyperaware of the size of every single character to render them correctly. Needless to say, this causes issues! Fun! Yay!
For instance, on some terminals with Unicode support, like Urxvt, the terminal does exactly this. However, if you install a custom icon font, the font size detection goes completely hogwild, and urxvt cannot render those characters whatsoever. Sometimes, it fails to render normal unicode characters, too. I’ve had this happen on other terminals, but Urxvt is the most notable because THE PATCHES DONT FUCKING WORK.
So, onto the issues with the terminal as a model of UI itself, unicode handling aside. Interfacing with the terminal is just horrible, both OS-side and user side. Ncurses and equivalent libraries are just… messy. Colour handling is damn impossible to get right, because so many programs use literal colour-codes rather than the terminal colour scheme. SSH can be incredibly laggy because it is literally sending every escape code at once rather than doing something sensible. Terminals are shit at nesting - many programs reinvent terminal window managers, and any internal terminal has to reimplement the entire terminal from scratch! See vim’s vterm, the entirety of ssh, etc. Termcap is a mess. Stderr and stdout are really hard to handle in a way that’s consistent. Every shell program has to be pipe-aware if it wants to look reasonably pleasant and include colours and formatting.
I won’t go much more into why the terminal sucks, but know that the implementation of a terminal that is commonly regarded to be the most simple is about 21k lines of C, and it doesn’t include scrolling, barely includes unicode support, or actual features. Scrolling is broken in most terminals, actually - I’ve not found a single terminal that handles my nano correctly. Also, the way special characters get inputted is frankly… terrifying? I’ve also not found a single terminal or settings combination that correctly recognises the difference between Ctrl-Backspace, Backspace and Ctrl-H. More in depth critiques of the terminal are available over at the Arcan (UI focused) website.
SO, where does this leave us? I think we need a new ‘model’ for Command Line Interfaces that goes beyond the frankly unfixable design of VT100s and Xterms. And, while we ditch the old design and model, why not try and make things more user accessible? What’s something almost every person in developed countries uses on an almost daily basis? Chat apps! SMS, Discord, Whatsapp, Telegram, etc. A fun little trend in some of these chat apps is the use of assistant bots. You use them when moderating, you use them to play music, install stickers, etc. Isn’t that a commandline interface? Why not base a new ‘terminal’ off these chat bots which any Discord user picks up use of in minutes?
So, my big revolutionary idea for fixing the VT100 is to replace it with a (of course custom) chat client! Hear me out. So, a big issue terminals have is separation - all input and output and formatting goes into the same buffer, which is text only. There’s no stdout, stdin, stderr separation. Well, a chat app has no problems with that! Every input is distinct, because theyre all in their own messages. Similarly, every output is also distinct - you can seperate out stderr into it’s own field, you could display them together, you can do whatever! Chatapps tend to be capable of displaying fancy formatting, too, so why not give that ability to the shell programs? You could have tables, markdown, images, etc. Another big thing that chat apps have is the ability to reference messages. You can reference a message by replying to it, which adds a whole new world of programmability and userinterface design.
Let me walk you through a hypothetical workflow here, with editing images. You ever used imagemagick? It’s a right pain if you’re wanting to constantly do small adjustments. You can edit the image in place, you can come up with lots of tiny variations on a filename, and you need to have an image viewer and a manual open at all times, because who the fuck knows how to use imagemagick?
What if we added the ability to attach images? Now, your constant edits and tweaks can be ephemeral! You attach an image into the chat with the imagemagick command, and type in a colourshift command. The command replies, and you get the image with the colourshift done. Now, get this. You reply to the original message that has the unedited image, with another, different command. The bot replies, with the corresponding image attached! Both of them coexist at once, and you haven’t had to fumble around with renaming files, moving around image parameters. Want to keep applying edits to an image? Just keep replying! Since the image is displayed inline, there’s no issues with opening a gallery. What’s more, is imagemagick could provide inline help, too - just a help command away! (I’m going to let you into a secret here; I’m pretty sure this actually exists as a discord bot, with the reply thing I described and everything.)
Call me crazy, but I think this is both more useful, cleaner, and more approachable than a traditional shell+VT100 combo. It does require some rethink of the way we handle terminal programs on a fundamental level, however. I think that that is very possible to do, and where it’s impossible, it would be easy enough to just fall back to the standard model with a quick and easy adapter (of course, you wouldn’t be running Vim in there.)