r/kernel • • 7d ago

Non-kernel "Platform" APIs on Linux

https://soc.me/interfaces/non-kernel-platform-apis
3 Upvotes

25 comments sorted by

2

u/Thin_Dragonfruit2254 7d ago

Is that all? Seems like a premature end of the writing..

1

u/simon_o 6d ago

I guess the post could add some overview over which "fundamental" functions for Unicode/locale/timezone handling would be required?

1

u/Thin_Dragonfruit2254 6d ago

Could help but mostly your suggestion to the problem, based on your experience.

2

u/hackingdreams 7d ago

That library is called glib. It's why it exists.

1

u/simon_o 6d ago

Not every program is written in C, nor should these operations require to opt into that specific language ecosystem I believe.

3

u/FloweyTheFlower420 6d ago

glibc is the de-facto linux runtime, independent of C

1

u/simon_o 6d ago

It's literally written in C, it's API is defined in C, to make sense of it you need (most of) a C compiler, and it needs C-specific setup in your process to work (like TLS for errno).

3

u/FloweyTheFlower420 6d ago

It's literally written in C

So is the kernel, who cares?

it's API is defined in C

No one cares about glibc's API. The ABI (which is what you care if you link against it) is the standard C ABI, which is basically just SystemV ABI (the standard one)

to make sense of it you need (most of) a C compiler

I can write an assembly program right now that links against glibc. Don't believe me?

and it needs C-specific setup in your process to work (like TLS for errno).

TLS is defined by SysV. Good luck writing software and linking against libraries that don't conform to SysV. If you reject SysV initialization of thread-local storage models, you cannot safely dynamically link against any library that conforms to SysV. Besides, this can be achieved by linking statically against... crt0!

0

u/simon_o 6d ago edited 6d ago

So is the kernel, who cares?

Good you ask! I thought you had understood this, but I'm happy to reiterate:
The kernel could be implemented in anything, as it does not leak language requirements across the kernel/user space divide.

The ABI (which is what you care if you link against it) is the standard C ABI

Which still requires C-specific knowledge to use. As demonstrated by literally any language that mistakenly thought they were ABI compatible by using the LLVM backend.

I can write an assembly program right now that links against glibc. Don't believe me?

Sure. Why wouldn't I?

...

I think "if you aren't interested in C's ecosystem you get to use less of C's ecosystem" is not the slam-dunk argument you think it is.

crt0

The crt0 is often written in assembly, not C. And you can just leave out the steps that are C-specific.

1

u/FloweyTheFlower420 6d ago

The kernel could be implemented in anything, as it does not leak language requirements across the kernel/user space divide.

It leaks an ABI. What calling convention would you propose we use for these kernel provided APIs? Either it's a syscall (terrible idea) so you follow the kernel syscall ABI (modified SysV), or it's a normal calling convention. I wonder what calling convention would be used here? Perhaps the SysV one?

What kind of calling convention do you think vDSO uses? Want to take a guess?

Ironically, you mention this yourself: "The kernel could be implemented in anything" - precisely because it exposes an ABI! And if you wanted it to be callable from userspace, it would use the C calling convention. Good luck convincing the kernel people to use something else.

Which still requires C-specific knowledge to link against. As demonstrated by literally any language that mistakenly thought they were ABI compatible by using the LLVM backend.

You always need an ABI. What kind of nonsense argument is this? The System V ABI works fine. Please tell me how to write a binary level API without proposing some kind of ABI. If you can figure this out, I'll give you a million dollars (hint, you can't by definition).

I think "if you aren't interested in C's ecosystem you get to use less of C's ecosystem" is not the slam-dunk argument you think it is.

Yeah man I'm sure real software developers enjoy reinventing the wheel and not using a standard ABI that literally everyone uses. Just invent your own TLS ABI right? I'm sure everyone is on board with this idea. Are you just ragebaiting or genuinely deficient in your mental facilities?

The crt0 is often written in assembly, not C. And you can just leave out the steps that are C-specific.

I'm aware of how crt0 works. I'm just saying you can trivially get crt0 to initialize glibc for you.

1

u/simon_o 6d ago edited 6d ago

[waffling about ABI]

I have no problem with the ABI itself.
What are you even trying to argue against?

Are you just ragebaiting or genuinely deficient in your mental facilities?

Sorry, I'm going to report this. This is not ok.

[...]

Hey man, it looks like you are just replying for the sake of arguing against something you clearly haven't thought through.

That's not something I'm interested in.

2

u/hackingdreams 6d ago

You're going to have a very hard time getting any "platform API" adopted by literally anyone if it doesn't expose a C-style library interface. The kernel is written in C. You're not getting away from that. And pretending to have a problem with that is just silly. Glib already exists and already smooths over the exact problems you complain about.

It makes your arguments much, much weaker.

1

u/simon_o 6d ago edited 6d ago

You're going to have a very hard time getting any "platform API" adopted by literally anyone

No plans for that. Getting things adopted happens by working for one of the big corpos that sponsor Linux development.

The kernel is written in C. You're not getting away from that.

Which is fine for the reasons I mentioned.

And pretending to have a problem with that is just silly.

Look, I'm not telling you which problems you should have and which not.
Why not offer me the same courtesy?

Glib already exists and already smooths over the exact problems you complain about.

(I assume you mean glibc?) It's part of what I consider the problem, not the solution. Not everyone wants to be a C programmer.

(Not to mention, even if I wanted to use C, there is nothing smooth about glibc, considering its API.)

It makes your arguments much, much weaker.

In which sense? Not wanting to be forced into a specific language ecosystem is kinda the core assumption of the post.

2

u/kolorcuk 7d ago

Hello. I belive this is very bad and horrible. Unicode and timezone is and should not be part of the kernel.

1

u/simon_o 6d ago

What's your reasoning?

2

u/yrro 4d ago

Kernel should be as simple as possible but no simpler. Dates and times and (most) unicode data is not it's concern.

If we could wave a magic want to enforce UTF-8 strings at all boundaries, with whatever is generally regarded as the best form of normalisation, could be an improvement, but it's too late for that now.

Look at the crypto API, after several critical vulnerabilities we've come around to agree that it should not be in the kernel at all.

Same for try handling, all that complexity just results in bugs, better hand it off to user space where it will do less harm when it goes wrong...

1

u/simon_o 4d ago

What's the difference to the time-related functions that the kernel already provides in the vDSO?

1

u/yrro 4d ago

I dunno, do they deal with dates/time zones/leap seconds or just seconds since 1970?

1

u/simon_o 4d ago

Then it might be educational for you to look that up, compare them to the C wrappers and think about the reasons we ended up that way.

1

u/yrro 4d ago

Or you could explain your point because I'm not following you

1

u/kolorcuk 6d ago

Kernel should have hardware abstractions. The article states as if a "mix of programs" is something bad. It is good. I want mix of programs. I want propertiary software not affect my software. I want to be able to run programs in different timezones with different timezone definitions on the same kernel. Forever, if i want to. I want to be able to update timezones and unicode on my 20 year old odroid with kernel 2.6 .

I want to ship my program with broken Unicode alongside propertiary programs with newest unicode and not have to care about it, all running in separate user namespaces.

Kernel should be minimal and insanely stable, forever. Kernel is not the place to share libraries between programs.

The article proposes things that in my opinion are in clear conflict to what kernel represents. The libraries to handle timezones and unicode are there and they should be used, and most importantly freedom to use them should be there. The idea of putting sonething into linux and specifically only linux kernel because there is "mix of code" and "duplicated efforts" is based on wrong grounds that this state is inherently bad. There will always be mix of code and duplicated efforts,and if the stuff will be in kernel it will be yet another mix and duplicated effort.

What about bsd? What about after 20 years i have raspberry pi that has a software that has a bug and requires to run with 20 year old timezone definitions?

Kernel should be minimal.

The solution is standarization and standarizing existing behavior, not adding yet another one to the mix.

Locales timezones unicode - these are just great examples of things defined to be in user space. Additionally "unicode" is not universal, and might never be, and assuming it will be has already been wrong.

What about mars timezones? Why shouldn't my 20 year old odroid not work on mars or the moon because of no interstellar kernel timezones support? Why shouldn't my odroid be able to collate in inexistent locales? I have my doubts the author of the artictle actually has knowledge of the actual ecosystem. It's great windows is mentioned, but windows and linux is not the whole world.

I'll finish with one word that bottom line makes this just unacceptable in the kernel. Security.

1

u/simon_o 6d ago edited 6d ago

I want propertiary software not affect my software.

Why would it?

as if a "mix of programs" is something bad

Not in the post.

I want to be able to run programs in different timezones with different timezone definitions on the same kernel. [...] I want to be able to update timezones and unicode on my 20 year old odroid with kernel 2.6.

Sure, why not? Nothing is preventing you from doing that.
The idea here is to give the 98% of applications that do not care about the specifics an easier, more reliable success path.

Kernel should be minimal.

I mean, which kernel are we talking about? Certainly not Linux then.

these are just great examples of things defined to be in user space

They are in userspace!

What about mars timezones? Why shouldn't my 20 year old odroid not work on mars or the moon because of no interstellar kernel timezones support? Why shouldn't my odroid be able to collate in inexistent locales?

Why would any of that not work? The suggested improvements would read the data from the same place your other libraries would.
Strawman/slippery slope much?

just unacceptable in the kernel. Security

Please explain. What makes the idea different from any other piece of code run in userspace?

1

u/kolorcuk 5d ago

Why would it?

The article states "These resources would be present on the system once and for everyone to use" implies a single point of failure. They shouldn't be present on the system once. Every docker container should have separate, every user namespace should have separate - separate locale, unicode, timezone information.

Not in the post.

The section "Problem" states "Instead, there is an uncontrolled mix of code …", implies that "mix of code" is a problem, it is in that section. Then the section follows with "duplicated efforts, and a continuous need to ship maintenance releases".

That is a good thing. Linux typically had one thing doing one job really well. Kernel should do kernel stuff. If you want, run a service to do timezones stuff.

Sure, why not?

Single point of failure.

They are in userspace!

And should stay there. Instead, if a "accessible and not tied to a language" and "without forcing an application or library author to figure out where or how the data is stored" method is needed, just do a systemd-unicode and systemd-localed service over DBUS. Systemd-timedated literally already exists. I see no reason to involve kernel.

Strawman/slippery slope?

I agree.

Please explain

The "Extending the vDSO with functions that query Unicode/locale/timezone data?" and "Extending the VVAR page" imply a kernel modifications, which is an additional source of bugs and an additional attack vector for kernel escalation or kernel panics.

Overall I do not get it. Unicode and locale is not that important to put it into kernel. It is not something "universal". You can write programs without unicode and without locale and without timezones, you can havegrams with multiple unicode and multiple locales and multiple timezones.

1

u/simon_o 5d ago edited 5d ago

implies a single point of failure.

In that regard, there is no change to status quo. Timezone data is available "system-wide" on the system once right now (optionally, but usually there), worse for locale and Unicode data which is bundled into individual applications more often it seems.

People can still do that if they want, nothing changes for them.

Every docker container should have separate, every user namespace should have separate - separate locale, unicode, timezone information.

Where does this requirement suddenly come from?

If you feel that way, go ahead and change that software to do that, I guess?

That is a good thing.

The disagreement with that is the core assumption of the post.

Single point of failure.

You are not making any sense.

Just do a systemd-unicode and systemd-localed service over DBUS.

That's an interesting approach, but I think that's way more heavy-weight than my approach, having more moving parts, more software involved, taking up more resources.

Systemd-timedated literally already exists

It literally does something completely different from the things proposed.

I see no reason to involve kernel.

Ok, then why do the whole effort here? The article is literally predicated on "could we improve things with some small kernel involvement?".

additional attack vector for kernel escalation or kernel panics

That's making a mountain out of a mole hill. It's not that the kernel adjustments would hit any new code path that was not exercised before.

Overall I do not get it.

Yeah, I have that impression too.

You can write programs without unicode and without locale and without timezones

... and nothing would change for them. You can also write programs without using all available syscalls too, but that doesn't mean we are starting to rip out the unused ones, right?

you can havegrams with multiple unicode and multiple locales and multiple timezones

... and nothing would change for them. Though I'd really like to see some examples for that. Because until now, this sounds like some hypothetical made-up scenario.

1

u/yotsubadesert 4d ago

How exactly is what you propose different then a unicode or zoneinfo library in the same spot on every linux pc? It just seems like the same thing but more brittle and harder to update

Can you find a program on your computer that doesn't use Glibc?