Page 2 of 2

Re: PERFORMANCE IMPROVEMENT: TESTERS NEEDED! (was: UNIX sysadmin observations)

Posted: Tue 28 May 2019 04:10
by gameblabla
It comes to no one's surprise that systemd turns out to be a major CPU hog : this is why i'm sticking with busybox for init and other stuff.
Much saner & lightweight.

You gonna have a bad time if you try to disable journald as systemd assumes it should be ran all the time. Expect weird crashes or freezes.

Also, you'll need udev for joysticks/external controllers and dynamic hotplugging. Unless of course the onboard buttons is just fine.

I hope i can make some of the binaries that comes with RetrOrangePi work on my distro so you don't have to hack it up like this...

Re: PERFORMANCE IMPROVEMENT: TESTERS NEEDED! (was: UNIX sysadmin observations)

Posted: Tue 28 May 2019 06:21
by Atari_Datacenter
You know what?

I don't think we're going to disagree on much. This seems more like a matter of perception and at what angle you approach at the problem. As mentioned: "It is only to test to see if we've identified a performance problem and that we addressed it in a way that improves performance." We've got a lot of maneuvering room in how we solve this.

For testing purposes, yes, we're almost certainly going further than we need to when we shut down acpid, journald, and udevd. So why do it, right? We want to start with a sledgehammer and if that does the trick, we know we're on the right track, and we can work towards an end product that is closer to a flyswatter. If we're on the right track we'll go back, investigate further, and then find the smallest change that'll do the job. But I'm not going to let any of that stop me from responding to all the great issues you raised! :)

Also: please forgive me if my response is a bit remedial. I'm also trying to explain this to others who aren't sysadmins. :)

Agreed on the downside of disabling udev, you'll see that I noted the controller issue. Actually, disabling udev causes far more problems than with the external controllers. Udev changes the permissions on the frame buffer device (which is required for the GUI to work) and it adjusts ownerships and permissions on many other devices which end up being required for retrorangepi to work. After learning what udevd actually does, I managed to script around the framebuffer permission issue and the other issues. When I was trying to work without it, I had only one last issue which prevented Retrorangepi from fully working on its default hardware.(Everything worked great until you actually launched an emulated game. Then none of the buttons worked.)

Anyhow, none of this is to say that the actual solution will be to disable udev. We can trim down many of the other things that it does, or perhaps we might find just one particular rule which drives the CPU consumptionand fix that. When I was exploring the udevd issue, I had something interesting happen on my own retropiorange image. I'm waiting to see if others see the same results as I did. But, you know... spoilers, right?

So as part of my all too quick education on all of this, it was interesting learning how retropiorange connects to Retrostone's configuration for its built-in buttons. It seems that the handler is put into a systemd service called "tz_gpio.service". In some cases, it can launch after the GUI and the user has to wait a bit before their buttons will work. Peronsally, I'd like to take this away from systemd and be put in its own script that is launched in the /etc/rc.local script. Putting it into /etc/inittab would have been perfect. Funny... you mentioned init, didn't you? :)

Regarding journald, I haven't run into that issue yet, but thanks for the heads-up. I'll check into it. We might be able to revert back to syslogd or we might not even need to disable journald. We could leave it running as-is, perform a minor configuration change, or we could strip it down. As a partially-related side-topic, at work I just uncovered how our default syslog configurations allowed it to consume up to 4GB of RAM on a thousand or so of our largest servers. For me, journald's default configuration seems questionable.

You mentioned busybox, right? Frankly, as someone who just left a sysadmin team that administrates thousands of high-end Solaris servers, when I look at the Linux systemd, I see a sea of arbitrary madness. In modern times, all the major UNIX vendors decided that they needed to bolt on some sort of dynamic service manager and nobody standardized with anyone else. For Linux, that ended up being systemd. But (from my perspective) they went too far and rolled too many things into their project. And it seems that they've thrown out some of the long-standing UNIX principles. But then again, Linux is not UNIX. And with the decline of the UNIX vendors, Linux can stand on its own and drive its own path forward and not worry too much about anyone else. (I hope the BSD crowd doesn't lynch me for saying that.)

Replacing systemd with busybox seems labor intensive, but you know what? If you pull it off and you've got something robust enough to share with others, allow me to be among the first to subscribe to your newsletter. Running a handheld game emulator with systemd seems like an excessive solution and (if our testing pans out) a confirmed threat to interactive multimedia performance.

You know what? I was concerned that nobody was still sticking around here. It was great reading your reply, thanks!
- The Atari Datacenter Guy

Re: PERFORMANCE IMPROVEMENT: TESTERS NEEDED! (was: UNIX sysadmin observations)

Posted: Wed 29 May 2019 02:00
by gameblabla
Agreed on the downside of disabling udev, you'll see that I noted the controller issue. Actually, disabling udev causes far more problems than with the external controllers. Udev changes the permissions on the frame buffer device (which is required for the GUI to work) and it adjusts ownerships and permissions on many other devices which end up being required for retrorangepi to work. After learning what udevd actually does, I managed to script around the framebuffer permission issue and the other issues. When I was trying to work without it, I had only one last issue which prevented Retrorangepi from fully working on its default hardware.(Everything worked great until you actually launched an emulated game. Then none of the buttons worked.)
Permissions are a bitch, hence why my cfw is using root as default for now rofl... (even though it's unsafe)
So as part of my all too quick education on all of this, it was interesting learning how retropiorange connects to Retrostone's configuration for its built-in buttons. It seems that the handler is put into a systemd service called "tz_gpio.service". In some cases, it can launch after the GUI and the user has to wait a bit before their buttons will work. Peronsally, I'd like to take this away from systemd and be put in its own script that is launched in the /etc/rc.local script. Putting it into /etc/inittab would have been perfect. Funny... you mentioned init, didn't you? :)
Yup, it basically works like uinput which then creates a virtual gamepad for emulators. I also did something similar in C for the keyboard :
https://github.com/retrostone-dev/remap-input
You mentioned busybox, right? Frankly, as someone who just left a sysadmin team that administrates thousands of high-end Solaris servers, when I look at the Linux systemd, I see a sea of arbitrary madness. In modern times, all the major UNIX vendors decided that they needed to bolt on some sort of dynamic service manager and nobody standardized with anyone else. For Linux, that ended up being systemd. But (from my perspective) they went too far and rolled too many things into their project. And it seems that they've thrown out some of the long-standing UNIX principles. But then again, Linux is not UNIX. And with the decline of the UNIX vendors, Linux can stand on its own and drive its own path forward and not worry too much about anyone else. (I hope the BSD crowd doesn't lynch me for saying that.)
You have absolutely no control on systemd : it's not configurable, it's huge (+1.2 millions LOC) and bloated. I do not understand why people are craving for systemd but i guess they needed to find a reason to throw sysvinit. (which, while not great, was certainly not as bad as systemd became)
Oh and yeah, even in the best case it consumes around 160MB of RAM or so. Mind you that this is simply to boot up to a basic desktop and we haven't done anything yet....
So yeah, this is why i decided to go with buildroot for the CFW as the Opendingux devs did. It's really good, though sadly the glibc port does not bootup successfully so that's kind of a bummer...
It wouldn't be an issue however if the mali drivers were not proprietary or if mainline kernel supported Composite output, which it doesn't.

I'm afraid that you'll only go so far with it. If buildroot is not the way to go then perhaps something like Lakka should be okay for that thing.
I'm not too familiar with Lakka though.

Re: PERFORMANCE IMPROVEMENT: TESTERS NEEDED! (was: UNIX sysadmin observations)

Posted: Thu 13 Jun 2019 11:48
by Admin
Sorry for the late reply.. I'm so focus on retrostone2 development that I don't do many things else (including visiting the forum >_<)

AMT630A datasheet : https://www.8bcraft.com/wp-content/uplo ... c_v1.1.pdf
Hex making instructions (dunno if that can help) : https://www.8bcraft.com/wp-content/uplo ... V1.0.2.doc