tl;dr: is there in the GDScript APIs the ability to launch a sub-process and then read its stdout and write to its stdin?

While I'm still doing some newb tutorials and toy projects, I already know I'll soon move to a more sizable scenario I have in mind.

There'll be plenty of simulatings / calculations on world and character states and events and flows and everything that I'm clearly seeing in their totality and concurrency and inter-connectedness shouldn't be in GDScript (big fan of it though! unlike Python or Lua... it hits the spot) or C++ (really don't wanna go there — wouldn't mind C with just structs and funcs, but not ++) — either Go / Golang or Zig would be my cup of tea for this "engine" / backend.

Well for Go or Zig (or even potentially Nim) — since I'll neither whip up a custom GDExtension nor use an existing, likely-WIP 3rd-party/FOSS one (albeit super-neat, over time that kind of stuff is most often playing laggy catch-up, then going unmaintained — I mean at a pace faster than Godot will, ie. a-few-years instead of a-decade-or-two =), I'd have to client-server with my backend code. We're talking same-machine though, inter-process communication.

Even UDP is a bit of overhead compared to the leanness of 2 processes talking bytes to each other via stdin and stdout. Though not applicable/supportable/feasible for non-desktop OS platforms such as web, mobile, consoles (other than Deck I guess =), for "desktop" OSes from Linux to Mac to Win it's always the very same sort of APIs really, and with Godot's (likely-already-existing) libc dependency it's even identical across all targets IIRC. It'd be a great "user API" to have in Godot, like popen or similar in the likes of NodeJS, PHP etc (and really anywhere that has a libc loaded under the hood) and so I searched the docs but nada.

UDP would be the alternative but yeah, for local inter-process bytes -pushing, kinda over the top. (Although at least I'd hope there should be no packet loss from localhost to localhost...)

(Either way, we're not talking about every-frame stuff here, perhaps every-few-seconds stuff. But as we know, "it all adds up" so I like to go as lean&mean as the scenario/requirements allows..)

  • xyz replied to this.

  • meta_leap Sounds like you might want to use godot on a source level, not gdextention but the same way that the godot editor is built. The documentation is far as I understand indeed a bit lacking on that front, but while there are a few users on the forums I think that do use godot this way... mostly you'll probably find users to help you with this on irc/matrix and rocket.chat which would be the venues the engine contributors/developers hand hang out on. In that case however I don't see you avoiding C++.

    Though again, I'm not sure what you are actually trying to do, so as @xyz said, you might want to clarify what you are actually trying to do...

    For your case A, I seem to recall OS class having an execute method. Not sure about listening for the output of the external process tho. Never needed to do something like this myself previously.

      meta_leap is there in the GDScript APIs the ability to launch a sub-process and then read its stdout and write to its stdin?

      What do you want to do with this? Please, without abstract talk, what specifically is your goal?

          xyz What do you want to do with this? Please, without abstract talk, what specifically is your goal?

          First, any use-case for GDExtension (or local-only UDP) is a use-case for inter-process stdio, for where both C++/C# as well as unpredictable/self-made "non-official" GDExtensions are out of the question.

          How about... SQLite =) or, simulating "millions" of world characters (eg. with goroutines and finely-tuned performance-oriented data-structure designs and flows), even though only 1-2 dozen are visibly/audibly in the scene — yet the world and even the visible/audible slice as a whole is potentially mildly affected by "far-away-or-around-the-corner-or-next-village stuff the sim calculated to happen" (with plausible delay also calculated there)... that includes "querying the sim engine about the here-and-now" informed by "the whole".

          Yeah as fantastic as GDScript and Godot's APIs are, there are scales and scenarios I know from experience not to throw at any VM/JIT language... doesn't mean C++ or socket comms are the only other feasible choices out there. I mean, in general =)

          If there's no stdio, I'll see how far UDP takes me. We're talking second-not-millisecond-ticks here anyway. Was just asking if I missed it in the docs and my search skills suck 😀

          • xyz replied to this.

              meta_leap We won't get far if you keep on going all over the place with excessive tech jargoning. If you want suggestions for optimal solutions - describe what you want to make using simple language, as if you were talking to your grandma, and without putting forward any presumed technical solutions.

                I think they are trying to ask if it's possible to spawn another instance of godot runtime as a sibling/sub-process and then call via remote process call..?

                reading again perhaps I misunderstand.

                  Megalomaniak reading again perhaps I misunderstand.

                  Poetry can always be interpreted in multiple ways. Even if it contains words like stdio and JIT 😂

                    Megalomaniak not really, I guess it's a little too technical, but just imagine from inside your Godot-built executable game program running your GDScript, you want to

                    • (A) start a-specific-(terminal-not-windowed)-program (ie. with specific parameters)...
                      • such as some magick tools to process/convert user-provided images (one's own game install logic would ensure it'll be available)
                    • (B) ...and then read its output (technically, a stream-of-bytes, not text or lines of text)
                      • that would usually appear on the terminal if the user had started that program directly, except here, the game program did and so it's the one with access to its output (and input)

                        Sorry then, I wrongly assumed with the Programming tag included, stdio and JIT would be plain-jane normal terms 😀 excuse my ignorance, for I am new here ^_^

                        • xyz replied to this.

                          meta_leap Sounds like you might want to use godot on a source level, not gdextention but the same way that the godot editor is built. The documentation is far as I understand indeed a bit lacking on that front, but while there are a few users on the forums I think that do use godot this way... mostly you'll probably find users to help you with this on irc/matrix and rocket.chat which would be the venues the engine contributors/developers hand hang out on. In that case however I don't see you avoiding C++.

                          Though again, I'm not sure what you are actually trying to do, so as @xyz said, you might want to clarify what you are actually trying to do...

                          For your case A, I seem to recall OS class having an execute method. Not sure about listening for the output of the external process tho. Never needed to do something like this myself previously.

                              meta_leap Sorry then, I wrongly assumed with the Programming tag included, stdio and JIT would be plain-jane normal terms 😀 excuse my ignorance, for I am new here ^_^

                              Salads of smart-sounding words do not clear communication make.

                              But as you wish. You can state clearly what you want to build and get some actually useful suggestions. Or you can continue to write as if you're desperate to prove you're computer-knowledgeable and not get any because nobody can decipher your intent. So please be punctual, succinct and simple. No need to act like a tryhard to prove you're not a noob. We get it 😉

                                Megalomaniak For your case A, I seem to recall OS class having an execute method. Not sure about listening for the output of the external process tho. Never needed to do something like this myself previously

                                That's the class I was looking for, not knowing the name, hooray! Thanks heaps! So I didn't phrase my own searches well enough.

                                So OS is cool... for my use-case it'll be create_process at game start and kill at game end, but since there's no "standard I/O" with that sub-process on offer in this API, I'll still will have to use UDP. But at least I can run-and-stop that 2nd program (in a hypothetical not-on-my-machine future scenario)... will pick UDP over C++ during my hobbyist/leisure-time tinkerings for sure. =)

                                  To be clear tho, if you are asking about making a server, there is already a headless godot server build and of course you can use RPC for UDP connections via GDScript or C#.net as well.

                                  • xyz replied to this.

                                    Megalomaniak From what I've gathered so far I think the OP's intent was to delegate some performance intensive simulation to an external (native or otherwise) process and stream or serve data from that process to Godot app, one-on-one locally. Not full featured server.

                                    They basically wanted to ask if this can be done via named pipe communication. The answer is yes but only through C# Process class. GDScript doesn't have support for pipes. There was a discussion on Godot's github about this but Linietsky said "better to use TCP instead"

                                    So if it must be via a pipe, use C# Process. Otherwise it'll need to be either TCP or UDP socket. I'd go with TCP if communication is not time critical.

                                        xyz From what I've gathered so far I think the OP's intent was to delegate some performance intensive simulation to an external (native or otherwise) process and stream or serve data from that process to Godot app, one-on-one locally. Not full featured server.

                                        To my mind that's still a server tho, it serves a purpose. But sure, localhost rather than networked. Take for an example neverwinter nights games, I don't recall if that applied to the first one, but I know for a fact the second game used the same server to host a local single player game just as the networked multiplayer. Either ways the game actually ran on the server side of it not the client side. It just bundled the server and the client together and if you wanted to play MP you had to host the game yourself but your singleplayer session also ran on that server.

                                        • xyz replied to this.

                                            Megalomaniak I guess it's just a matter of how loose you want to be with terminology 🙂 Sure, you can call something like this a server. I personally wouldn't use that term for a program that can only do one-to-one communication, regardless of if being local or over a network. I'd argue that one-to-many is the defining feature of a server.

                                                xyz Would you consider Godot's "servers" i.e. RenderingServer to be not-servers?

                                                • xyz replied to this.

                                                    award Yes. But again, it's just a matter of semantics and conceptualization. You could for instance say that each node talks to RenderingServer as a client so then by my "definition" we should consider it a server. It's a moot point.

                                                      Yeah better to call it "backend" maybe than server... a rose is a rose by any other name 😎

                                                      OK, for anyone from the future with a similar need searching around, here's how I see this going down:

                                                      • in local-dev / debug builds, godot game UDP-connects to localhost "server"
                                                      • for hypothetical release builds, the "server" prog (and not the godot game binary) is the game-starting prog and it runs the godot game binary, this let's both communicate via standard-I/O pipes, on the Godot side via printraw (print line to stdout aka the "server") and OS.read_string_from_stdin (in sep thread — read next line from stdin aka the "server").

                                                      The line protocol could be no-whitespace-JSON or custom. Will use the former, assuming GDScripts JSON APIs are anyway C++ underneath =)

                                                      The two different modes are of course so I can F5-run with the GDScript debugger/profiler etc.

                                                      Thanks for the lively discussion, good lookin' out 😁

                                                      • xyz replied to this.

                                                        meta_leap Exchanging bytes by printing to console is a very inefficient and troublesome way to do it.

                                                            xyz the bytes are a given either way: they go over sockets or over std pipes. Are you truly telling me Godot's C++-based UDP networking is faster (or less "troubling") than Godot's C-side stdout/stdin pipes which are opened by libc before even main(), have no network checksums etc? I'm hyper-eager to learn more here, I mean it without snark..

                                                            What is it, are there costly string/byte conversions and happening script-side rather than C-side?

                                                            • xyz replied to this.