Mercurial

comparison third_party/libuv/docs/src/guide/processes.rst @ 160:948de3f54cea

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
[ThirdParty] Added libuv
author June Park <parkjune1995@gmail.com>
date Wed, 14 Jan 2026 19:39:52 -0800
parents
children
comparison
equal deleted inserted replaced
159:05cf9467a1c3 160:948de3f54cea
1 Processes
2 =========
3
4 libuv offers considerable child process management, abstracting the platform
5 differences and allowing communication with the child process using streams or
6 named pipes.
7
8 A common idiom in Unix is for every process to do one thing and do it well. In
9 such a case, a process often uses multiple child processes to achieve tasks
10 (similar to using pipes in shells). A multi-process model with messages
11 may also be easier to reason about compared to one with threads and shared
12 memory.
13
14 A common refrain against event-based programs is that they cannot take
15 advantage of multiple cores in modern computers. In a multi-threaded program
16 the kernel can perform scheduling and assign different threads to different
17 cores, improving performance. But an event loop has only one thread. The
18 workaround can be to launch multiple processes instead, with each process
19 running an event loop, and each process getting assigned to a separate CPU
20 core.
21
22 Spawning child processes
23 ------------------------
24
25 The simplest case is when you simply want to launch a process and know when it
26 exits. This is achieved using ``uv_spawn``.
27
28 .. rubric:: spawn/main.c
29 .. literalinclude:: ../../code/spawn/main.c
30 :language: c
31 :linenos:
32 :lines: 6-8,15-
33 :emphasize-lines: 11,13-17
34
35 .. NOTE::
36
37 ``options`` is implicitly initialized with zeros since it is a global
38 variable. If you change ``options`` to a local variable, remember to
39 initialize it to null out all unused fields::
40
41 uv_process_options_t options = {0};
42
43 The ``uv_process_t`` struct only acts as the handle, all options are set via
44 ``uv_process_options_t``. To simply launch a process, you need to set only the
45 ``file`` and ``args`` fields. ``file`` is the program to execute. Since
46 ``uv_spawn`` uses :man:`execvp(3)` internally, there is no need to supply the full
47 path. Finally as per underlying conventions, **the arguments array has to be
48 one larger than the number of arguments, with the last element being NULL**.
49
50 After the call to ``uv_spawn``, ``uv_process_t.pid`` will contain the process
51 ID of the child process.
52
53 The exit callback will be invoked with the *exit status* and the type of *signal*
54 which caused the exit.
55
56 Note that it is important **not** to call ``uv_close`` before the exit callback.
57
58 .. rubric:: spawn/main.c
59 .. literalinclude:: ../../code/spawn/main.c
60 :language: c
61 :linenos:
62 :lines: 9-12
63 :emphasize-lines: 3
64
65 It is **required** to close the process watcher after the process exits.
66
67 Changing process parameters
68 ---------------------------
69
70 Before the child process is launched you can control the execution environment
71 using fields in ``uv_process_options_t``.
72
73 Change execution directory
74 ++++++++++++++++++++++++++
75
76 Set ``uv_process_options_t.cwd`` to the corresponding directory.
77
78 Set environment variables
79 +++++++++++++++++++++++++
80
81 ``uv_process_options_t.env`` is a null-terminated array of strings, each of the
82 form ``VAR=VALUE`` used to set up the environment variables for the process. Set
83 this to ``NULL`` to inherit the environment from the parent (this) process.
84
85 Option flags
86 ++++++++++++
87
88 Setting ``uv_process_options_t.flags`` to a bitwise OR of the following flags,
89 modifies the child process behaviour:
90
91 * ``UV_PROCESS_SETUID`` - sets the child's execution user ID to ``uv_process_options_t.uid``.
92 * ``UV_PROCESS_SETGID`` - sets the child's execution group ID to ``uv_process_options_t.gid``.
93
94 Changing the UID/GID is only supported on Unix, ``uv_spawn`` will fail on
95 Windows with ``UV_ENOTSUP``.
96
97 * ``UV_PROCESS_WINDOWS_VERBATIM_ARGUMENTS`` - No quoting or escaping of
98 ``uv_process_options_t.args`` is done on Windows. Ignored on Unix.
99 * ``UV_PROCESS_DETACHED`` - Starts the child process in a new session, which
100 will keep running after the parent process exits. See example below.
101
102 Detaching processes
103 -------------------
104
105 Passing the flag ``UV_PROCESS_DETACHED`` can be used to launch daemons, or
106 child processes which are independent of the parent so that the parent exiting
107 does not affect it.
108
109 .. rubric:: detach/main.c
110 .. literalinclude:: ../../code/detach/main.c
111 :language: c
112 :linenos:
113 :lines: 9-30
114 :emphasize-lines: 12,19
115
116 Just remember that the handle is still monitoring the child, so your program
117 won't exit. Use ``uv_unref()`` if you want to be more *fire-and-forget*.
118
119 Sending signals to processes
120 ----------------------------
121
122 libuv wraps the standard ``kill(2)`` system call on Unix and implements one
123 with similar semantics on Windows, with *one caveat*: all of ``SIGTERM``,
124 ``SIGINT`` and ``SIGKILL``, lead to termination of the process. The signature
125 of ``uv_kill`` is::
126
127 uv_err_t uv_kill(int pid, int signum);
128
129 For processes started using libuv, you may use ``uv_process_kill`` instead,
130 which accepts the ``uv_process_t`` watcher as the first argument, rather than
131 the pid. In this case, **remember to call** ``uv_close`` on the watcher _after_
132 the exit callback has been called.
133
134 Signals
135 -------
136
137 libuv provides wrappers around Unix signals with `some Windows support
138 <http://docs.libuv.org/en/v1.x/signal.html#signal>`_ as well.
139
140 Use ``uv_signal_init()`` to initialize
141 a handle and associate it with a loop. To listen for particular signals on
142 that handler, use ``uv_signal_start()`` with the handler function. Each handler
143 can only be associated with one signal number, with subsequent calls to
144 ``uv_signal_start()`` overwriting earlier associations. Use ``uv_signal_stop()`` to
145 stop watching. Here is a small example demonstrating the various possibilities:
146
147 .. rubric:: signal/main.c
148 .. literalinclude:: ../../code/signal/main.c
149 :language: c
150 :linenos:
151 :emphasize-lines: 17-18,27-28
152
153 .. NOTE::
154
155 ``uv_run(loop, UV_RUN_NOWAIT)`` is similar to ``uv_run(loop, UV_RUN_ONCE)``
156 in that it will process only one event. UV_RUN_ONCE blocks if there are no
157 pending events, while UV_RUN_NOWAIT will return immediately. We use NOWAIT
158 so that one of the loops isn't starved because the other one has no pending
159 activity.
160
161 Send ``SIGUSR1`` to the process, and you'll find the handler being invoked
162 4 times, one for each ``uv_signal_t``. The handler just stops each handle,
163 so that the program exits. This sort of dispatch to all handlers is very
164 useful. A server using multiple event loops could ensure that all data was
165 safely saved before termination, simply by every loop adding a watcher for
166 ``SIGINT``.
167
168 Child Process I/O
169 -----------------
170
171 A normal, newly spawned process has its own set of file descriptors, with 0,
172 1 and 2 being ``stdin``, ``stdout`` and ``stderr`` respectively. Sometimes you
173 may want to share file descriptors with the child. For example, perhaps your
174 applications launches a sub-command and you want any errors to go in the log
175 file, but ignore ``stdout``. For this you'd like to have ``stderr`` of the
176 child be the same as the stderr of the parent. In this case, libuv supports
177 *inheriting* file descriptors. In this sample, we invoke the test program,
178 which is:
179
180 .. rubric:: proc-streams/test.c
181 .. literalinclude:: ../../code/proc-streams/test.c
182 :language: c
183
184 The actual program ``proc-streams`` runs this while sharing only ``stderr``.
185 The file descriptors of the child process are set using the ``stdio`` field in
186 ``uv_process_options_t``. First set the ``stdio_count`` field to the number of
187 file descriptors being set. ``uv_process_options_t.stdio`` is an array of
188 ``uv_stdio_container_t``, which is:
189
190 .. code-block:: c
191
192 typedef struct uv_stdio_container_s {
193 uv_stdio_flags flags;
194
195 union {
196 uv_stream_t* stream;
197 int fd;
198 } data;
199 } uv_stdio_container_t;
200
201 where flags can have several values. Use ``UV_IGNORE`` if it isn't going to be
202 used. If the first three ``stdio`` fields are marked as ``UV_IGNORE`` they'll
203 redirect to ``/dev/null``.
204
205 Since we want to pass on an existing descriptor, we'll use ``UV_INHERIT_FD``.
206 Then we set the ``fd`` to ``stderr``.
207
208 .. rubric:: proc-streams/main.c
209 .. literalinclude:: ../../code/proc-streams/main.c
210 :language: c
211 :linenos:
212 :lines: 15-17,27-
213 :emphasize-lines: 6,10,11,12
214
215 If you run ``proc-stream`` you'll see that only the line "This is stderr" will
216 be displayed. Try marking ``stdout`` as being inherited and see the output.
217
218 It is dead simple to apply this redirection to streams. By setting ``flags``
219 to ``UV_INHERIT_STREAM`` and setting ``data.stream`` to the stream in the
220 parent process, the child process can treat that stream as standard I/O. This
221 can be used to implement something like CGI_.
222
223 .. _CGI: https://en.wikipedia.org/wiki/Common_Gateway_Interface
224
225 A sample CGI script/executable is:
226
227 .. rubric:: cgi/tick.c
228 .. literalinclude:: ../../code/cgi/tick.c
229 :language: c
230
231 The CGI server combines the concepts from this chapter and :doc:`networking` so
232 that every client is sent ten ticks after which that connection is closed.
233
234 .. rubric:: cgi/main.c
235 .. literalinclude:: ../../code/cgi/main.c
236 :language: c
237 :linenos:
238 :lines: 49-63
239 :emphasize-lines: 10
240
241 Here we simply accept the TCP connection and pass on the socket (*stream*) to
242 ``invoke_cgi_script``.
243
244 .. rubric:: cgi/main.c
245 .. literalinclude:: ../../code/cgi/main.c
246 :language: c
247 :linenos:
248 :lines: 16, 25-45
249 :emphasize-lines: 8-9,18,20
250
251 The ``stdout`` of the CGI script is set to the socket so that whatever our tick
252 script prints, gets sent to the client. By using processes, we can offload the
253 read/write buffering to the operating system, so in terms of convenience this
254 is great. Just be warned that creating processes is a costly task.
255
256 .. _pipes:
257
258 Parent-child IPC
259 ----------------
260
261 A parent and child can have one or two way communication over a pipe created by
262 settings ``uv_stdio_container_t.flags`` to a bit-wise combination of
263 ``UV_CREATE_PIPE`` and ``UV_READABLE_PIPE`` or ``UV_WRITABLE_PIPE``. The
264 read/write flag is from the perspective of the child process. In this case,
265 the ``uv_stream_t* stream`` field must be set to point to an initialized,
266 unopened ``uv_pipe_t`` instance.
267
268 New stdio Pipes
269 +++++++++++++++
270
271 The ``uv_pipe_t`` structure represents more than just `pipe(7)`_ (or ``|``),
272 but supports any streaming file-like objects. On Windows, the only object of
273 that description is the `Named Pipe`_. On Unix, this could be any of `Unix
274 Domain Socket`_, or derived from `mkfifo(1)`_, or it could actually be a
275 `pipe(7)`_. When ``uv_spawn`` initializes a ``uv_pipe_t`` due to the
276 `UV_CREATE_PIPE` flag, it opts for creating a `socketpair(2)`_.
277
278 This is intended for the purpose of allowing multiple libuv processes to
279 communicate with IPC. This is discussed below.
280
281 .. _pipe(7): https://man7.org/linux/man-pages/man7/pipe.7.html
282 .. _mkfifo(1): https://man7.org/linux/man-pages/man1/mkfifo.1.html
283 .. _socketpair(2): https://man7.org/linux/man-pages/man2/socketpair.2.html
284 .. _Unix Domain Socket: https://man7.org/linux/man-pages/man7/unix.7.html
285 .. _Named Pipe: https://docs.microsoft.com/en-us/windows/win32/ipc/named-pipes
286
287
288 Arbitrary process IPC
289 +++++++++++++++++++++
290
291 Since domain sockets [#]_ can have a well known name and a location in the
292 file-system they can be used for IPC between unrelated processes. The D-BUS_
293 system used by open source desktop environments uses domain sockets for event
294 notification. Various applications can then react when a contact comes online
295 or new hardware is detected. The MySQL server also runs a domain socket on
296 which clients can interact with it.
297
298 .. _D-BUS: https://www.freedesktop.org/wiki/Software/dbus
299
300 When using domain sockets, a client-server pattern is usually followed with the
301 creator/owner of the socket acting as the server. After the initial setup,
302 messaging is no different from TCP, so we'll re-use the echo server example.
303
304 .. rubric:: pipe-echo-server/main.c
305 .. literalinclude:: ../../code/pipe-echo-server/main.c
306 :language: c
307 :linenos:
308 :lines: 70-
309 :emphasize-lines: 5,10,14
310
311 We name the socket ``echo.sock`` which means it will be created in the local
312 directory. This socket now behaves no different from TCP sockets as far as
313 the stream API is concerned. You can test this server using `socat`_::
314
315 $ socat - /path/to/socket
316
317 A client which wants to connect to a domain socket will use::
318
319 void uv_pipe_connect(uv_connect_t *req, uv_pipe_t *handle, const char *name, uv_connect_cb cb);
320
321 where ``name`` will be ``echo.sock`` or similar. On Unix systems, ``name`` must
322 point to a valid file (e.g. ``/tmp/echo.sock``). On Windows, ``name`` follows a
323 ``\\?\pipe\echo.sock`` format.
324
325 .. _socat: http://www.dest-unreach.org/socat/
326
327 Sending file descriptors over pipes
328 +++++++++++++++++++++++++++++++++++
329
330 The cool thing about domain sockets is that file descriptors can be exchanged
331 between processes by sending them over a domain socket. This allows processes
332 to hand off their I/O to other processes. Applications include load-balancing
333 servers, worker processes and other ways to make optimum use of CPU. libuv only
334 supports sending **TCP sockets or other pipes** over pipes for now.
335
336 To demonstrate, we will look at an echo server implementation that hands off
337 clients to worker processes in a round-robin fashion. This program is a bit
338 involved, and while only snippets are included in the book, it is recommended
339 to read the full code to really understand it.
340
341 The worker process is quite simple, since the file-descriptor is handed over to
342 it by the master.
343
344 .. rubric:: multi-echo-server/worker.c
345 .. literalinclude:: ../../code/multi-echo-server/worker.c
346 :language: c
347 :linenos:
348 :lines: 7-9,81-
349 :emphasize-lines: 6-8
350
351 ``queue`` is the pipe connected to the master process on the other end, along
352 which new file descriptors get sent. It is important to set the ``ipc``
353 argument of ``uv_pipe_init`` to 1 to indicate this pipe will be used for
354 inter-process communication! Since the master will write the file handle to the
355 standard input of the worker, we connect the pipe to ``stdin`` using
356 ``uv_pipe_open``.
357
358 .. rubric:: multi-echo-server/worker.c
359 .. literalinclude:: ../../code/multi-echo-server/worker.c
360 :language: c
361 :linenos:
362 :lines: 51-79
363 :emphasize-lines: 10,15,20
364
365 First we call ``uv_pipe_pending_count()`` to ensure that a handle is available
366 to read out. If your program could deal with different types of handles,
367 ``uv_pipe_pending_type()`` can be used to determine the type.
368 Although ``accept`` seems odd in this code, it actually makes sense. What
369 ``accept`` traditionally does is get a file descriptor (the client) from
370 another file descriptor (The listening socket). Which is exactly what we do
371 here. Fetch the file descriptor (``client``) from ``queue``. From this point
372 the worker does standard echo server stuff.
373
374 Turning now to the master, let's take a look at how the workers are launched to
375 allow load balancing.
376
377 .. rubric:: multi-echo-server/main.c
378 .. literalinclude:: ../../code/multi-echo-server/main.c
379 :language: c
380 :linenos:
381 :lines: 9-13
382
383 The ``child_worker`` structure wraps the process, and the pipe between the
384 master and the individual process.
385
386 .. rubric:: multi-echo-server/main.c
387 .. literalinclude:: ../../code/multi-echo-server/main.c
388 :language: c
389 :linenos:
390 :lines: 51,61-95
391 :emphasize-lines: 17,20-21
392
393 In setting up the workers, we use the nifty libuv function ``uv_cpu_info`` to
394 get the number of CPUs so we can launch an equal number of workers. Again it is
395 important to initialize the pipe acting as the IPC channel with the third
396 argument as 1. We then indicate that the child process' ``stdin`` is to be
397 a readable pipe (from the point of view of the child). Everything is
398 straightforward till here. The workers are launched and waiting for file
399 descriptors to be written to their standard input.
400
401 It is in ``on_new_connection`` (the TCP infrastructure is initialized in
402 ``main()``), that we accept the client socket and pass it along to the next
403 worker in the round-robin.
404
405 .. rubric:: multi-echo-server/main.c
406 .. literalinclude:: ../../code/multi-echo-server/main.c
407 :language: c
408 :linenos:
409 :lines: 31-49
410 :emphasize-lines: 9,12-13
411
412 The ``uv_write2`` call handles all the abstraction and it is simply a matter of
413 passing in the handle (``client``) as the right argument. With this our
414 multi-process echo server is operational.
415
416 Thanks to Kyle for `pointing out`_ that ``uv_write2()`` requires a non-empty
417 buffer even when sending handles.
418
419 .. _pointing out: https://github.com/nikhilm/uvbook/issues/56
420
421 ----
422
423 .. [#] In this section domain sockets stands in for named pipes on Windows as
424 well.