Mercurial

comparison third_party/libuv/docs/src/guide/filesystem.rst @ 160:948de3f54cea

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
[ThirdParty] Added libuv
author June Park <parkjune1995@gmail.com>
date Wed, 14 Jan 2026 19:39:52 -0800
parents
children
comparison
equal deleted inserted replaced
159:05cf9467a1c3 160:948de3f54cea
1 Filesystem
2 ==========
3
4 Simple filesystem read/write is achieved using the ``uv_fs_*`` functions and the
5 ``uv_fs_t`` struct.
6
7 .. note::
8
9 The libuv filesystem operations are different from :doc:`socket operations
10 <networking>`. Socket operations use the non-blocking operations provided
11 by the operating system. Filesystem operations use blocking functions
12 internally, but invoke these functions in a `thread pool`_ and notify
13 watchers registered with the event loop when application interaction is
14 required.
15
16 .. _thread pool: https://docs.libuv.org/en/v1.x/threadpool.html#thread-pool-work-scheduling
17
18 All filesystem functions have two forms - *synchronous* and *asynchronous*.
19
20 The *synchronous* forms automatically get called (and **block**) if the
21 callback is null. The return value of functions is a :ref:`libuv error code
22 <libuv-error-handling>`. This is usually only useful for synchronous calls.
23 The *asynchronous* form is called when a callback is passed and the return
24 value is 0.
25
26 Reading/Writing files
27 ---------------------
28
29 A file descriptor is obtained using
30
31 .. code-block:: c
32
33 int uv_fs_open(uv_loop_t* loop, uv_fs_t* req, const char* path, int flags, int mode, uv_fs_cb cb)
34
35 ``flags`` and ``mode`` are standard
36 `Unix flags <https://man7.org/linux/man-pages/man2/open.2.html>`_.
37 libuv takes care of converting to the appropriate Windows flags.
38
39 File descriptors are closed using
40
41 .. code-block:: c
42
43 int uv_fs_close(uv_loop_t* loop, uv_fs_t* req, uv_file file, uv_fs_cb cb)
44
45
46 Filesystem operation callbacks have the signature:
47
48 .. code-block:: c
49
50 void callback(uv_fs_t* req);
51
52 Let's see a simple implementation of ``cat``. We start with registering
53 a callback for when the file is opened:
54
55 .. rubric:: uvcat/main.c - opening a file
56 .. literalinclude:: ../../code/uvcat/main.c
57 :language: c
58 :linenos:
59 :lines: 41-53
60 :emphasize-lines: 4, 6-7
61
62 The ``result`` field of a ``uv_fs_t`` is the file descriptor in case of the
63 ``uv_fs_open`` callback. If the file is successfully opened, we start reading it.
64
65 .. rubric:: uvcat/main.c - read callback
66 .. literalinclude:: ../../code/uvcat/main.c
67 :language: c
68 :linenos:
69 :lines: 26-39
70 :emphasize-lines: 2,8,12
71
72 In the case of a read call, you should pass an *initialized* buffer which will
73 be filled with data before the read callback is triggered. The ``uv_fs_*``
74 operations map almost directly to certain POSIX functions, so EOF is indicated
75 in this case by ``result`` being 0. In the case of streams or pipes, the
76 ``UV_EOF`` constant would have been passed as a status instead.
77
78 Here you see a common pattern when writing asynchronous programs. The
79 ``uv_fs_close()`` call is performed synchronously. *Usually tasks which are
80 one-off, or are done as part of the startup or shutdown stage are performed
81 synchronously, since we are interested in fast I/O when the program is going
82 about its primary task and dealing with multiple I/O sources*. For solo tasks
83 the performance difference usually is negligible and may lead to simpler code.
84
85 Filesystem writing is similarly simple using ``uv_fs_write()``. *Your callback
86 will be triggered after the write is complete*. In our case the callback
87 simply drives the next read. Thus read and write proceed in lockstep via
88 callbacks.
89
90 .. rubric:: uvcat/main.c - write callback
91 .. literalinclude:: ../../code/uvcat/main.c
92 :language: c
93 :linenos:
94 :lines: 17-24
95 :emphasize-lines: 6
96
97 .. warning::
98
99 Due to the way filesystems and disk drives are configured for performance,
100 a write that 'succeeds' may not be committed to disk yet.
101
102 We set the dominos rolling in ``main()``:
103
104 .. rubric:: uvcat/main.c
105 .. literalinclude:: ../../code/uvcat/main.c
106 :language: c
107 :linenos:
108 :lines: 55-
109 :emphasize-lines: 2
110
111 .. warning::
112
113 The ``uv_fs_req_cleanup()`` function must always be called on filesystem
114 requests to free internal memory allocations in libuv.
115
116 Filesystem operations
117 ---------------------
118
119 All the standard filesystem operations like ``unlink``, ``rmdir``, ``stat`` are
120 supported asynchronously and have intuitive argument order. They follow the
121 same patterns as the read/write/open calls, returning the result in the
122 ``uv_fs_t.result`` field. The full list:
123
124 .. rubric:: Filesystem operations
125 .. code-block:: c
126
127 int uv_fs_close(uv_loop_t* loop, uv_fs_t* req, uv_file file, uv_fs_cb cb);
128 int uv_fs_open(uv_loop_t* loop, uv_fs_t* req, const char* path, int flags, int mode, uv_fs_cb cb);
129 int uv_fs_read(uv_loop_t* loop, uv_fs_t* req, uv_file file, const uv_buf_t bufs[], unsigned int nbufs, int64_t offset, uv_fs_cb cb);
130 int uv_fs_unlink(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
131 int uv_fs_write(uv_loop_t* loop, uv_fs_t* req, uv_file file, const uv_buf_t bufs[], unsigned int nbufs, int64_t offset, uv_fs_cb cb);
132 int uv_fs_copyfile(uv_loop_t* loop, uv_fs_t* req, const char* path, const char* new_path, int flags, uv_fs_cb cb);
133 int uv_fs_mkdir(uv_loop_t* loop, uv_fs_t* req, const char* path, int mode, uv_fs_cb cb);
134 int uv_fs_mkdtemp(uv_loop_t* loop, uv_fs_t* req, const char* tpl, uv_fs_cb cb);
135 int uv_fs_mkstemp(uv_loop_t* loop, uv_fs_t* req, const char* tpl, uv_fs_cb cb);
136 int uv_fs_rmdir(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
137 int uv_fs_scandir(uv_loop_t* loop, uv_fs_t* req, const char* path, int flags, uv_fs_cb cb);
138 int uv_fs_scandir_next(uv_fs_t* req, uv_dirent_t* ent);
139 int uv_fs_opendir(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
140 int uv_fs_readdir(uv_loop_t* loop, uv_fs_t* req, uv_dir_t* dir, uv_fs_cb cb);
141 int uv_fs_closedir(uv_loop_t* loop, uv_fs_t* req, uv_dir_t* dir, uv_fs_cb cb);
142 int uv_fs_stat(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
143 int uv_fs_fstat(uv_loop_t* loop, uv_fs_t* req, uv_file file, uv_fs_cb cb);
144 int uv_fs_rename(uv_loop_t* loop, uv_fs_t* req, const char* path, const char* new_path, uv_fs_cb cb);
145 int uv_fs_fsync(uv_loop_t* loop, uv_fs_t* req, uv_file file, uv_fs_cb cb);
146 int uv_fs_fdatasync(uv_loop_t* loop, uv_fs_t* req, uv_file file, uv_fs_cb cb);
147 int uv_fs_ftruncate(uv_loop_t* loop, uv_fs_t* req, uv_file file, int64_t offset, uv_fs_cb cb);
148 int uv_fs_sendfile(uv_loop_t* loop, uv_fs_t* req, uv_file out_fd, uv_file in_fd, int64_t in_offset, size_t length, uv_fs_cb cb);
149 int uv_fs_access(uv_loop_t* loop, uv_fs_t* req, const char* path, int mode, uv_fs_cb cb);
150 int uv_fs_chmod(uv_loop_t* loop, uv_fs_t* req, const char* path, int mode, uv_fs_cb cb);
151 int uv_fs_utime(uv_loop_t* loop, uv_fs_t* req, const char* path, double atime, double mtime, uv_fs_cb cb);
152 int uv_fs_futime(uv_loop_t* loop, uv_fs_t* req, uv_file file, double atime, double mtime, uv_fs_cb cb);
153 int uv_fs_lutime(uv_loop_t* loop, uv_fs_t* req, const char* path, double atime, double mtime, uv_fs_cb cb);
154 int uv_fs_lstat(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
155 int uv_fs_link(uv_loop_t* loop, uv_fs_t* req, const char* path, const char* new_path, uv_fs_cb cb);
156 int uv_fs_symlink(uv_loop_t* loop, uv_fs_t* req, const char* path, const char* new_path, int flags, uv_fs_cb cb);
157 int uv_fs_readlink(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
158 int uv_fs_realpath(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
159 int uv_fs_fchmod(uv_loop_t* loop, uv_fs_t* req, uv_file file, int mode, uv_fs_cb cb);
160 int uv_fs_chown(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_uid_t uid, uv_gid_t gid, uv_fs_cb cb);
161 int uv_fs_fchown(uv_loop_t* loop, uv_fs_t* req, uv_file file, uv_uid_t uid, uv_gid_t gid, uv_fs_cb cb);
162 int uv_fs_lchown(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_uid_t uid, uv_gid_t gid, uv_fs_cb cb);
163 int uv_fs_statfs(uv_loop_t* loop, uv_fs_t* req, const char* path, uv_fs_cb cb);
164
165
166 .. _buffers-and-streams:
167
168 Buffers and Streams
169 -------------------
170
171 The basic I/O handle in libuv is the stream (``uv_stream_t``). TCP sockets, UDP
172 sockets, and pipes for file I/O and IPC are all treated as stream subclasses.
173
174 Streams are initialized using custom functions for each subclass, then operated
175 upon using
176
177 .. code-block:: c
178
179 int uv_read_start(uv_stream_t*, uv_alloc_cb alloc_cb, uv_read_cb read_cb);
180 int uv_read_stop(uv_stream_t*);
181 int uv_write(uv_write_t* req, uv_stream_t* handle,
182 const uv_buf_t bufs[], unsigned int nbufs, uv_write_cb cb);
183
184 The stream based functions are simpler to use than the filesystem ones and
185 libuv will automatically keep reading from a stream when ``uv_read_start()`` is
186 called once, until ``uv_read_stop()`` is called.
187
188 The discrete unit of data is the buffer -- ``uv_buf_t``. This is simply
189 a collection of a pointer to bytes (``uv_buf_t.base``) and the length
190 (``uv_buf_t.len``). The ``uv_buf_t`` is lightweight and passed around by value.
191 What does require management is the actual bytes, which have to be allocated
192 and freed by the application.
193
194 .. ERROR::
195
196 **THIS PROGRAM DOES NOT ALWAYS WORK, NEED SOMETHING BETTER**
197
198 To demonstrate streams we will need to use ``uv_pipe_t``. This allows streaming
199 local files [#]_. Here is a simple tee utility using libuv. Doing all operations
200 asynchronously shows the power of evented I/O. The two writes won't block each
201 other, but we have to be careful to copy over the buffer data to ensure we don't
202 free a buffer until it has been written.
203
204 The program is to be executed as::
205
206 ./uvtee <output_file>
207
208 We start off opening pipes on the files we require. libuv pipes to a file are
209 opened as bidirectional by default.
210
211 .. rubric:: uvtee/main.c - read on pipes
212 .. literalinclude:: ../../code/uvtee/main.c
213 :language: c
214 :linenos:
215 :lines: 62-80
216 :emphasize-lines: 4,5,15
217
218 The third argument of ``uv_pipe_init()`` should be set to 1 for IPC using named
219 pipes. This is covered in :doc:`processes`. The ``uv_pipe_open()`` call
220 associates the pipe with the file descriptor, in this case ``0`` (standard
221 input).
222
223 We start monitoring ``stdin``. The ``alloc_buffer`` callback is invoked as new
224 buffers are required to hold incoming data. ``read_stdin`` will be called with
225 these buffers.
226
227 .. rubric:: uvtee/main.c - reading buffers
228 .. literalinclude:: ../../code/uvtee/main.c
229 :language: c
230 :linenos:
231 :lines: 19-22,44-60
232
233 The standard ``malloc`` is sufficient here, but you can use any memory allocation
234 scheme. For example, node.js uses its own slab allocator which associates
235 buffers with V8 objects.
236
237 The read callback ``nread`` parameter is less than 0 on any error. This error
238 might be EOF, in which case we close all the streams, using the generic close
239 function ``uv_close()`` which deals with the handle based on its internal type.
240 Otherwise ``nread`` is a non-negative number and we can attempt to write that
241 many bytes to the output streams. Finally remember that buffer allocation and
242 deallocation is application responsibility, so we free the data.
243
244 The allocation callback may return a buffer with length zero if it fails to
245 allocate memory. In this case, the read callback is invoked with error
246 UV_ENOBUFS. libuv will continue to attempt to read the stream though, so you
247 must explicitly call ``uv_close()`` if you want to stop when allocation fails.
248
249 The read callback may be called with ``nread = 0``, indicating that at this
250 point there is nothing to be read. Most applications will just ignore this.
251
252 .. rubric:: uvtee/main.c - Write to pipe
253 .. literalinclude:: ../../code/uvtee/main.c
254 :language: c
255 :linenos:
256 :lines: 9-13,23-42
257
258 ``write_data()`` makes a copy of the buffer obtained from read. This buffer
259 does not get passed through to the write callback trigged on write completion. To
260 get around this we wrap a write request and a buffer in ``write_req_t`` and
261 unwrap it in the callbacks. We make a copy so we can free the two buffers from
262 the two calls to ``write_data`` independently of each other. While acceptable
263 for a demo program like this, you'll probably want smarter memory management,
264 like reference counted buffers or a pool of buffers in any major application.
265
266 .. WARNING::
267
268 If your program is meant to be used with other programs it may knowingly or
269 unknowingly be writing to a pipe. This makes it susceptible to `aborting on
270 receiving a SIGPIPE`_. It is a good idea to insert::
271
272 signal(SIGPIPE, SIG_IGN)
273
274 in the initialization stages of your application.
275
276 .. _aborting on receiving a SIGPIPE: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#The_special_problem_of_SIGPIPE
277
278 File change events
279 ------------------
280
281 All modern operating systems provide APIs to put watches on individual files or
282 directories and be informed when the files are modified. libuv wraps common
283 file change notification libraries [#fsnotify]_. This is one of the more
284 inconsistent parts of libuv. File change notification systems are themselves
285 extremely varied across platforms so getting everything working everywhere is
286 difficult. To demonstrate, I'm going to build a simple utility which runs
287 a command whenever any of the watched files change::
288
289 ./onchange <command> <file1> [file2] ...
290
291 .. note::
292
293 Currently this example only works on OSX and Windows.
294 Refer to the `notes of uv_fs_event_start`_ function.
295
296 .. _notes of uv_fs_event_start: https://docs.libuv.org/en/v1.x/fs_event.html#c.uv_fs_event_start
297
298 The file change notification is started using ``uv_fs_event_init()``:
299
300 .. rubric:: onchange/main.c - The setup
301 .. literalinclude:: ../../code/onchange/main.c
302 :language: c
303 :linenos:
304 :lines: 26-
305 :emphasize-lines: 15
306
307 The third argument is the actual file or directory to monitor. The last
308 argument, ``flags``, can be:
309
310 .. code-block:: c
311
312 /*
313 * Flags to be passed to uv_fs_event_start().
314 */
315 enum uv_fs_event_flags {
316 UV_FS_EVENT_WATCH_ENTRY = 1,
317 UV_FS_EVENT_STAT = 2,
318 UV_FS_EVENT_RECURSIVE = 4
319 };
320
321 ``UV_FS_EVENT_WATCH_ENTRY`` and ``UV_FS_EVENT_STAT`` don't do anything (yet).
322 ``UV_FS_EVENT_RECURSIVE`` will start watching subdirectories as well on
323 supported platforms.
324
325 The callback will receive the following arguments:
326
327 #. ``uv_fs_event_t *handle`` - The handle. The ``path`` field of the handle
328 is the file on which the watch was set.
329 #. ``const char *filename`` - If a directory is being monitored, this is the
330 file which was changed. Only non-``null`` on Linux and Windows. May be ``null``
331 even on those platforms.
332 #. ``int events`` - one of ``UV_RENAME`` or ``UV_CHANGE``, or a bitwise OR of
333 both.
334 #. ``int status`` - If ``status < 0``, there is an :ref:`libuv error<libuv-error-handling>`.
335
336 In our example we simply print the arguments and run the command using
337 ``system()``.
338
339 .. rubric:: onchange/main.c - file change notification callback
340 .. literalinclude:: ../../code/onchange/main.c
341 :language: c
342 :linenos:
343 :lines: 9-24
344
345 ----
346
347 .. [#fsnotify] inotify on Linux, FSEvents on Darwin, kqueue on BSDs,
348 ReadDirectoryChangesW on Windows, event ports on Solaris, unsupported on Cygwin
349 .. [#] see :ref:`pipes`