Skip to content

Streams and I/O errors

stdlib/stream.kod defines the error type and the two interfaces that byte streams share, and stdlib/posix.kod implements them over file descriptors: files, standard input and output and, later, sockets. Code that reads or writes takes a Reader or a Writer and doesn't care what is behind it.

import "stream" { Reader }
import "posix" { open_read }

func count_bytes(anon r: Reader) -> int64 or Error {
    let data = try r.read_all()
    return len(data)
}

func main() -> int64 or Error {
    let f = try open_read("data.bin")
    print_int(try count_bytes(f))
    try f.close()
    return 0
}

Errors

Every operation that can fail returns T or Error and throws an IoError:

type IoKind = enum {
    NotFound
    PermissionDenied
    AlreadyExists
    InvalidInput
    WouldBlock      // reserved; nothing returns it yet
    TimedOut
    ConnectionRefused
    ConnectionReset
    BrokenPipe
    UnexpectedEof
    Other
    HostNotFound
    Tls
    Protocol
}

type IoError = struct {
    kind: IoKind
    op: str        // what was attempted: "open data.bin", "read", "write"
    errno: int64   // the OS error number, 0 if the error didn't come from the OS
    detail: str    // the OS's message, or a short explanation
}

to_str gives open data.bin: No such file or directory. Recover the kind with a downcast:

match open_read(path) {
    Ok(f) -> use(f)
    Err(e) ->
        if e is IoError(io) and io.kind == .NotFound {
            print("no such file")
        } else {
            print(e.to_str())
        }
}

kind_of_errno(errno) maps an OS error number to an IoKind; anything it doesn't know is Other. EAGAIN is TimedOut, because a descriptor here is only ever non-blocking through a timeout (see TCP). HostNotFound is for a name that doesn't resolve and has no OS error number (errno is 0). Tls is for a TLS failure that isn't a plain I/O error, such as a certificate that doesn't verify (see TLS). Protocol is for a peer that breaks the rules of the protocol being spoken, such as a malformed HTTP response (see HTTP). An interrupted system call is retried inside the runtime, so EINTR never reaches a caller.

Reader and Writer

interface Reader {
    func read(self, anon max: int64) -> bytes or Error

    func read_all(self) -> bytes or Error { ... }
    func read_exact(self, anon n: int64) -> bytes or Error { ... }
}

interface Writer {
    func write(self, anon data: bytes) -> int64 or Error

    func write_all(self, anon data: bytes) -> none or Error { ... }
    func write_str(self, anon text: str) -> none or Error { ... }
}

read returns between 1 and max bytes, waiting until there is something to return, and returns an empty bytes at the end of the stream. With max <= 0 it returns an empty bytes without reading, so don't call it that way to test for the end. write returns how many bytes it accepted, which may be fewer than len(data); an empty data writes nothing and returns 0.

A type only has to declare read or write; the rest are provided methods built on them, so every Reader and Writer has them, called on the value itself or through the interface:

Method Meaning
r.read_all() read until the end of the stream; bytes or Error
r.read_exact(n) exactly n bytes, or an UnexpectedEof error; bytes or Error
w.write_all(data) write every byte, repeating write as needed; a write that accepts 0 bytes of a non-empty data is an Other error (write returned 0); none or Error
w.write_str(s) write_all of a str

In-memory streams

MemReader(data: bytes, pos: 0) reads from a bytes value, which is how tests and parsers get a stream without a file. MemWriter(buf: StrBuilder()) collects what is written, and to_bytes() / to_str() return a copy of it. Neither can fail.

BufReader

BufReader wraps any Reader with a buffer, for protocols that mix lines and counted bodies such as HTTP:

let br = BufReader(inner: conn)
let status = try br.read_line()          // str?, none at the end of the stream
let body = try br.read_exact(length)     // takes what is buffered first
Method Meaning
read(max) a Reader: the buffered bytes first, else one read of the inner stream
read_exact(n) the Reader's, so exactly n bytes, or UnexpectedEof
read_until(delim) the bytes up to and excluding the byte delim, which is consumed; bytes?, none at the end of the stream; the last piece comes back even without a delimiter
read_line() read_until(10) as a str, minus a trailing \r (str?)

Fields: inner, limit: int64 = 1048576 (a read_until or read_line longer than this fails with an InvalidInput error line too long instead of growing without bound), chunk: int64 = 8192 (the size of each inner read). ended is set when a read_until or read_line returned a last piece that the stream's end cut short, with no delimiter after it; a protocol that needs every line terminated can tell a truncation from a complete line by it. A BufReader doesn't close its inner stream.

Files and descriptors

Fd is a file descriptor and implements both Reader and Writer:

Function Meaning
open_read(path) an existing file for reading
create(path) a file for writing, created (mode 0644) or truncated
append(path) a file for writing, created if missing, written at its end
stdin(), stdout(), stderr() descriptors 0, 1 and 2
Fd(fd: n) wrap a descriptor from elsewhere
f.close() close it and set f.fd to -1, so closing it again fails; none or Error
f.is_terminal() whether it is a terminal, rather than a file or a pipe
signal(pid, sig) send the signal numbered sig to the process pid, as C's kill; none or Error (for a child of this program, process.Child.kill is the easier way)

Waiting on several descriptors

poll(read:, write:, timeout:) waits until at least one of the descriptors is ready, and says which, so a program can serve several pipes or connections without blocking on any one of them:

let ready = try posix.poll(read: [out, err], write: [input], timeout: Duration(seconds: 5))
if ready.read[0] {
    let piece = try out.read(65536)         // doesn't wait
}

It returns a posix.Ready or an Error. ready.read and ready.write hold a bool for each descriptor in read and write, in the same order.

  • Ready for reading means a read won't wait: there is data, the stream has ended, or an error is pending.
  • Ready for writing means a write of up to 512 bytes won't wait, or will fail at once because the reader has gone.
  • Ignored descriptors: a descriptor of -1, such as a closed Fd, is never ready.
  • Timeout: without timeout: it waits as long as it takes. When the timeout passes first, every entry is false. With a zero Duration it only looks.
  • Nothing to wait for: if both lists are empty or hold only -1, it fails with an InvalidInput error instead of waiting forever.

stdout() writes straight to descriptor 1, bypassing the buffer that print uses. Call flush_stdout() first when you mix the two. Closing is explicit: an Fd that is dropped stays open until the program exits.

Colour

stdlib/term.kod colours text for a terminal. term.colour(fd) says whether to: fd is a terminal, NO_COLOR is unset or empty, and TERM isn't dumb. The other functions wrap text in the escape codes of a style, whatever the output is, so check first:

import "io"
import "posix"
import "term"

func main() -> int64 {
    mut label = "error:"
    if term.colour(posix.stderr()) {
        label = term.red(term.bold(label))
    }
    io.write_stderr(f"{label} no such file\n")
    return 1
}

The styles are bold, dim, red, green and yellow.

The older io.read_file and io.write_file keep their behavior: an empty string or -1 on failure, with no reason.

TCP

stdlib/net/tcp.kod opens TCP connections. A connection is an Fd, so it is a Reader and a Writer: it works with BufReader and has read_all, write_str and the rest. It is Kod throughout, calling the C library's socket functions through unsafe.

import "net/tcp" { connect, set_timeout }
import "time" { Duration }
import "stream" { BufReader }

func fetch_status_line(host: str) -> str or Error {
    let conn = try connect(host, port: 80)
    try set_timeout(conn, timeout: Duration(seconds: 5))
    try conn.write_str("HEAD / HTTP/1.0\r\n\r\n")
    let br = BufReader(inner: conn)
    let line = try br.read_line()
    try conn.close()
    return line? or ""
}
Function Meaning
connect(host, port, timeout = Duration(seconds: 10)) resolve host (a name or an IP literal, IPv4 or IPv6) and connect to the first address that answers; Fd or Error
set_timeout(f, timeout) make a read or a write on f fail with TimedOut after a Duration without progress; zero means wait forever; none or Error
listen(host, port, backlog = 128) a Listener bound to host:port; an empty host means every IPv4 interface and port 0 picks a free port; Listener or Error
l.accept() the next connection, waiting for one; Fd or Error
is_idle(f) true if nothing is waiting to be read from f; false if it has data, an end of stream (the peer closed it) or an error pending
l.port() the port it is bound to
l.close() stop listening; none or Error

connect gives up on each address after timeout, so a host with several unreachable addresses can take longer in all. A refused connection is ConnectionRefused, a timeout TimedOut, a name that doesn't resolve HostNotFound, and a port outside 0..65535 or an empty host InvalidInput. Writing to a peer that has closed is a BrokenPipe error and doesn't kill the program with SIGPIPE. A connection has TCP_NODELAY set.

TLS

stdlib/net/tls.kod runs TLS over a TCP connection, using the system's SecureTransport (the Security framework), so it needs nothing installed and checks certificates against the macOS keychain. It is Kod throughout, calling the framework through unsafe, its I/O callbacks included. A program that imports it links -framework Security -framework CoreFoundation; no other program does.

import "net/tls" { connect }
import "stream" { BufReader }

func main() -> int64 or Error {
    let conn = try connect("example.com", port: 443)
    try conn.write_str("GET / HTTP/1.0\r\nHost: example.com\r\n\r\n")
    let br = BufReader(inner: conn)
    print((try br.read_line())? or "")
    try conn.close()
    return 0
}

A TlsStream is a Reader and a Writer, so everything above works on it as on a plain connection.

Function Meaning
connect(host, port, trust_file = "", timeout = Duration(seconds: 10)) open a TCP connection with net/tcp and run the TLS handshake; TlsStream or Error
s.read(max), s.write(data) as for any stream
s.set_timeout(timeout) as net/tcp.set_timeout: a read or write fails with TimedOut after a Duration without progress
s.close() send a close notice, then close the connection; none or Error

The server's certificate must be valid for host, chain to a trusted root and be in date; otherwise connect throws an IoError of kind Tls whose detail says why, and the connection is closed. timeout limits the TCP connect and, separately, the handshake. A server that can't speak TLS 1.2 is a Tls error: SecureTransport has no TLS 1.3, and the minimum version is 1.2.

trust_file is the path of a PEM file of certificates. When it isn't empty, those certificates are the only trusted roots, in place of the keychain: use it for a private CA or a self-signed server. The host name is still checked.

A server that closes the connection without a close notice reads as an ordinary end of stream, since plain HTTP/1.0 servers do that; a response that carries its own length (Content-Length or chunked) shows a truncation anyway.