linux - Is it possible to write a file that is not a multiple of page size with libaio? - Stack Overflow

admin管理员组

文章数量:1294326

I'm doing some large I/O on a bunch of SSDs and found that I can get the required performance using async read/write with libaio (io_uring isn't available to me). The problem is that I sometimes need to do writes that are not a multiples of the page size and also append to files that are already not a multiple of block size.

Is there a good strategy here? Here are some things I've thought of, which don't sound great.

a) I could do a read on the last incomplete block of the file, then write a full block with the old and new data. Then at the end I could pad the last block and then truncate the file to the desired size when I'm done. I'm a little reluctant to do this since a bug in my code will corrupt the end of the existing file.

b) does it make sense to combine non-direct and direct writes to the same file? I could write the first incomplete block and last incomplete block with a non O_DIRECT fd and do the large middle with async IO. Then fsync? I'm not sure if this is a good idea.

Are there better options?

Here is a sample program that fails to write a non-page size amount of data using linux native aio:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <linux/aio_abi.h>
#include <sys/syscall.h>
#include <errno.h>
#include <memory>
#include <utility>
#include <iostream>

template<typename FuncT> class ScopeExit {
public:
    explicit ScopeExit(FuncT&& fn) : f(std::forward<FuncT>(fn)) {}

    ~ScopeExit() { f(); }

    ScopeExit(const ScopeExit&) = delete;
    ScopeExit& operator=(const ScopeExit&) = delete;
    ScopeExit(ScopeExit&&) = delete;
    ScopeExit& operator=(ScopeExit&&) = delete;

private:
    FuncT f;
};
template<typename F> ScopeExit(F&&) -> ScopeExit<F>;

template<typename... Args> long checked_syscall(long syscall_number, Args... args) {
    long ret = syscall(syscall_number, args...);
    if (ret < 0) {
        auto error = errno;
        std::cerr << "Syscall " << syscall_number << " failed: " 
                  << strerror(error) << " (errno=" << error << ")\n";
        std::exit(1);
    }
    return ret;
}
    
int io_setup(unsigned nr_events, aio_context_t *ctx_idp) {
    return checked_syscall(__NR_io_setup, nr_events, ctx_idp);
}

int io_submit(aio_context_t ctx_id, long nr, struct iocb **iocbpp) {
    return checked_syscall(__NR_io_submit, ctx_id, nr, iocbpp);
}

int io_getevents(aio_context_t ctx_id, long min_nr, long nr,
                             struct io_event *events, struct timespec *timeout) {
    return checked_syscall(__NR_io_getevents, ctx_id, min_nr, nr, events, timeout);
}

int io_destroy(aio_context_t ctx_id) {
    return checked_syscall(__NR_io_destroy, ctx_id);
}

long checked_libcall(const char* msg, long ret) {
    if (ret < 0) {
        auto error = errno;
        std::cerr << msg << " failed: "  << strerror(error) << " (errno=" << error << ")\n";
        std::exit(1);
    }
    return ret; 
}

long checked_libcall(const char* msg, long want, long ret) {
    ret = checked_libcall(msg, ret);
    if (ret != want) {
        std::cerr << msg << " failed: want '" << want << "' got '" << ret << "'\n";
        std::exit(1);
    }
    return ret;
}
    
#define CHECKED_LIBCALL(...) checked_libcall(#__VA_ARGS__, (__VA_ARGS__))
    
int main() {
    const char *filename = "onebyte.txt";

    // Initialize the AIO context
    aio_context_t ctx = 0;
    io_setup(1, &ctx);
    auto destroyCtx = ScopeExit([&]{ io_destroy(ctx); });
    
    // Open the file for write
    int fd = CHECKED_LIBCALL(open(filename, O_WRONLY | O_CREAT | O_DIRECT));
    auto closeFile = ScopeExit([&]{ close(fd); });

    // Prepare 4K aligned buffer for writing
    auto buffer = std::unique_ptr<char[]>(new (std::align_val_t{4 << 10}) char[1]);
    buffer[0] = 'A';
    
    // Prepare the I/O control block
    struct iocb cb;
    struct iocb *cbs[1];
    memset(&cb, 0, sizeof(cb));
    cb.aio_fildes = fd;
    cb.aio_lio_opcode = IOCB_CMD_PWRITE;
    cb.aio_buf = reinterpret_cast<uint64_t>(buffer.get());
    cb.aio_offset = 0;
    cb.aio_nbytes = 1;
    cbs[0] = &cb;

    // Submit the I/O request
    CHECKED_LIBCALL(io_submit(ctx, 1, cbs), 1);

    // Wait for completion
    struct io_event event;
    CHECKED_LIBCALL(io_getevents(ctx, 1, 1, &event, NULL), 1);

    // Check results
    if (event.res < 0) {
        auto error = -event.res;
        std::cerr << "AIO write error: errno=" << error << " [" << strerror(error) << "]\n";
    } else {
        std::cerr << "Wrote " << event.res << " byte: " << buffer[0] << "\n";
    }

    // remove the file
    unlink(filename);
    
    return 0;
}

本文标签： linuxIs it possible to write a file that is not a multiple of page size with libaioStack Overflow