File I/O (os, bufio, io)

Reviewed & published by Brayan K

Go reads and writes files through the os package for whole-file operations, bufio for efficient line-by-line and buffered access, and io for streaming data of any size between readers and writers.

Part of the free Go course at LearnCodingFast — hands-on lessons with worked examples and the output they print, plus practice exercises and a quick quiz.

What You'll Learn in This Lesson

• Read and write whole files with os.ReadFile/WriteFile
• Open files and always defer file.Close()
• Read line by line with a bufio.Scanner
• Batch writes with a bufio.Writer (and Flush!)
• Stream any size with io.Copy
• Work safely under os.TempDir()

1️⃣ Whole-file: WriteFile & ReadFile

The simplest operations. os.WriteFile(path, data, perm) creates or truncates a file and writes all the bytes; os.ReadFile(path) returns the whole file as a []byte. Both handle opening and closing for you — ideal for small files. We write under os.TempDir() so it runs and tidies up.

package main

import (
    "fmt"
    "os"
    "path/filepath"
)

func main() {
    // Work inside the OS temp dir so this RUNS anywhere and cleans up easily.
    path := filepath.Join(os.TempDir(), "lcf_demo.txt")

    // WriteFile creates (or truncates) the file and writes all the bytes.
    // 0644 = owner read/write, others read.
    data := []byte("line one\nline two\n")
    if err := os.WriteFile(path, data, 0644); err != nil {
        fmt.Println("write failed:", err)
        return
    }

    // ReadFile slurps the ENTIRE file into memory — great for small files.
    got, err := os.ReadFile(path)
    if err != nil {
        fmt.Println("read failed:", err)
        return
    }
    fmt.Printf("read %d bytes:\n%s", len(got), got)

    os.Remove(path) // tidy up
}

2️⃣ Line by line with bufio.Scanner

For larger files, os.Open the file (and defer f.Close()), then wrap it in a bufio.NewScanner. Each Scan() advances to the next line; Text() gives it to you without the newline. Always check scanner.Err() after the loop to catch read errors.

package main

import (
    "bufio"
    "fmt"
    "os"
    "path/filepath"
)

func main() {
    path := filepath.Join(os.TempDir(), "lcf_scan.txt")
    os.WriteFile(path, []byte("apple\nbanana\ncherry\n"), 0644)
    defer os.Remove(path)

    // Open returns an *os.File you must Close. defer guarantees it.
    f, err := os.Open(path)
    if err != nil {
        fmt.Println("open failed:", err)
        return
    }
    defer f.Close()

    // bufio.Scanner reads efficiently, one LINE at a time by default —
    // the right tool for large files you don't want fully in memory.
    scanner := bufio.NewScanner(f)
    lineNo := 1
    for scanner.Scan() {
        fmt.Printf("%d: %s\n", lineNo, scanner.Text()) // Text() strips the newline
        lineNo++
    }
    if err := scanner.Err(); err != nil { // check for read errors after the loop
        fmt.Println("scan error:", err)
    }
}

3️⃣ Buffered writing & streaming with io.Copy

A bufio.Writer batches many small writes into fewer syscalls — just remember to Flush() before closing or the last bytes never land. And io.Copy(dst, src) streams from any reader to any writer in chunks, copying files of any size with constant memory.

package main

import (
    "bufio"
    "fmt"
    "io"
    "os"
    "path/filepath"
)

func main() {
    src := filepath.Join(os.TempDir(), "lcf_src.txt")
    dst := filepath.Join(os.TempDir(), "lcf_dst.txt")
    defer os.Remove(src)
    defer os.Remove(dst)

    // --- bufio.Writer: batch many small writes into fewer syscalls ---
    f, _ := os.Create(src)
    w := bufio.NewWriter(f)
    for i := 1; i <= 3; i++ {
        fmt.Fprintf(w, "row %d\n", i)
    }
    w.Flush() // IMPORTANT: flush buffered bytes to the file...
    f.Close() // ...then close.

    // --- io.Copy: stream from one file to another without loading it all ---
    in, _ := os.Open(src)
    defer in.Close()
    out, _ := os.Create(dst)
    defer out.Close()

    n, err := io.Copy(out, in) // returns bytes copied
    if err != nil {
        fmt.Println("copy failed:", err)
        return
    }
    out.Close() // flush to disk before reading back

    copied, _ := os.ReadFile(dst)
    fmt.Printf("copied %d bytes; dst contains:\n%s", n, copied)
}

🎯 Your Turn

Write a file and read it straight back. Fill in the two blanks marked ___, then run it.

package main

import (
    "fmt"
    "os"
    "path/filepath"
)

func main() {
    path := filepath.Join(os.TempDir(), "lcf_turn.txt")
    defer os.Remove(path)

    // 🎯 YOUR TURN — fill in the blanks marked ___

    // 1) Write the text below to the file with mode 0644
    err := ___ // 👉 os.WriteFile(path, []byte("saved!\n"), 0644)
    if err != nil {
        fmt.Println(err)
        return
    }

    // 2) Read the whole file back into a byte slice
    data, err := ___ // 👉 os.ReadFile(path)
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Print(string(data))

    // ✅ Expected output:
    //    saved!
}

Common Errors (and the fix)

❌ Forgetting f.Close() after os.Open/Create — leaks file descriptors.

✅ defer f.Close() right after a successful open.

❌ Not calling w.Flush() on a bufio.Writer — the last buffered bytes vanish.

✅ w.Flush() before closing the underlying file.

❌ Reading a huge file with ReadFile — out-of-memory.

✅ Stream it with a bufio.Scanner or io.Copy.

❌ Ignoring scanner.Err() — a read error looks just like end-of-file.

✅ Check it after the for loop.

🧠 Quick Recall

wrote with bufio.Writer, no Flush — file content?

Possibly empty or truncated. Buffered bytes sit in memory until Flush(); without it (or if you close the file first) they never reach disk.

scanner.Text() on "hello\n" returns?

hello — Text() returns the line without the trailing newline. Use scanner.Bytes() for the raw bytes.

🎯 Mini-Challenge: Count Non-Empty Lines

Write the sample text to a temp file, then scan it and count the lines that aren't empty. The text has 5 lines but 2 are blank, so print non-empty lines: 3.

package main

import (
    "bufio"
    "fmt"
    "os"
    "path/filepath"
)

func main() {
    // 🎯 MINI-CHALLENGE: count non-empty lines
    // Write the multi-line text below to a temp file, then open it and use a
    // bufio.Scanner to count how many lines are NOT empty. Print the count.
    //
    // ✅ Expected output:
    //    non-empty lines: 3

    path := filepath.Join(os.TempDir(), "lcf_count.txt")
    defer os.Remove(path)
    os.WriteFile(path, []byte("go\n\nrust\n\nzig\n"), 0644)

    // your code here: os.Open, defer Close, bufio.NewScanner, loop, count
    // lines where scanner.Text() != ""
    _ = bufio.NewScanner
}

🎉 Lesson Complete!

• ✅ os.WriteFile/os.ReadFile handle whole small files (and the Close for you)
• ✅ os.Open + defer f.Close() + bufio.Scanner for line-by-line reading
• ✅ bufio.Writer batches writes — always Flush() before closing
• ✅ io.Copy streams any size with constant memory
• ✅ Check scanner.Err() and write under os.TempDir() for portable demos

Practice quiz

Continue this course

• Previous: HTTP Middleware
• Next: Custom Errors & errors.Is/As — Sentinel errors, custom error types, wrapping with %w, and inspecting chains

Frequently asked questions

When should I use os.ReadFile versus a bufio.Scanner?

os.ReadFile reads the whole file into a single byte slice in memory — simple and perfect for small files like configs. A bufio.Scanner streams the file one line (or token) at a time, so memory stays constant no matter how big the file is. Use ReadFile for small files you want all at once; use a Scanner for large files or anything you process line by line.

Why do I have to call file.Close(), and why with defer?

An open file holds an OS file descriptor; leaving it open leaks descriptors and, on Windows, locks the file. Writing defer f.Close() right after a successful Open guarantees it closes when the function returns, on every path including errors. Note os.ReadFile and os.WriteFile open and close the file for you, so you only manage Close yourself when you call os.Open or os.Create.

What does the 0644 mode mean?

It's the Unix file permission in octal: 6 (read+write) for the owner, 4 (read) for the group, 4 (read) for others — so 'owner can edit, everyone can read'. Common choices are 0644 for data files and 0755 for executables and directories. On Windows the bits are largely ignored, but you still pass a mode for portability.

Why must I call Flush on a bufio.Writer?

A bufio.Writer collects your writes in an in-memory buffer and only sends them to the file in larger chunks, which is far more efficient than one syscall per small write. But that means data sits in the buffer until you call w.Flush(). Forget Flush (or close the underlying file before flushing) and the last buffered bytes never reach disk. Always Flush before closing.

What is io.Copy and why prefer it over reading then writing?

io.Copy(dst, src) streams from any io.Reader to any io.Writer in fixed-size chunks, so it copies files of any size with constant memory — you never hold the whole thing at once. It also returns the number of bytes copied. Because it works on the Reader/Writer interfaces, the same call copies between files, network connections, HTTP bodies, and buffers.

How do I append to a file instead of overwriting it?

os.WriteFile always truncates. To append, open the file with os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644), then Write to it and Close. The O_APPEND flag makes every write go to the end of the file, and O_CREATE creates it if it doesn't exist yet.