How to compress and decompress large data streams efficiently in Go?

In Go, you can efficiently compress and decompress large data streams using the compress/zlib package, which provides the functionality to work with the zlib compression format.

Here's an example of how to compress and decompress large data streams in Go:

Import the necessary packages:

import (
    "compress/zlib"
    "io"
    "os"
)

To compress the data stream, create a zlib writer and pass the output stream as its parameter:

func Compress(src io.Reader, dst io.Writer) error {
    zw := zlib.NewWriter(dst)
    defer zw.Close()

    _, err := io.Copy(zw, src)
    return err
}

In this example, src is the input data stream, and dst is the output data stream (e.g., a file or network connection). The Copy function is used to efficiently copy the data from the input stream to the zlib writer, compressing it in the process.

To decompress the data stream, create a zlib reader and pass the input stream as its parameter:

func Decompress(src io.Reader, dst io.Writer) error {
    zr, err := zlib.NewReader(src)
    if err != nil {
        return err
    }
    defer zr.Close()

    _, err = io.Copy(dst, zr)
    return err
}

In this example, src is the input compressed data stream, and dst is the output data stream where the decompressed data will be written to.

To use these functions, open the input and output files, then call the compression or decompression function:

func main() {
    inputFile, _ := os.Open("input.txt")
    outputFile, _ := os.Create("output.txt")

    // Compress the input stream
    err := Compress(inputFile, outputFile)
    if err != nil {
        panic(err)
    }

    outputFile.Close()

    // Decompress the output stream
    inputFile2, _ := os.Open("output.txt")
    outputFile2, _ := os.Create("decompressed.txt")

    err = Decompress(inputFile2, outputFile2)
    if err != nil {
        panic(err)
    }

    outputFile2.Close()
}

In this example, the input.txt file is compressed and written to output.txt. Then, output.txt is decompressed back to decompressed.txt.

Note that error handling has been omitted for brevity, so make sure to handle errors appropriately in your real-world code.

By using the compress/zlib package, you can efficiently compress and decompress large data streams in Go.