在 Golang 中建立 Google Drive 下載器（第 1 部分）-Golang-PHP中文網

Building a Google Drive Downloader in Golang (Part 1)

Introduction

In this tutorial, we’ll build a powerful downloader that allows downloading files from Google Drive and other cloud providers. With Golang’s efficient concurrency patterns, you'll be able to manage multiple downloads concurrently, stream large files, and track progress in real-time. Whether you’re downloading a few small files or handling large data sets, this project will showcase how to build a scalable and robust downloader that can easily be extended to support multiple cloud platforms.

If you're looking for a way to simplify and automate downloading large files, this tutorial is perfect for you. By the end, you’ll have a flexible and customizable Go-based downloader to suit your needs.

In a hurry?

If you're just looking to use this downloader with a UI, visit evolveasdev.com for read the full article & Go Downloader's Github. You'll find the docs to get it running fast.

What You’ll Learn

Go Concurrency Patterns:
Learn how to use Goroutines, channels and mutexes to handle multiple concurrent file downloads efficiently.
Streaming Large Downloads:
Explore how to stream large files while managing memory and system resources effectively.
Concurrent File Downloads:
Understand how to download files concurrently, speeding up the process and improving performance.
Real-Time Progress Updates:
Implement progress tracking to provide real-time feedback on download status.
Handling Interruptions and Cancellations:
Learn how to gracefully cancel one or all ongoing downloads.

Note: This tutorial will only focus on the core downloading logic.

Environment Setup

First before doing anything make sure to properly setup your environment to avoid potential bugs in future.

Prerequisites

Install Go
AIR for auto reloading
Makefile to run complex commands
Goose for PostgreSQL migrations

Configuring Makefile

Create a makefile at the root of the project with the following.

# Load environment variables from .env file
include ./.env

# To run the application
run: build
    @./bin/go-downloader

# Build the application
build:
    @go build -tags '!dev' -o bin/go-downloader

# Database migration status
db-status:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) status

# Run database migrations
up:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) up

# Roll back the last database migration
down:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) down

# Reset database migrations
reset:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) reset

登入後複製

High-Level Folder Structure Overview

go-downloader/
├── api
├── config
├── migrations
├── service
├── setting
├── store
├── types
├── util
├── .env
├── .air.toml
├── Makefile
├── go.mod
├── go.sum
└── main.go

登入後複製

Setting environment variables

Create a .env file in root or handle environment variables however you like, we'll use joho/godotenv package.

GOOGLE_CLIENT_ID
GOOGLE_CLIENT_SECRET
SESSION_SECRET=something-super-secret
APP_URL=http://localhost:3000
POSTGRES_USER
POSTGRES_PASSWORD
POSTGRES_DB

登入後複製

Creating a Web Server

We'll now start creating the web server that'll handle all the incoming requests.

Heads Up! The main part of this guide begins here. Get ready to dive in!

API Layer

To start, create the following files inside api folder api.go and route.go

route.go file setup

All API routes will be defined in this here. We create a NewRouter struct that takes the env configuration, allowing all routes and handlers to access environment variables.

package api

import (
    "github.com/gofiber/fiber/v2"
    "github.com/nilotpaul/go-downloader/config"
)

type Router struct {
    env       config.EnvConfig
}

func NewRouter(env config.EnvConfig) *Router {
    return &Router{
        env:      env,
    }
}

func (h *Router) RegisterRoutes(r fiber.Router) {
    r.Get("/healthcheck", func(c *fiber.Ctx) error {
        return c.JSON("OK")
    })
}

登入後複製

api.go file setup

Here, we’ll add all the necessary middlewares, such as CORS and logging, before starting the server.

type APIServer struct {
    listenAddr string
    env        config.EnvConfig
}

func NewAPIServer(listenAddr string, env config.EnvConfig) *APIServer {
    return &APIServer{
        listenAddr: listenAddr,
        env:        env,
    }
}

func (s *APIServer) Start() error {
    app := fiber.New(fiber.Config{
        AppName:      "Go Downloader",
    })

    handler := NewRouter()
    handler.RegisterRoutes(app)

    log.Printf("Server started on http://localhost:%s", s.listenAddr)

    return app.Listen(":" + s.listenAddr)
}

登入後複製

Main Entrypoint

This is the main package in main.go file which will act as a entrypoint to the whole.

func main() {
    // Loads all Env vars from .env file.
    env := config.MustLoadEnv()

    log.Fatal(s.Start())
}

登入後複製

This is enough to start up the server and test it.

Start the server

air

登入後複製

that's it.?

Testing

curl http://localhost:3000/healthcheck

登入後複製

The response should be OK with status 200

Creating a Provider Store

We need to implement a scalable solution for adding support for multiple cloud providers if necessary.

Working on provider registry

// Better to keep it in a seperate folder.
// Specific only to OAuth Providers.
type OAuthProvider interface {
    Authenticate(string) error
    GetAccessToken() string
    GetRefreshToken() string
    RefreshToken(*fiber.Ctx, string, bool) (*oauth2.Token, error)
    IsTokenValid() bool
    GetAuthURL(state string) string
    CreateOrUpdateAccount() (string, error)
    CreateSession(c *fiber.Ctx, userID string) error
    UpdateTokens(*GoogleAccount) error
}

type ProviderRegistry struct {
    Providers map[string]OAuthProvider
}

func NewProviderRegistry() *ProviderRegistry {
    return &ProviderRegistry{
        Providers: make(map[string]OAuthProvider),
    }
}

func (r *ProviderRegistry) Register(providerName string, p OAuthProvider) {
    r.Providers[providerName] = p
}

func (r *ProviderRegistry) GetProvider(providerName string) (OAuthProvider, error) {
    p, exists := r.Providers[providerName]
    if !exists {
        return nil, fmt.Errorf("Provider not found")
    }

    return p, nil
}

登入後複製

The ProviderRegistry serves as a central map to hold all our OAuth providers. When we initialize our providers, we'll register them in this map. This allows us to easily access any registered provider's functionalities throughout our service.

You'll see this action later.

Initializing the Provider Store

We'll register our providers based on the environment variables provided.

func InitStore(env config.EnvConfig) *ProviderRegistry {
    r := NewProviderRegistry()

    if len(env.GoogleClientSecret) != 0 || len(env.GoogleClientID) != 0 {
        googleProvider := NewGoogleProvider(googleProviderConfig{
            googleClientID:     env.GoogleClientID,
            googleClientSecret: env.GoogleClientSecret,
            googleRedirectURL:  env.AppURL + "/callback/google",
        }, env)

        r.Register("google", googleProvider)
    }

    return r
}

登入後複製

Read the full article here.