ponzi2

How to Write Second Generation AppEngine Apps with Go 1.11

By Brian Muramatsu

2019-01-07

Overview

What is the ponzi2.io web site written in?

Go 1.11 with just the standard library! It also runs for FREE on the Second Generation AppEngine runtime without any AppEngine dependencies!* Since the new runtime allows any extension or library, I decided to write a server from scratch inspired by suckless.org. In other words, I wanted to drop static content into folders, serve them at URLs matching those folder paths, and render shared headers and footers using the template package. Let's check out the full ~200 lines!**

* You still need an AppEngine app.yaml file. :-| Also, you might get charged a literal penny or two for Cloud Storage which is needed to compile and deploy your app.

** The 200 lines does not include HTML files nor the GitHub and GitLab integration code. ;-)

Bursting the Bubble: app.yaml Lives!

Although the Go code has no dependencies on AppEngine libraries, you will still need an app.yaml file to eventually deploy it on AppEngine. Meanwhile, you can still write and run your server locally as if it were a regular Go program without an app.yaml. Let's get it out of the way though...

# https://cloud.google.com/appengine/docs/standard/go111/config/appref

runtime: go111

env_variables:
    PAYPAL_ENVIRONMENT: production
    CACHE_TEMPLATES: true

handlers:
- url: /.*
    script: auto
    secure: always
    redirect_http_response_code: 301
    

Notes:

Live Free with Environment Variables

Before diving into main, let's check out how to use environment variables to stay AppEngine-free. In this snippet, we read in a value from an environment variable to enable or disable caching templates, a custom feature specific to ponzi2.io. What this feature does is not important. What's interesting is that we use os.Getenv to read in the environment variable and have our code act on it. When testing locally, we start the server as a normal Go program which doesn't read from app.yaml. On the other hand, when we run our app on AppEngine, AppEngine will the start the server with the app.yaml and use the values defined there. In other words, environment variables allow us to remain independent from AppEngine yet they allow us to control our server's behavior when it does run on AppEngine.

var cacheTemplates = mustCacheTemplates()

func mustCacheTemplates() bool {
    v := strings.ToLower(os.Getenv("CACHE_TEMPLATES")) // YAML converts "true" to "True".
    switch v {
    case "", "false":
        return false
    case "true":
        return true
    default:
        log.Fatalf("bad cache templates: %s, want true or false", v)
    }
    return false
}    
    

Notes:

Getting Down to Business with main

OK! Now, in main, we use the standard library's net/http package to route URLs to handler functions and then launch the server on a port.

func main() {
    ctx := context.Background()

    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }

    log.Printf("cacheTemplates: %t", cacheTemplates)

    http.HandleFunc("/", templateFSHandler)
    http.HandleFunc("/favicon.ico", faviconHandler)
    http.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.Dir("static"))))

    log.Printf("serving on %s", port)
    log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}

func faviconHandler(w http.ResponseWriter, r *http.Request) {
    http.ServeFile(w, r, "static/favicon.ico")
}
    

Notes:

The Meat and Potatoes: Template File System Handler

templateFSHandler is the handler that allows me to easily create HTML files, drop them into a folder, and start serving them at a corresponding URL. It maps request URLs to templates on the server's file system and executes them. No code modifications needed! Let's check out the helper function, requestedTemplate, which does the mapping part before templateFSHandler calls Execute on it.

func requestedTemplate(r *http.Request) (*template.Template, error) {
    filepath := "templates" + r.URL.Path
    if strings.HasSuffix(filepath, "/") {
        filepath = path.Join(filepath, "index.html")
    }

    if cacheTemplates {
        if t, ok := filepath2Templates[filepath]; ok {
            return t, nil
        }
    }

    t, err := template.ParseFiles(filepath, "templates/base.html")
    if err != nil {
        return nil, err
    }

    if cacheTemplates {
        filepath2Templates[filepath] = t
        log.Printf("cached template: %s, num cached: %d", filepath, len(filepath2Templates))
    }

    return t, nil
}
    

Notes:

Finish Line: Running Locally and Deploying to AppEngine!

Let's run the app locally and deploy it on AppEngine. First, here is the folder structure and files on my local workstation:

Next, here's the command to run things locally. I run it in the same directory as main.go:

go generate ./... && go build -v -x && ./APP_NAME

Finally, here is what I do to deploy to AppEngine... and it works!

gcloud app deploy

In conclusion, this is just one example of how to write a simple static site on the new Second Generation AppEngine Runtime using Go 1.11. You can do whatever you write like maybe storing content in a DB... Oh, the possibilities are endless! Good luck!

Full Code

app.yaml

# https://cloud.google.com/appengine/docs/standard/go111/config/appref

runtime: go111

env_variables:
    PAYPAL_ENVIRONMENT: production
    CACHE_TEMPLATES: true

handlers:
- url: /.*
    script: auto
    secure: always
    redirect_http_response_code: 301
    

main.go

// This server can run on App Engine.
package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "os"
    "path"
    "strings"
    "text/template"
    "time"

    "gitlab.com/btmura/kittyhawk/github"
    "gitlab.com/btmura/kittyhawk/gitlab"
)

var paypalEnvironment = mustPaypalEnvironment()

// mustPaypalEnvironment returns a valid Paypal environment or dies.
// This is used to fail early at startup, because getting paid is important!
func mustPaypalEnvironment() string {
    v := strings.ToLower(os.Getenv("PAYPAL_ENVIRONMENT"))
    switch v {
    case "", "sandbox":
        return "sandbox"
    case "production":
        return "production"
    default:
        log.Fatalf("bad paypal environment: %s, want sandbox or production", v)
    }
    return ""
}

var cacheTemplates = mustCacheTemplates()

func mustCacheTemplates() bool {
    v := strings.ToLower(os.Getenv("CACHE_TEMPLATES")) // YAML converts "true" to "True".
    switch v {
    case "", "false":
        return false
    case "true":
        return true
    default:
        log.Fatalf("bad cache templates: %s, want true or false", v)
    }
    return false
}

var (
    cachedPrivateCommitCount *int
    cachedPublicCommitCount  *int
)

var (
    privateCommitCount = gitlab.CommitCount
    publicCommitCount  = github.CommitCount
)

var filepath2Templates = map[string]*template.Template{}

func main() {
    ctx := context.Background()

    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }

    log.Printf("paypalEnvironment: %s", paypalEnvironment)
    log.Printf("cacheTemplates: %t", cacheTemplates)

    t := time.NewTicker(1 * time.Hour)
    defer t.Stop()
    go func() {
        refresh := func() {
            privc, err := privateCommitCount(ctx)
            if err != nil {
                log.Printf("privateCommitCount: %v", err)
                return
            }
            cachedPrivateCommitCount = &privc

            pubc, err := publicCommitCount(ctx)
            if err != nil {
                log.Printf("publicCommitCount: %v", err)
                return
            }
            cachedPublicCommitCount = &pubc

            log.Printf("private: %d, public: %d", *cachedPrivateCommitCount, *cachedPublicCommitCount)
        }

        refresh()

        for t := range t.C {
            log.Printf("refreshing commit counts at %v", t)
            refresh()
        }
    }()

    http.HandleFunc("/", templateFSHandler)
    http.HandleFunc("/favicon.ico", faviconHandler)
    http.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.Dir("static"))))

    log.Printf("serving on %s", port)
    log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}

func templateFSHandler(w http.ResponseWriter, r *http.Request) {
    handleErr := func(tag string, err error) {
        var code int
        switch {
        case os.IsNotExist(err):
            code = http.StatusNotFound
        default:
            code = http.StatusInternalServerError
        }
        log.Printf("[%d] %s: %s", code, tag, err)
        w.WriteHeader(code)
    }

    t, err := requestedTemplate(r)
    if err != nil {
        handleErr("requestTemplate", err)
        return
    }

    if cachedPrivateCommitCount == nil {
        c, err := privateCommitCount(r.Context())
        if err != nil {
            handleErr("privateCommitCount", err)
            return
        }
        cachedPrivateCommitCount = &c
    }

    if cachedPublicCommitCount == nil {
        c, err := publicCommitCount(r.Context())
        if err != nil {
            handleErr("publicCommitCount", err)
            return
        }
        cachedPublicCommitCount = &c
    }

    // Don't show a negative diff count...
    diff := *cachedPrivateCommitCount - *cachedPublicCommitCount
    if diff < 0 {
        diff = 0
    }

    args := struct {
        PrivateCommitCount int
        PublicCommitCount  int
        CommitCountDiff    int
        PaypalEnvironment  string
    }{
        PrivateCommitCount: *cachedPrivateCommitCount,
        PublicCommitCount:  *cachedPublicCommitCount,
        CommitCountDiff:    diff,
        PaypalEnvironment:  paypalEnvironment,
    }

    if err := t.Execute(w, args); err != nil {
        handleErr("t.Execute", err)
        return
    }
}

func requestedTemplate(r *http.Request) (*template.Template, error) {
    filepath := "templates" + r.URL.Path
    if strings.HasSuffix(filepath, "/") {
        filepath = path.Join(filepath, "index.html")
    }

    if cacheTemplates {
        if t, ok := filepath2Templates[filepath]; ok {
            return t, nil
        }
    }

    t, err := template.ParseFiles(filepath, "templates/base.html")
    if err != nil {
        return nil, err
    }

    if cacheTemplates {
        filepath2Templates[filepath] = t
        log.Printf("cached template: %s, num cached: %d", filepath, len(filepath2Templates))
    }

    return t, nil
}

func faviconHandler(w http.ResponseWriter, r *http.Request) {
    http.ServeFile(w, r, "static/favicon.ico")
}
    

Support

Donate to keep development chugging (992 commits) or just star the repository like 15 others!

bitcoin:38vo2oWYmqBUXCxL3avpueye6dPRahX7gC

© 2019 Brian Muramatsu