Setting up a https backend using Go and Let’s Encrypt

I recently needed to set up a dynamic web backend to both serve dynamically generated files as well as HTTP POST forms. Thus, I thought long and deeply of a solution that is both modern and versatile yet hassle-free to set up — since my systems administration skills are rather sparse.
The solution I found is to write a Go webserver; implicitly concurrent and enabling a high level of control.

Of course, nowadays you cannot run a (esp. dynamic) website without https; every browser worth its salt will display potential visitors a message framing you as a ruthless criminal. Fortunately, Let’s Encrypt gifts you the required bits: an SSL certificate!
Since I need a URL for the upcoming snippet and am personally thinking of switching my own homepage from a webserver to a self-hosted VPS setup, I chose to use http://www.jfrech.com as an example domain herein.

% # ... installing certbot (for example via `apt update && apt install certbot`) ...
% certbot certonly --standalone --preferred-challenges http -d www.jfrech.com
% # ... cli certbot interaction ...

% # ... installing Go ... (for example via `apt install golang`) ...

% # ... periodically renewing certificates ...

Once certbot has issued a certificate (the URL has to have a DNS entry linked to the current VPS and port 80 has to be unoccupied), incorporating it into the server is made straight forward by http.ListenAndServeTLS:

func handle(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintln(w, `<!doctype html><html><body><p>ip <code>` + template.HTMLEscapeString(r.RemoteAddr) + `</code> requesting <code>` + template.HTMLEscapeString(r.URL.Path) + `</code></p></body></html>`)
}
func main() {
    http.HandleFunc("/", handle)
    log.Fatal(http.ListenAndServeTLS(":443", "/etc/letsencrypt/live/www.jfrech.com/fullchain.pem", "/etc/letsencrypt/live/www.jfrech.com/privkey.pem", nil))
}

Since now the server only listens on port 443, one can add a http-to-https redirection on port 80:

func handleHTTP(w http.ResponseWriter, r *http.Request) {
    target := "https://" + r.Host + r.URL.Path
    if len(r.URL.RawQuery) > 0 {
        target += "?" + r.URL.RawQuery
    }
    http.Redirect(w, r, target, http.StatusTemporaryRedirect)
}
func main() {
    http.HandleFunc("/", handle)
    go http.ListenAndServe(":80", http.HandlerFunc(handleHTTP))
    log.Fatal(http.ListenAndServeTLS(":443", "/etc/letsencrypt/live/www.jfrech.com/fullchain.pem", "/etc/letsencrypt/live/www.jfrech.com/privkey.pem", nil))
}

One advantage of writing one’s one web server is full control of how the website behaves. One implication, which can be seen as a downside, is that you have to do everything; even the most basic of logging. However, writing your own log files enables you to confidently follow the logging policy you employ.
From a security standpoint, you can exactly control which files are served and which result in a 404 response; avoiding accidently exposing the whole server’s directory structure for the world to see.

Something quite amusing about seeing every http request to a publicly available URL are the attempts of information or identity theft; only running a freshly registered URL (with fresh DNS entries to a fresh VPS), I got URL requests from all over the world (geolocations were deduced from the ips using dbip, yet are not shown):

  • /manager/html
  • /cgi-bin/ViewLog.asp
  • /solr/admin/info/system?wt=json
  • /?XDEBUG_SESSION_START=phpstorm
  • /?a=fetch&content=die(@md5(HelloThinkCMF))
  • /index.php?s=/Index/\\think\\app/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP
  • /api/jsonws/invoke
  • //httpbin.org:443
  • //g.alicdn.com:443
  • //sm.bdimg.com:443
  • /muieblackcat
  • //phpMyAdmin/scripts/setup.php
  • //phpmyadmin/scripts/setup.php
  • //pma/scripts/setup.php
  • //myadmin/scripts/setup.php
  • //MyAdmin/scripts/setup.php
  • /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php
  • /olux.php
  • /.git/HEAD
  • /cgi-bin/mainfunction.cgi
  • /cgi-bin/ViewLog.asp
  • /UPnP/IGD.xml
  • /config/getuser?index=0

Especially the attempt to sniff out all git repositories on a server is scary, since it is known to work on a lot of smaller git servers.
Not being a system administrator, I cannot say much to most of the request listed above other than to be happy knowing that my Go server responds to each one with a 404 page.

The full server’s source code only contains the bare minimum of functionality. This was a conscious design decision; logging, 404 page serving (ref. w.WriteHeader(http.StatusNotFound)) and general behavior have to be built on top, maximizing its overall utility.
Full source code: setting-up-a-https-backend-using-go-and-lets-encrypt.go

Sources

    • Alan A. A. Donovan, Brain W. Kernighan: The Go Programming Language. New York: Addison-Wesley, 2016.
    • Using certbot in standalone-mode: DigitalOcean [2020-08-08]
    • An autocert example: blog.kowalcyk [2020-08-09]
    • Redirecting http requests: d-schmidt’s gist [2020-08-08]
    • Open git servers: c’t article [2020-08-08]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.