jblog has moved.

Having read Free as in Freedom ¹ and being appalled at the digitalization-driven annihilation of once fought for values paired with de-facto government-mandated acceptance of this new brutal reality, I concluded to no longer be morally capable of supporting the web in its current form and thus began work on my own blogging software in April of 2020.
Now, beginning of September 2020, this project has reached a presentable state; having transferred all 232 jblog posts, reformatting and annotating them as well as improving minor syntactically or grammaticallly uneven sections, I can announce my blog’s movement:

https://www.jfrech.com/blog

If you choose to visit my blog, I can assure you that no trackers, cookies or malicious scripts will be employed against you. I will not try to collect a lot of data about your visit and the few inherently technically required data points in form of non-aggregated system logs will not be shared with third parties.

This 233rd post is the last to be published on https://jonathanfrech.wordpress.com, which will be shut down on the 31st of December 2020.
If you are interested in my blog, you can use my RSS feed to get a full list of my blog posts, updated as new are written. RSS is not like other subscription models — there is no server which knows your details and eagerly sends traffic your way to coerce you to a click. RSS requires you, the user, to request a feed and only then will transmit said feed.

Note that this blog’s content is subject to a clearly defined set of licenses specifying the terms and conditions with which it can be used.

¹) Williams, Sam: Free as in Freedom ; Richard Stallman’s crusade for free software. Sebastopol, CA: O’Reilly, 2002.

Setting up a https backend using Go and Let’s Encrypt

I recently needed to set up a dynamic web backend to both serve dynamically generated files as well as HTTP POST forms. Thus, I thought long and deeply of a solution that is both modern and versatile yet hassle-free to set up — since my systems administration skills are rather sparse.
The solution I found is to write a Go webserver; implicitly concurrent and enabling a high level of control.

Of course, nowadays you cannot run a (esp. dynamic) website without https; every browser worth its salt will display potential visitors a message framing you as a ruthless criminal. Fortunately, Let’s Encrypt gifts you the required bits: an SSL certificate!
Since I need a URL for the upcoming snippet and am personally thinking of switching my own homepage from a webserver to a self-hosted VPS setup, I chose to use http://www.jfrech.com as an example domain herein.

% # ... installing certbot (for example via `apt update && apt install certbot`) ...
% certbot certonly --standalone --preferred-challenges http -d www.jfrech.com
% # ... cli certbot interaction ...

% # ... installing Go ... (for example via `apt install golang`) ...

% # ... periodically renewing certificates ...

Once certbot has issued a certificate (the URL has to have a DNS entry linked to the current VPS and port 80 has to be unoccupied), incorporating it into the server is made straight forward by http.ListenAndServeTLS:

func handle(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintln(w, `<!doctype html><html><body><p>ip <code>` + template.HTMLEscapeString(r.RemoteAddr) + `</code> requesting <code>` + template.HTMLEscapeString(r.URL.Path) + `</code></p></body></html>`)
}
func main() {
    http.HandleFunc("/", handle)
    log.Fatal(http.ListenAndServeTLS(":443", "/etc/letsencrypt/live/www.jfrech.com/fullchain.pem", "/etc/letsencrypt/live/www.jfrech.com/privkey.pem", nil))
}

Since now the server only listens on port 443, one can add a http-to-https redirection on port 80:

func handleHTTP(w http.ResponseWriter, r *http.Request) {
    target := "https://" + r.Host + r.URL.Path
    if len(r.URL.RawQuery) > 0 {
        target += "?" + r.URL.RawQuery
    }
    http.Redirect(w, r, target, http.StatusTemporaryRedirect)
}
func main() {
    http.HandleFunc("/", handle)
    go http.ListenAndServe(":80", http.HandlerFunc(handleHTTP))
    log.Fatal(http.ListenAndServeTLS(":443", "/etc/letsencrypt/live/www.jfrech.com/fullchain.pem", "/etc/letsencrypt/live/www.jfrech.com/privkey.pem", nil))
}

One advantage of writing one’s one web server is full control of how the website behaves. One implication, which can be seen as a downside, is that you have to do everything; even the most basic of logging. However, writing your own log files enables you to confidently follow the logging policy you employ.
From a security standpoint, you can exactly control which files are served and which result in a 404 response; avoiding accidently exposing the whole server’s directory structure for the world to see.

Something quite amusing about seeing every http request to a publicly available URL are the attempts of information or identity theft; only running a freshly registered URL (with fresh DNS entries to a fresh VPS), I got URL requests from all over the world (geolocations were deduced from the ips using dbip, yet are not shown):

  • /manager/html
  • /cgi-bin/ViewLog.asp
  • /solr/admin/info/system?wt=json
  • /?XDEBUG_SESSION_START=phpstorm
  • /?a=fetch&content=die(@md5(HelloThinkCMF))
  • /index.php?s=/Index/\\think\\app/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP
  • /api/jsonws/invoke
  • //httpbin.org:443
  • //g.alicdn.com:443
  • //sm.bdimg.com:443
  • /muieblackcat
  • //phpMyAdmin/scripts/setup.php
  • //phpmyadmin/scripts/setup.php
  • //pma/scripts/setup.php
  • //myadmin/scripts/setup.php
  • //MyAdmin/scripts/setup.php
  • /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php
  • /olux.php
  • /.git/HEAD
  • /cgi-bin/mainfunction.cgi
  • /cgi-bin/ViewLog.asp
  • /UPnP/IGD.xml
  • /config/getuser?index=0

Especially the attempt to sniff out all git repositories on a server is scary, since it is known to work on a lot of smaller git servers.
Not being a system administrator, I cannot say much to most of the request listed above other than to be happy knowing that my Go server responds to each one with a 404 page.

The full server’s source code only contains the bare minimum of functionality. This was a conscious design decision; logging, 404 page serving (ref. w.WriteHeader(http.StatusNotFound)) and general behavior have to be built on top, maximizing its overall utility.
Full source code: setting-up-a-https-backend-using-go-and-lets-encrypt.go

Sources

    • Alan A. A. Donovan, Brain W. Kernighan: The Go Programming Language. New York: Addison-Wesley, 2016.
    • Using certbot in standalone-mode: DigitalOcean [2020-08-08]
    • An autocert example: blog.kowalcyk [2020-08-09]
    • Redirecting http requests: d-schmidt’s gist [2020-08-08]
    • Open git servers: c’t article [2020-08-08]