## Setting up a https backend using Go and Let’s Encrypt

I recently needed to set up a dynamic web backend to both serve dynamically generated files as well as HTTP POST forms. Thus, I thought long and deeply of a solution that is both modern and versatile yet hassle-free to set up — since my systems administration skills are rather sparse.
The solution I found is to write a Go webserver; implicitly concurrent and enabling a high level of control.

Of course, nowadays you cannot run a (esp. dynamic) website without https; every browser worth its salt will display potential visitors a message framing you as a ruthless criminal. Fortunately, Let’s Encrypt gifts you the required bits: an SSL certificate!
Since I need a URL for the upcoming snippet and am personally thinking of switching my own homepage from a webserver to a self-hosted VPS setup, I chose to use `http://www.jfrech.com` as an example domain herein.

```% # ... installing certbot (for example via `apt update && apt install certbot`) ...
% certbot certonly --standalone --preferred-challenges http -d www.jfrech.com
% # ... cli certbot interaction ...

% # ... installing Go ... (for example via `apt install golang`) ...

% # ... periodically renewing certificates ...```

Once certbot has issued a certificate (the URL has to have a DNS entry linked to the current VPS and port `80` has to be unoccupied), incorporating it into the server is made straight forward by `http.ListenAndServeTLS`:

```func handle(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, `<!doctype html><html><body><p>ip <code>` + template.HTMLEscapeString(r.RemoteAddr) + `</code> requesting <code>` + template.HTMLEscapeString(r.URL.Path) + `</code></p></body></html>`)
}
func main() {
http.HandleFunc("/", handle)
log.Fatal(http.ListenAndServeTLS(":443", "/etc/letsencrypt/live/www.jfrech.com/fullchain.pem", "/etc/letsencrypt/live/www.jfrech.com/privkey.pem", nil))
}```

Since now the server only listens on port `443`, one can add a http-to-https redirection on port `80`:

```func handleHTTP(w http.ResponseWriter, r *http.Request) {
target := "https://" + r.Host + r.URL.Path
if len(r.URL.RawQuery) > 0 {
target += "?" + r.URL.RawQuery
}
http.Redirect(w, r, target, http.StatusTemporaryRedirect)
}
func main() {
http.HandleFunc("/", handle)
go http.ListenAndServe(":80", http.HandlerFunc(handleHTTP))
log.Fatal(http.ListenAndServeTLS(":443", "/etc/letsencrypt/live/www.jfrech.com/fullchain.pem", "/etc/letsencrypt/live/www.jfrech.com/privkey.pem", nil))
}
```

One advantage of writing one’s one web server is full control of how the website behaves. One implication, which can be seen as a downside, is that you have to do everything; even the most basic of logging. However, writing your own log files enables you to confidently follow the logging policy you employ.
From a security standpoint, you can exactly control which files are served and which result in a `404` response; avoiding accidently exposing the whole server’s directory structure for the world to see.

Something quite amusing about seeing every http request to a publicly available URL are the attempts of information or identity theft; only running a freshly registered URL (with fresh DNS entries to a fresh VPS), I got URL requests from all over the world (geolocations were deduced from the ips using dbip, yet are not shown):

• `/manager/html`
• `/cgi-bin/ViewLog.asp`
• `/solr/admin/info/system?wt=json`
• `/?XDEBUG_SESSION_START=phpstorm`
• `/?a=fetch&content=die(@md5(HelloThinkCMF))`
• `/index.php?s=/Index/\\think\\app/invokefunction&function=call_user_func_array&vars[0]=md5&vars[1][]=HelloThinkPHP`
• `/api/jsonws/invoke`
• `//httpbin.org:443`
• `//g.alicdn.com:443`
• `//sm.bdimg.com:443`
• `/muieblackcat`
• `//phpMyAdmin/scripts/setup.php`
• `//phpmyadmin/scripts/setup.php`
• `//pma/scripts/setup.php`
• `//myadmin/scripts/setup.php`
• `//MyAdmin/scripts/setup.php`
• `/vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php`
• `/olux.php`
• `/.git/HEAD`
• `/cgi-bin/mainfunction.cgi`
• `/cgi-bin/ViewLog.asp`
• `/UPnP/IGD.xml`
• `/config/getuser?index=0`

Especially the attempt to sniff out all git repositories on a server is scary, since it is known to work on a lot of smaller git servers.
Not being a system administrator, I cannot say much to most of the request listed above other than to be happy knowing that my Go server responds to each one with a `404` page.

The full server’s source code only contains the bare minimum of functionality. This was a conscious design decision; logging, `404` page serving (ref. `w.WriteHeader(http.StatusNotFound)`) and general behavior have to be built on top, maximizing its overall utility.
Full source code: setting-up-a-https-backend-using-go-and-lets-encrypt.go

## Sources

• Alan A. A. Donovan, Brain W. Kernighan: The Go Programming Language. New York: Addison-Wesley, 2016.
• Using `certbot` in standalone-mode: DigitalOcean [2020-08-08]
• An `autocert` example: blog.kowalcyk [2020-08-09]
• Redirecting http requests: d-schmidt’s gist [2020-08-08]
• Open git servers: c’t article [2020-08-08]

## Web Sudoku Solver

In my last post I used Web Sudoku to get a Sudoku as an example for my solver.

After that I wanted to automate the process of looking up a Sudoku, solving it and typing it in. But while trying to get the Sudoku’s numbers, I noticed that the whole, solved Sudoku was stored in plaintext! (Look at this page‘s source code)
So I just needed to get that information, open the Web Sudoku page in a browser and type in the already solved Sudoku.

To accomplish said goal I used the python module urllib to get the Web Sudoku page’s source code and the module webbrowser to open the page in a browser. To type in the Sudoku I used AutoHotkey.

The finished program takes a level (easy, medium, hard or evil) and an id (the Sudoku’s identification number) to get a Sudoku, create an AHK file, execute it and open a web browser.
All you have to do is to click into the first box, press a key (F1 in this case) and the Sudoku gets solved! You then just need to wait a minute, which is the minimum time Web Sudoku wants you to take to solve a Sudoku, and the AHK script hits enter.
You can get really good times with this.

``````# Python 2.7.7 Code
# Jonathan Frech  9th of August, 2016
#         edited 10th of August, 2016
#         edited 11th of August, 2016``````

## Random Resource Locator

I once heard, that every domain containing three letters of the alphabet and ending with ‘.com’ was assigned to some web server. This would mean, all 17576¹ domains were used.
To test it, I wrote this little Python program.

#### Usage

• To use it, simply press enter to open a random page. Because you often will get redirected, the program will print out the url on the screen.

¹Any three letters of the twenty-six in the alphabet result in $26^3 = 17576$.

``````# Python 2.7.7 Code
# Jonathan Frech 18th of August, 2015``````