| Title: Reduce httpd web server bandwidth usage by serving compressed | |
| files | |
| Author: Solène | |
| Date: 22 April 2022 | |
| Tags: openbsd selfhosting nocloud | |
| Description: The OpenBSD httpd web server received a new feature | |
| allowing to serve precompressed files to save bandwidth. | |
| # Introduction | |
| When reaching a website, most web browsers will send a header (some | |
| metadata about the requestion) informing the web server that you | |
| supported compressed content. In OpenBSD 7.1, the httpd web server | |
| received a new feature allowing it to serves a pre-compressed file of a | |
| requested file if the web browser supports compression. The benefits | |
| are a bandwidth usage reduced by 2x to 10x depending on the file | |
| content, this is particularly interesting for people who self-host and | |
| for high traffic websites. | |
| # Configuration | |
| In your httpd.conf, in a server block add the "gzip-static" keyword, | |
| save the file and reload the httpd service. | |
| A simple server block would look like this: | |
| ```httpd.conf configuration file | |
| server "perso.pw" { | |
| root "/htdocs/solene" | |
| listen on * port 80 | |
| gzip-static | |
| } | |
| ``` | |
| # Creating the files | |
| In addition to this change, I added a new flag to the gzip command to | |
| easily compress files while keeping the original files. Run "gzip -k" | |
| on the files you want to serve compressed when the clients support the | |
| feature. | |
| It's best to compress text files, such as HTML, JS or CSS for the most | |
| commons. Compressing binary files like archives, pictures, audio or | |
| videos files won't provide any benefit. | |
| # How does it work? | |
| When the client connects to the httpd server requesting "foobar.html", | |
| if gzip-static is used for this location/server, httpd will look for a | |
| file named "foobar.html.gz" that is not older than "foobar.html". When | |
| found, "foobar.html.gz" is transparently transferred to the client | |
| requesting "foobar.html". | |
| Take care to regenerate the gz files when you update the original | |
| files, remember that the gz files must be newer to be used. | |
| # Conclusion | |
| This is for me a major milestone for using httpd in self-hosting and | |
| with static websites. We battle tested this change with the webzine | |
| server often hitting big news websites leading to many people visiting | |
| the website in a short time span, this drastically reduced the | |
| bandwidth usage of the server, allowing it to support more clients per | |
| second. |