This is a text-only version of the following page on https://raymii.org:

This is a text-only version of the following page on https://raymii.org:
---
Title : OpenVMS 9.2 for x86, Installing HAProxy and troubleshooting UNIX file paths
Author : Remy van Elst
Date : 19-04-2023 22:30
URL : https://raymii.org/s/blog/OpenVMS_9.2_for_x86_Installing_HAProxy.html
Format : Markdown/HTML
---

![openvms logo][1]

This article shows you how to install HAProxy on OpenVMS 9.2 for x86. I've often used HAProxy in my career as a sysadmin and find it a very useful tool. HAProxy is an open source, fast, reliable load balancer for TCP and HTTP-based applications. This guide assumes you've set up your OpenVMS system via [my guide](https://raymii.org/s/blog/OpenVMS_9.2_for_x86_Getting_Started.html) and the [second part](https://raymii.org/s/blog/OpenVMS_9.2_for_x86_Getting_Started_part_2.html) of my guide, that will give you a fully licensed OpenVMS installation with networking and SSH access. Since I've used HAProxy so very often to set up high-available clusters and load balancers, I was surprised but happy to see it ported to OpenVMS. This guide shows the setup but also a few OpenVMS specific quirks, like file paths and troubleshooting error messages / logs.

[You can read all my OpenVMS articles here][2].

<p class="ad"> <b>Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:</b><br><br> <a href="https://leafnode.nl">I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!</a><br><br> <a href="https://github.com/sponsors/RaymiiOrg/">Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.</a><br><br> <a href="https://www.digitalocean.com/?refcode=7435ae6b8212">You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $100 credit for 60 days. </a><br><br> </p>

[HAProxy][6] is an open source, fast, reliable load balancer for
TCP and HTTP-based applications. It is particularly suited for high traffic web
sites and powers quite a number of the world's most popular sites. It is
arguably the de-facto standard Open Source load balancer, shipping with most
mainstream Linux distributions, and often deployed by default in cloud
platforms.

This OpenVMS port of the HAProxy includes all core functionality provided by the
Open Source release, including SSL/TLS support. SSL/TLS support is statically
linked into the HAProxy image and uses OpenSSL 1.0.2u

The current release of HAProxy for OpenVMS is based on the HAProxy 1.7.9
distribution.

This guide covers installation and a few OpenVMS specific quirks regarding
files and line endings.

This is a screenshot of HAProxy running on OpenVMS, showing the built-in
statistics page:

![haproxy stats][9]

### Installing HAProxy on OpenVMS

Make sure you've set up your OpenVMS installation as I described in my
guides, [part 1 regarding installation][3] and [part 2 regarding networking][4].

Please also read [part 3 which sets up unzip and shows software installation][5].

Login to the [Software Portal][7] and download HAProxy, filename is `VSI-X86VMS-HAPROXY-V0107-9A-1.ZIP`.

Create a folder on your OpenVMS system where packages will be stored:

CREATE /DIRECTORY /VERSION_LIMIT=2 DKA0:[SW]

Go into that folder:

SET DEFAULT DKA0:[SW]

Copy over the `VSI-X86VMS-HAPROXY-V0107-9A-1.ZIP` file to your OpenVMS
system in the (newly created) `sw` folder:

$ scp VSI-X86VMS-HAPROXY-V0107-9A-1.ZIP [email protected]:/SW/
[email protected]'s password:
VSI-X86VMS-HAPROXY-V0107-9A-1.ZIP 100% 6538KB 1.6MB/s 00:03

Read [part 3 of my guide][5] to learn more about the "foreign command"
for `unzip`. If you haven't read it, execute this line otherwise
`unzip` won't work:

unzip :== $SYS$COMMON:[SYSHLP.UNSUPPORTED.UNZIP]UNZIP.EXE

Unzip the installation package:

unzip VSI-X86VMS-HAPROXY-V0107-9A-1.ZIP

Output:

Archive: DKA0:[SW]VSI-X86VMS-HAPROXY-V0107-9A-1.ZIP;1
[...]
**********************************************
* VSI HAProxy Version 1.7-9A-1 for VSI x86_64*
* Version E9.2 Release Notes are included in*
* the ZIP file. *
**********************************************
inflating: HAPROXY-1_7_9A-X86-RELEASE-NOTES.PDF
inflating: VSI-X86VMS-HAPROXY-V0107-9A-1.PCSI$COMPRESSED
extracting: VSI-X86VMS-HAPROXY-V0107-9A-1.PCSI$COMPRESSED_VNC
inflating: Manifest.txt

Start the installation:

product install haproxy

Output:

Performing product kit validation of signed kits ...
%PCSI-I-VSIVALPASSED, validation of DKA0:[SW]VSI-X86VMS-HAPROXY-V0107-9A-1.PCSI$COMPRESSED;1 succeeded

The following product has been selected:
VSI X86VMS HAPROXY V1.7-9A Layered Product

Do you want to continue? [YES]

Confirm with `ENTER`. Installation starts:

Configuration phase starting ...

You will be asked to choose options, if any, for each selected product and for
any products that may be installed to satisfy software dependency requirements.

Configuring VSI X86VMS HAPROXY V1.7-9A: HAProxy for OpenVMS is based on HAProxy Version 1.7.9

Copyright 2022 VMS Software Inc.

VSI Software Inc.

* This product does not have any configuration options.

Execution phase starting ...

The following product will be installed to destination:
VSI X86VMS HAPROXY V1.7-9A DISK$X86SYS:[VMS$COMMON.]

Portion done: 0%...90%

When the installation is finished you will see the following output:

The following product has been installed:
VSI X86VMS HAPROXY V1.7-9A Layered Product

VSI X86VMS HAPROXY V1.7-9A: HAProxy for OpenVMS is based on HAProxy Version 1.7.9

Post-installation tasks are required.

To start HAProxy at system boot time, modify the HAProxy
configuration file as necessary and add the following lines
to SYS$MANAGER:SYSTARTUP_VMS.COM:

$ file := SYS$STARTUP:HAPROXY$STARTUP.COM
$ if f$search("''file'") .nes. "" then @'file'

To shutdown HAProxy at system shutdown, add the following lines
to SYS$MANAGER:SYSHUTDWN.COM:

$ file := SYS$STARTUP:HAPROXY$SHUTDOWN.COM
$ if f$search("''file'") .nes. "" then @'file'

To make `HAProxy` run at startup, use the editor to add those lines to the file
as stated:

EDIT SYS$MANAGER:SYSTARTUP_VMS.COM

Almost at the bottom of the file, before the `EXIT` line, insert the following:

$ file := SYS$STARTUP:HAPROXY$STARTUP.COM
$ if f$search("''file'") .nes. "" then @'file'

Save with `CTRL+Z`.

### Configuring HAProxy on OpenVMS

The configuation file is a regular HAProxy configuration file. The only thing
that is a bit different is the file paths, but we'll get to that. Here is
a basic example showing an example of `HAProxy` in TCP mode, load balancing
between two MQTT servers, on port `1883` and the statistics page, accessible
on port `8100`, url `/stats`. Normally you'd put those statistics on a private
IP behind a password because it allows you to turn off servers and change settings,
but for a hobbyist setup that is not required. The IP of the OpenVMS server is
`192.168.1.23`.

Edit the configuration file:

EDIT sys$startup:haproxy.cfg

Contents:

global
log 192.168.1.23 local0
log 192.168.1.23 local1 notice
maxconn 256

defaults
log global
retries 3
maxconn 256
timeout connect 5s
timeout client 50s
timeout server 50s

listen mqtt
bind :1883
balance
mode tcp
log global
retries 3
server mqtt1 192.168.1.210:1883 check
server mqtt2 192.168.1.220:1883 check

listen statictics
bind :8100
mode http
option httplog
stats enable
stats uri /stats
stats refresh 5s

To test the configuration before (re)starting the service, execute the following
commands. The first makes `haproxy` a "forgein command" allowing it to execute
with arguments, the seconds starts `haproxy` with our config file:

haproxy :== $sys$system:haproxy.exe
haproxy "-f" "/sys$startup/haproxy.cfg"

You should be able to visit the statistics page now, or even, if you have
the backend servers running, use them:

![haproxy stats][9]

HTTP forwarding is possible as well:

listen site1
bind :80
mode http
server http1 192.168.1.110:80 maxconn 32
server http2 192.168.1.210:80 maxconn 32

If you want certificates or HTTPS, read on.

### SSL, OpenVMS file paths, unix style and line endings

One of the things I like to do is have custom error pages
in the case of haproxy errors. This is supported by creating a file with
the entire HTTP response. Not just HTML, but also headers and stuff like
`HTTP/1.1 404 Not Found`.

This turned out to be a nice learning experience regarding OpenVMS, log files,
ported UNIX software and logical names. Join me in the experience.

I added a `listen` block forwarding port `80` to another server with a `errorfile`
directive:

listen site1
bind :80
mode http
errorfile 404 /dka0/haproxy/404.http
server http1 192.168.1.110:80 maxconn 32

I also created a folder for the files:

create /dir DKA0:[HAPROXY]
set def DKA0:[HAPROXY]

I then added the following contents to the file:

EDIT 404.http

Contents:

HTTP/1.1 404 Not Found
Cache-Control: no-cache
Connection: close
Content-Type: text/html

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>404 Not Found</title>
</head>
<body>
<main>
<h1>404 Not Found</h1>

This is my custom 404 Not Found page, for HAProxy on OpenVMS! <br>
Visit <a href="https://raymii.org>Raymii.org</a>.
</main>
</body>
</html>

Remember the file naming scheme is different on OpenVMS:

node$device:[root.][directory]file-name.file-type;version

The file separator is the period (`.`), not the slash (`/`).

However, starting HAProxy failed with the following error:

$ haproxy "-f" "/sys$startup/haproxy.cfg"
[ALERT] 105/222650 (1069) : parsing [/sys$startup/haproxy.cfg:35] : error reading file </dka0/haproxy/404.http> for custom error message <404>.
[ALERT] 105/222650 (1069) : Error(s) found in configuration file : /sys$startup/haproxy.cfg
[ALERT] 105/222650 (1069) : Fatal errors found in configuration.

At first I thought that it couldn't find the file. File paths on OpenVMS are
way different than on UNIX / Linux systems. I looked at a few `CWSD` (Apache
for OpenVMS) manuals and config files and that seemed to be the correct syntax.

I also didn't start haproxy with the command above, I just stopped and started
the service, then refreshed the web browser. To figure out the correct command,
I took a look at the startup script:

type SYS$STARTUP:HAPROXY$Startup.com

Output:

$ ! HAPROXY$STARTUP.COM
$ !+
$ ! 24-Mar-2022
$ !-
$
$ set noon
$
$ run/detach -
/input=sys$startup:haproxy$run.com/output=sys$manager:haproxy.log -
/process_name="HAProxy" -
/authorize sys$system:loginout.exe

That points to the following file for startup: `sys$startup:haproxy$run.com`
and this file for logs `sys$manager:haproxy.log`. The startup file contains
the haproxy command I used above:

type sys$startup:haproxy$run.com

Output:

$ set verify
$
$ haproxy :== $sys$system:haproxy.exe
$ haproxy "-f" "/sys$startup/haproxy.cfg"
$
$ exit

The log file contaied the following:

$ Set NoOn
$ VERIFY = F$VERIFY(F$TRNLNM("SYLOGIN_VERIFY"))
$
$ haproxy :== $sys$system:haproxy.exe
$ haproxy "-f" "/sys$startup/haproxy.cfg"
[ALERT] 108/193349 (1060) : parsing [/sys$startup/haproxy.cfg:35] : error opening file </dka0/haproxy/404.http> for custom error message <404>.
[ALERT] 108/193349 (1060) : Error(s) found in configuration file : /sys$startup/haproxy.cfg
[ALERT] 108/193349 (1060) : Fatal errors found in configuration.
$
$ exit
SYSTEM job terminated at 16-APR-2023 20:55:20.95

Accounting information:
Buffered I/O count: 581 Peak working set size: 16416
Direct I/O count: 29 Peak virtual size: 249088
Page faults: 736 Mounted volumes: 0
Charged CPU time: 0 00:00:00.26 Elapsed time: 0 00:00:00.28

I tried all different kinds of syntaxes for the error file:

- `/sys$sysdisk/haproxy/404.http`
- `/dka0/haproxy/404.http`
- `/sys$disk/haproxy/404.http`
- `dka0:[haproxy]404.http` (didn't work at all)

I even defined a few [Logical Names][13], sort of symlinks but then for an
entire filesystem, specific to OpenVMS:

define haproxy DKA0:[HAPROXY]

Then checking that new logical name:

dir haproxy

Output:

Directory DKA0:[HAPROXY]

404.HTTP;1 MAINTENANCE.HTML;1

Total of 2 files.

Using `/haproxy/404.http` in the haproxy configuration gave the same error,
no matter what logical name I used.

I found the following command on stackoverflow, which shows all the file-related
stuff an OpenVMS program does. Reminds me of `strace` on Linux.

set watch file/class=(all,nodump)

It outputs a lot of logs when running a program:

haproxy "-f" "/sys$startup/haproxy.cfg"
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (13,1,0): Access requested: 00000004, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read only directory access (13,1,0)
%XQP, Thread #0, Directory scan for: HAPROXY.EXE;0, Status: 00000000
%XQP, Thread #0, Access (0,0,0) Status: 00000910
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (16,1,0): Access requested: 00000004, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read only directory access (16,1,0)
%XQP, Thread #0, Directory scan for: HAPROXY.EXE;0, Status: 00000001
%XQP, Thread #0, Alternate access requested: 00000001, Required: 0
%XQP, Thread #0, File protection (11728,39373,0): Access requested: 00000005, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read attributes: Access mode haproxy.exe;1 (11728,39373,0)
%XQP, Thread #0, Read attributes: Owner UIC haproxy.exe;1 (11728,39373,0)
%XQP, Thread #0, Read attributes: Header 1 accessibility haproxy.exe;1 (11728,39373,0)
%XQP, Thread #0, Read attributes: File protection haproxy.exe;1 (11728,39373,0)
[...]
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (4581,1,0): Access requested: 00000004, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read only directory access (4581,1,0)
%XQP, Thread #0, Directory scan for: MESSAGES.;0, Status: 00000000
%XQP, Thread #0, Directory scan for: MESSAGES.DIR;1, Status: 00000000
%XQP, Thread #0, Lookup (0,0,0) Status: 00000910
[...]
%XQP, Thread #0, Directory scan for: HAPROXY.CFG;0, Status: 00000000
%XQP, Thread #0, Lookup (0,0,0) Status: 00000910
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (20,1,0): Access requested: 00000004, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read only directory access (20,1,0)
%XQP, Thread #0, Directory scan for: HAPROXY.CFG;0, Status: 00000001
%XQP, Thread #0, Lookup (11744,2,0) Status: 00000001
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (11744,2,0): Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read attributes: Creation date haproxy.cfg;17 (11744,2,0)
%XQP, Thread #0, Read attributes: Revision date haproxy.cfg;17 (11744,2,0)
%XQP, Thread #0, Read attributes: Record attributes haproxy.cfg;17 (11744,2,0)
%XQP, Thread #0, Read attributes: Owner UIC haproxy.cfg;17 (11744,2,0)
%XQP, Thread #0, Read attributes: File protection haproxy.cfg;17 (11744,2,0)
%XQP, Thread #0, Read attributes: User file characteristics haproxy.cfg;17 (11744,2,0)
%XQP, Thread #0, Read attributes: hardlink count haproxy.cfg;17 (11744,2,0)
%XQP, Thread #0, Lookup haproxy.cfg;17 (11744,2,0) Status: 00000001

As you can see, it does a `directory scan` for `haproxy.cfg`. It does not list
the directory is searches in however. In the output I can see that it does try to
access my error file, `404.http`:

%XQP, Thread #0, Control function (11744,2,0) Status: 00000001
%XQP, Thread #0, Final status: 1C000870
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (11566,4,0): Access requested: 00000004, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read only directory access (11566,4,0)
%XQP, Thread #0, Directory scan for: 404.HTTP;0, Status: 00000001
%XQP, Thread #0, File protection (11570,2,0): Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read attributes: Access mode 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Creation date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Expiration date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Backup date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Last access date/time 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Last attribute update date/time 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Data modification date/time 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Revision date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: ASCII dates 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Access mode 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Journal flags 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: RU active 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Statistics block 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Find ACE by type 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Record attributes 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: User file characteristics 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: File length hint field 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Symlink meta-data 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Access 404.HTTP;1 (11570,2,0) Status: 00000001

So it could access the file (I used the filename `/dka0/haproxy/404.http`).

Debugging further got me nowhere, permissions or other rabbit holes. But
then I thought of one thing, which are SSL certificates.

#### SSL Certificates in HAProxy on OpenVMS

I do know of one other way to get HAProxy to read files, which is by using
SSL certificated. I generated a standard HAProxy SSL certificate (which
is a private key and a cert plus a chain contatenated in PEM format). I
copied over that file and placed the following in my config:

listen site1
bind :443 ssl crt /dka0/haproxy/mydomain.pem
mode http
server http1 192.168.1.120:443 maxconn 32

Starting HAProxy with this config file gave me no errors whatsoever, in a browser
I could visit the site and get the correct certificate. So then I started thinking,
why can it read an SSL certificate but not an HTTP Error file, in the same folder?
It wasn't the file path syntax that was incorrect, it must be something else...

I went back to [the haproxy manual on errorfile][10] for this version and one
thing stood out to me:

> For better HTTP compliance, it is recommended that all header lines end with CR-LF and not LF alone.

Back [in 2018 when I was involved with the AXPBox Alpha emulator][11] I wrote
a few articles on OpenVMS and [one specific article involved line endings][11].
That article links to [the OpenVMS wizard][12] which explains more than I can do
here. I tried the following command to convert the line endings:

SET FILE/ATTRIBUTE=(RFM=STMLF) 404.http

After which I started HAProxy again and to my big surprise, this time, no error
message regarding the `errorfile`! It was line endings all along. The error
message it logged wasn't `error ACCESSING` the file, it clearly said, `error READING`
the file. The path variable was correct all along, it was the file contents
that were wrong!

I hope you enjoyed this trip down the rabbit hole of debugging in OpenVMS. We
looked into the startup script to find the actual executable and the location
of the log files. We looked at different ways to define a file path in a
config file that expects UNIX style file paths and we debugged file access
with a hidden undocumented command. In the end it wasn't an error in the file
path syntax, but in the file contents, namely the line endings.

[1]: /s/inc/img/Vms-arrow-logo.jpg
[2]: /s/tags/openvms.html
[3]: /s/blog/OpenVMS_9.2_for_x86_Getting_Started.html
[4]: /s/blog/OpenVMS_9.2_for_x86_Getting_Started_part_2.html
[5]: /s/blog/OpenVMS_9.2_for_x86_Getting_Started_part_3.html
[6]: http://www.haproxy.org/
[7]: https://sp.vmssoftware.com
[8]: https://web.archive.org/web/20230419185902/https://forum.vmssoftware.com/viewtopic.php?f=30&t=8622&sid=298dd4b426377beb658b1f240083363e
[9]: /s/inc/img/openvms-x86-part4-1.png
[10]: https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4-errorfile
[11]: /s/blog/OpenVMS_Mount_ISO_and_execute_scripts.html#toc_5
[12]: https://web.archive.org/web/20180507190424/http://h41379.www4.hpe.com/wizard/wiz_3707.html
[13]: https://web.archive.org/web/20230419200129/https://wiki.vmssoftware.com/Logical_Name

---

License:
All the text on this website is free as in freedom unless stated otherwise.
This means you can use it in any way you want, you can copy it, change it
the way you like and republish it, as long as you release the (modified)
content under the same license to give others the same freedoms you've got
and place my name and a link to this site with the article as source.

This site uses Google Analytics for statistics and Google Adwords for
advertisements. You are tracked and Google knows everything about you.
Use an adblocker like ublock-origin if you don't want it.

All the code on this website is licensed under the GNU GPL v3 license
unless already licensed under a license which does not allows this form
of licensing or if another license is stated on that page / in that software:

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.

Just to be clear, the information on this website is for meant for educational
purposes and you use it at your own risk. I do not take responsibility if you
screw something up. Use common sense, do not 'rm -rf /' as root for example.
If you have any questions then do not hesitate to contact me.

See https://raymii.org/s/static/About.html for details.