Assembly Headaches

By Jeremiah Stoddard on January 18, 2023

Just a mundane post about the standard dumb mistakes that virtually all
programmers make now and then. I left the software development business
some years ago, but have been doing some programming on some old 8-bit
systems for fun. Today I faced the 6502 assembly language equivalent of
forgetting a semicolon at the end of a C statement, and had some fun
troubleshooting.

I'm working on a little toy app that's supposed to go through some Dragon
Quest style battle routines in text mode, hopefully ultimately resulting
in a melee engine that can be used in a graphical RPG for the Apple IIe.
Nothing big and serious like Nox Archaist, just a little hobby project in
my spare time to do the type of game I wanted, but never managed, to make
as a kid.

Well yeah, battle routines. So I started by writing some assembly routines
to print out text to the screen, then turned it into a menu and a loop to
get user input and repeat until the user asks to 'Quit' the program. Since
a battle engine is the goal, at this point I should have put in a menu
entry to start a battle, like an arena-type game, I guess. But no, I
started thinking that the game is going to need sound effects, and
although I know accessing memory location $C030 "tweaks" the speaker, I
hadn't really tried to do anything with sound. So I naturally forgot about
the melee system and the first menu option other than 'Quit' was to play a
sound effect.

I actually got more or less the noise I wanted out of the speaker on the
first try. That's not what this post is about. The issue I had arose out
of some additional tweaks to the menu. I had set up something like this:

       ; menu display code
       ...
LOOP    JSR  RDKEY              ; read a character from the keyboard
       AND  #$5F               ; force to uppercase
       CMP  #$50               ; 'P' (for play sound)
       BEQ  PLYSND
       ...
       CMP  #$51               ; 'Q' (for quit)
       BNE  LOOP
       RTS
PLYSND  JSR  SUBROUTINE
       JMP  LOOP
       ...

At least, that's what I thought I had set up. This was written up in the
editor for the ORCA/M assembler. I assembled the file, then in the linking
stage the linker gave me an error along the lines of 'Relative address out
of range' after MAIN at location 67, program counter 2067, blah blah blah.

Because of the error, and because it was in the MAIN section, I knew it
had to be one of the BEQs or BNEs in the menu loop. I was confused,
though, since I glanced through the beginning of the code, counting in my
head the number of bytes I expected each instruction to take up, and 67
would land me in the middle of printing the menu. There are JSRs there,
but nothing that would use a relative address. Moreover, the branches in
the loop should have all been not more than a couple dozen bytes from
their destination, at most. Of course it didn't occur to me that I should
be counting in hexadecimal, not decimal. Anyway, I could check the
assembler listing and linker output easily enough, since the ORCA system
provides Unix-like output redirection:

# ASML TEST.ASM >OUTPUT.TXT

Sure enough, at address 2067 in the assembly listing was the BEQ after
checking for the 'P' keypress. The error was simple. Instead of branching
to the nearby label that JSRs to the sound SUBROUTINE, I was trying to
directly branch to the subroutine:

       BEQ  SUBROUTINE

Had that subroutine been close enough to the menu loop, there would have
been a tricky bug to find, since no return address would have been pushed
to the stack for the subroutine. I don't want to think about the headaches
that would have caused.