Commodore Maze Generator on AWS Lambda via API Gateway

The Commodore Maze Generator Program

I needed something “useful” to output from my Commodore VICE Lambda Custom Runtime that also didn’t require much input (if any) from the requester. Otherwise, I’d have to sanitize the input that is being passed to a shell script and potentially validate it if I didn’t want unpredictable behavior. I settled on a Commodore 128 version of the Commodore Maze Generator (the fast and slow will be unnecessary once we launch from the Lambda with the -warp option on VICE, but I never changed the program so it’s left in here. The fast and slow and everything before line 50 is Commodore 128 BASIC 7.0-specific, but the Commodore 64 version just requires substitution of POKE commands instead. (See vice-lambda repo on GitHub for latest version)

5 fast
10 scnclr
20 color 0,2
30 color 4,2
40 color 5,1
50 for y=1 to 25
60 for x=1 to 40
70 print chr$(205.5+rnd(1));
80 next x
90 next y
95 slow
100 goto 100

Making sure that the Lambda returns an image for Lambda

I want to be able to use Lambda Proxy Integration when setting up the API Gateway, so I need to return a format that maps the information properly. The response JSON needs to have:

  • isBase64Encoded set to true (necessary for the translation of the PNG back from base64)
  • headers with "Content-type" and "content-disposition" set (image/png and inline for this example)
  • statusCode of 200 (right now, if we have an error it’s either not caught or happens at the Lambda invocation level)
  • body containing the base64 data. Note the -w 0 argument to turn off line breaks when wrapping… Lambda Proxy Integration doesn’t handle that well.
function handler() {
  cd vice
  SEED=$((0 - `od -An -N2 -i /dev/random`))
  ./x128 -silent -sound -keybuf "
  3 i=rnd($SEED)
  `cat ../maze.bas`
  run
  " -warp -limitcycles 20000000 -exitscreenshotvicii /tmp/$1.png 2>&1 >/dev/null
  cd ..

  RESPONSE="{\"isBase64Encoded\": true, \"headers\": {\"Content-type\": \"image/png\", \"content-disposition\":\"inline\"}, \"statusCode\":200, \"body\":\"`base64 -w 0 /tmp/$1.png`\"}"

  echo $RESPONSE
}

Also note that we’re passing in a SEED value because, otherwise, the Commodore 128 will generate the same maze every time.

Repackage the Lambda and upload as in the Commodore VICE Lambda Custom Runtime blog post.

Setting up the API Gateway

Go to API Gateway in the AWS Console and select [Create API]

APIs Create API

Choose [Build] under “REST API” (we won’t be using OIDC/OAuth2/CORS…)

REST API -> Build

Choose REST, New API, and name your API:

REST, New API, API Name

Create a GET Method and check the checkmark

Create Method
API Method verb selection -> GET
Check the checkmark to commit
  • Choose “Lambda Function” for your Integration Type
  • Check “Use Lambda Proxy Integration”
  • Type the name of the Lambda that you’ve calling and select it and [Save] and confirm that you want to add role/permission for the Lambda.
GET API configuration

Deploy the API

Back on your API view, select the [Actions] dropdown and [Deploy API]

API Actions "Deploy API"

Just create a new dev stage… we’re not going to “production” with this experiment

Create new dev stage

and [Deploy]

It’s broken

If you click the Invoke URL presented, you will get a broken image

Invoke URL
Broken image from the API

Since we’re only sending back PNG files, we can just tell the API that */* is a Binary Media Type by going to Settings on the left sidebar for the API and [(+) Add Binary Media Type]

Add Binary Media Type

Be sure to save the settings, redeploy the API and be sure to use the Invoke URL for the correct API! (I had two identical and was “fixing” the wrong one… hence the different API URL snapshots)

Commodore Maze Generator PNG 1

If you refresh, you should get a new maze from the Commodore maze generator program:

Commodore Maze Generator run 2

Some Troubleshooting

  • Be sure to save your settings
  • Test your API call to Lambda in the Method Execution view (Resources -> GET -> Test [lightning bolt])
  • Be sure to redeploy your API after settings change.

Commodore 128 (VICE) Custom Runtime on AWS Lambda

Why a Commodore 128 Custom Runtime??

The first reason is wanted to do a Commodore 128 Custom Runtime on AWS Lambda is because it’s an absolutely ridiculous thing to do. The Commodore 128 runs at 1 MHz (2 MHz in “fast” mode) natively on the 8502 processor vs. multi-GHz scale.

Another reason is because I’ve had challenges with the slight differences with Amazon Linux vs. Ubuntu for EC2 instance. This seemed like a good way to exercise working through those differences.

The last reason is that I wanted a platform that would require me to dig a little deeper into how things are managed for a custom Lambda. The only directory that’s writable is /tmp. There are specific conventions used to honor the function.handler format. The responses (if you don’t have a supported runtime) are handled through callback URLs.

Compiling VICE

It’s a bit of an adventure getting VICE running on Amazon Linux. Ubuntu provides a package for VICE (of course, you’re on your own for the ROMs themselves.) With Amazon Linux, you’ll need to download the tarball and compile. Here’s my video recreating that process through to getting the Lambda running:

Compile VICE and Building the Custom Runtime

The steps… fire up an Amazon Linux EC2 instance and:

  • sudo yum update
  • sudo yum install links -y
  • links https://vice-emu.sourceforge.io and download the vice-3.5.tar.gz
  • tar zxvf vice-3.5.tar.gz from the home directory
  • sudo yum install -y gcc gcc-c++ flex bison dos2unix libpng-devel.x86_64
  • links https://www.floodgap.com/retrotech/xa/ and download xa-2.3.11.tar.gz or whatever version is available
  • tar zxvf xa-2.3.11.tar.gz
  • cd xa-2.3.11 and make
  • Add xa to the path with PATH=~/xa-2.3.11:$PATH
  • cd ~
  • cd vice-3.5 and ./configure --disable-pdf-docs --enable-headlessui --without-pulse --without-alsa and make

You should be able to cd ~/vice-3.5/data/C128 and run:

../src/x128 -silent -sound -ntsc -keybuf "10 graphic 1
20 scnclr
30 circle 1,100,100,20
run
" -warp -limitcycles 8000000 -exitscreenshotvicii image.png

and be able to scp the image.png and open it and see a screenshot something like (bump cycles to 10,000,000 if necessary): (Edit: use -warp not +warp to enable warp mode)

Testing x128 for Custom Runtime
C128 Circle

Packaging the Necessary Pieces

Make a vice-package directory or similar and grab the contents of vice-3.5/data/C128 (some files can be excluded, such as build files if you want to trim the package down) and the vice-3.5/src/x128 file and tar the vice-package directory (tar cvf vice-package.tar vice-package/*) and scp it down to your local computer.

Get the Sample Lambda for Custom Runtime

Go to Lambda -> Functions -> Create Function -> Author from Scratch and select the “Provide your own bootstrap on Amazon Linux 2” option:

Custom Runtime provide your own bootstrap
Custom Runtime template

Copy the bootstrap.sample to the root of the lambda package you are going to create and name it bootstrap and the hello.sh.sample as function or whatever the first part of your function.handler name is (see the Runtime settings below the code window):

Runtime settings where your function.handler is named.

Constructing the bootstrap

There are a couple of environment variables (XDG_CACHE_HOME and XDG_CONFIG_HOME) that have to be set to /tmp so that VICE can write to them. Be sure the handler in Runtime settings matches <script_name>.<bash_function_name> or else Lambda won’t be able to find it to invoke (actually the bootstrap below won’t… you can skip using $_HANDLER and hard code, but then the AWS console won’t help you for function configuration.) I disabled the -e option because we’re actually going to exit VICE ungracefully on purpose for simplicity. Be aware that this is the ON ERROR RESUME NEXT or “try with empty catch block” in that your code will ignore all the other potential failures along the way.

#!/bin/sh
# set -euo pipefail
# we're going to exit VICE on clock cycles so -e option would fail in this case
set -uo pipefail

# otherwise vice tries to write to the 'home' directory that isn't a [writeable] thing in Lambda
export XDG_CACHE_HOME=/tmp
export XDG_CONFIG_HOME=/tmp

# Handler format: <script_name>.<bash_function_name>
#
# The script file <script_name>.sh  must be located at the root of your
# function's deployment package, alongside this bootstrap executable.
source $(dirname "$0")/"$(echo $_HANDLER | cut -d. -f1).sh"

while true
do
    # Request the next event from the Lambda runtime
    HEADERS="$(mktemp)"
    EVENT_DATA=$(curl -v -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
    INVOCATION_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

    # Execute the handler function from the script
    RESPONSE=$($(echo "$_HANDLER" | cut -d. -f2) $INVOCATION_ID "$EVENT_DATA")

    # Send the response to Lambda runtime
    curl -v -sS -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$INVOCATION_ID/response" -d "$RESPONSE"
done

Constructing the handler

The handler for this setup needs to only output what is intended as a response. I’m redirecting stderr and stdout to /dev/null because there are some messages that pop-up in the current state of the emulator. I am also using the -silent option to suppress all the errors about missing disk drive and other device ROMs that I don’t care about for this case.

function handler () {
  EVENT_DATA=$2

  cd vice-package

  ./x128 -silent -sound -ntsc -keybuf "10 graphic 1
    20 scnclr
    30 circle 1,100,100,20
    run
  " +warp -limitcycles 8000000 -exitscreenshotvicii /tmp/$1.png 2>&1 >/dev/null
  cd ..


  RESPONSE="{\"isBase64Encoded\": true, \"headers\": {\"Content-type\": \"image/png\", \"content-disposition\":\"attachment; filename=$1.png\"}, \"statusCode\":200, \"body\":\"`base64 /tmp/$1.png`\"}"

  echo $RESPONSE
}

The response

The above response is intended to output JSON in preparation for API Gateway Lambda integration. statusCode is required, and to convert the image back to an image isBase64Encoded and the headers Content-type is needed. The content-disposition is to tell it to download. All this gets POSTed back by the bootstrap script to the invocation response callback. The body is the base64 encoded png file, but in the current invocation in Amazon Linux, I’m getting newlines in the output, so that’s a problem to debug before attaching to API Gateway.

One more missing piece

We also need to pull libpng from our EC2 instance and place in the root of the lambda function at the same level as the bootstrap file. Just scp ec2-user@ec2IPaddress:/usr/lib64/libpng\* . for that.

Structure of the zip file and Deploy

zip up the following pieces into your lambda.zip (zip file name doesn’t really matter, just the organization of the contents):

  • bootstrap
  • function.sh
  • libpng*
  • vice-package/*

Once uploaded, you should be able to [Test] the function and check the logs. Add a set -x to your bootstrap if things aren’t behaving. You may need to chmod +x your bootstrap if you haven’t tried to run it locally for testing.

Using C to Program the Commodore 64 SID Chip

Why use C to Program the Commodore 64 SID Chip?

If you grew up with an original Commodore 64, you probably had a manual along with it that showed you how to program the SID chip. There was even a multi voice example that took forever to load its data and was impossible to read and debug. This is what it looked like (lines 1 and 2 mine) (GitHub link):

1 rem c64 programmers reference guide
2 rem pages 187-188
10 s=54272: forl=stos+24: pokel,0:next
20 dimh(2,200),l(2,200),c(2,200)
30 dimfq(11)
40 v(0)=17:v(1)=65:v(2)=33
50 pokes+10,8:pokes+22,128:pokes+23,244
60 fori=0to11:readfq(i):next
100 fork=0to2
110 i=0
120 readnm
130 ifnm=0then250
140 wa=v(k):wb=wa-1:ifnm<0thennm=-nm:wa=0:wb=0
150 dr%=nm/128:oc%=(nm-128*dr%)/16
160 nt=nm-128*dr%-16*oc%
170 fr=fq(nt)
180 ifoc%=7then200
190 forj=6tooc%step-1:fr=fr/2:next
200 hf%=fr/256:lf%=fr-256*hf%
210 ifdr%=1thenh(k,i)=hf%:l(k,i)=lf%:c(k,i)=wa:i=i+1:goto120
220 forj=1todr%-1:h(k,i)=hf%:l(k,i)=lf%:c(k,i)=wa:i=i+1:next
230 h(k,i)=hf%:l(k,i)=lf%:c(k,i)=wb
240 i=i+1:goto120
250 ifi>imthenim=i
260 next
500 pokes+5,0:pokes+6,240
510 pokes+12,85:pokes+13,133
520 pokes+19,10:pokes+20, 197
530 pokes+24,31
540 fori=0toim
550 pokes,l(0,i):pokes+7,l(1,i):pokes+14,l(2,i)
560 pokes+1,h(0,i):pokes+8,h(1,i):pokes+15,h(2,i)
570 pokes+4,c(0,i):pokes+11,c(1,i):pokes+18,c(2,i)
580 fort=1to80:next:next
590 fort=1to200:next:pokes+24,0
600 data34334,36376,38539,40830
610 data43258,45830,48556,51443
620 data54502,57743,61176,64814
1000 data594,594,594,596,596
1010 data1618,587,592,587,585,331,336
1020 data1097,583,585,585,585,587,587
1030 data1609,585,331,337,594,594,593
1040 data1618,594,596,594,592,587
1050 data1616,587,585,331,336,841,327
1060 data1607
1999 data0
2000 data583,585,583,583,327,329
2010 data1611,583,585,578,578,578
2020 data196,198,583,326,578
2030 data326,327,329,327,329,326,578,583
2040 data1606,582,322,324,582,587
2050 data329,327,1606,583
2060 data327,329,587,331,329
2070 data329,328,1609,578,834
2080 data324,322,327,585,1602
2999 data0
3000 data567,566,567,304,306,308,310
3010 data1591,567,311,310,567
3020 data306,304,299,308
3030 data304,171,176,306,291,551,306,308
3040 data310,308,310,306,295,297,299,304
3050 data1586,562,567,310,315,311
3060 data308,313,297
3070 data1586,567,560,311,309
3080 data308,309,306,308
3090 data1577,299,295,306,310,311,304
3100 data562,546, 1575
3999 data0

I copied this from a text / pdf version which was in all uppercase (because only one case is valid for BASIC keywords and such). It’s in lowercase above so that you can copy/paste into the VICE emulator, because that “one case” is the lower in lower/uppercase mode, and uppercase pastes in as graphics characters in upper/graphics mode. This example is a fairly impressive one, but it also takes 45 seconds to load all of the data and a couple of typos can mean hours of debugging. I took several hours to find what was ultimately mistakenly typing an I instead of a 1.

Spacing Matters in Commodore BASIC

While this code is not *required* to be so compact in spacing, those in-line spaces almost 100% impact the size of the program in memory and on disk. So if you’re taking up nearly all of the BASIC RAM, eliminating spaces actually matters. So you have code that reads pokes or fori or dimfq.

In BASIC, data has to be part of code but still explicitly read in every time

The DATA statements in BASIC can create a nice little data block, but it’s an implicit data block that requires explicit reading in. So you take up all of the space required for the DATA statements, but then load that data into arrays, and, in this case, take 45 seconds to read in and encode the data used.

Variable names are 2 characters max in Commodore BASIC

PIPE, PIKE, and PINE are the same variable, so if you set PIPE to 2, PIKE to 3, and PINE to 4, you will get 4 from all of them. However Commodore 64 BASIC will let you *think* that up to 5 characters is legal (you get a syntax error if you use something like THINGS=2)

Functions can only receive one argument and be named 2 chars (5 can be specified)

The def fn statement allows you to specify code like def fn xp(x) = x * 300 + 12, which then is required to be invoked by fn xp(x) or (fn xp(x)) + y * 12 in context. (The parentheses are probably unnecessary, but readability can be helped by using 2 extra bytes for the parens.)

Subroutines take no arguments and have to be referred to by line number

If you are good at modularizing your code, you’ll find Commodore 64 BASIC a bit limiting. This isn’t even compared to modern and higher level programming languages. Even a decent Assembler can at least track symbolic locations of the subroutines and variables.

Commodore BASIC is slow

Spacing and variable names have an impact on the performance of Commodore BASIC

Setting up for C programming

Install cc65 for your platform. For macOS, it’s available via brew install cc65 and in Ubuntu via sudo apt install cc65. (My Windows install of cc65 is via WSL2).

You may want to find the include files for reference for your platform. They’re in /usr/share/cc65/include for my Ubuntu install and /opt/homebrew/Cellar/cc65/2.19/share/cc65/include (your version may vary).

I created a c64_note_values.h header file for note #defines for all octave 0-7 notes. They’re proportional to their frequency values, but scaled up so that the low ranges can be defined by integer values. If you want to tweak the values and regenerate, this is the C file I created to generate it.

The rudimentary Canon in D (so far)

Thanks to Recreating the Commodore 64 User Guide code samples in cc65. Part four: Sound, I was inspired to try my own hand at the C code.

#include <stdio.h>
#include <stdlib.h>
#include <c64.h>
#include "c64_note_values.h"

const unsigned int notes[][8] = {
  { D3, A2, B2, Fs2, G2, D2, G2, A2 },
  { Fs5, E5, D5, Cs5, B4, A4, B4, Cs5 },
  { D5, Cs5, B4, A4, G4, Fs4, G4, E4 },
};


int main(void) {
  unsigned int i,d;
  unsigned int measure=0;

  SID.amp = 0x1F;
  SID.v1.ad = 0x0f;
  SID.v2.ad = 0x0f;
  SID.v3.ad = 0x0f;

  while(1) {
    for(i = 0; i < sizeof(notes[0]) / sizeof(int); i++) {
      SID.v1.freq = notes[0][i];
      SID.v1.ctrl = 0x11;
      if(measure) {
        SID.v2.freq = notes[1][i];
        SID.v2.ctrl = 0x11;
      }
      if(measure > 1) {
        SID.v3.freq = notes[2][i];
        SID.v3.ctrl = 0x11;
      }
      for(d=0; d<10000; d++) { }
      SID.v1.ctrl = 0x10;
      if(measure) {
        SID.v2.ctrl = 0x10;
      }
      if(measure > 1) {
        SID.v3.ctrl = 0x10;
      }
    }
    measure++;
    if(measure>2) { break; }
  }
  return EXIT_SUCCESS;
}

In this example, I haven’t actually created any functions yet, beyond the main.

SID is a struct pointer cast of the base memory location of the default SID chip. (See here for a “multiple SID” example if you enable in hardware or VICE emulator)

.v1 through .v3 align to the base address of the voice 1 through 3 controls.

.ad is an attack / decay value… 0-f for the two nibbles. If you want the note to start as quickly as possible, use 0, if you want the note to never fade out, use f for the second nibble. For this case, I’m using just that… 0x0f.

.freq is the scaled frequency value I created earlier from the header file which is in the notes arrays.

.ctrl value of 0x11 turns the note on and 0x10 turns it off. You’re actually setting the triangle waveform bit to true and the gate bit to on and off.

.amp is a volume number (actually volume and mode… location 54296 in the Commodore 64 memory map). LP is 1 and volume nibble is 0xf.

Compiling and running

Assuming that you’ve saved the file to canonind.c, the command to compile and link is cl65 -t c64 -O canonind.c (you may be able to remove the -t c64 parameter… my install appears to assume c64.)

Provided you have a successful build, you should have a canonind binary file. In VICE, you can use “File -> Smart attach disk/tape/cartridge…” or “Autostart disk/tape image…”. For the first version, you’ll want to click the “[Autostart]” button after selecting the extensionless filename. For the second version, you’ll want to look for All Files *.* instead of .d64 etc… Either version should autoplay once loaded.

See my video (choppy sound from VICE and morning voice warning) of the process of using C to program the Commodore 64 SID Chip here: