Commodore 128 (VICE) Custom Runtime on AWS Lambda

Why a Commodore 128 Custom Runtime??

The first reason is wanted to do a Commodore 128 Custom Runtime on AWS Lambda is because it’s an absolutely ridiculous thing to do. The Commodore 128 runs at 1 MHz (2 MHz in “fast” mode) natively on the 8502 processor vs. multi-GHz scale.

Another reason is because I’ve had challenges with the slight differences with Amazon Linux vs. Ubuntu for EC2 instance. This seemed like a good way to exercise working through those differences.

The last reason is that I wanted a platform that would require me to dig a little deeper into how things are managed for a custom Lambda. The only directory that’s writable is /tmp. There are specific conventions used to honor the function.handler format. The responses (if you don’t have a supported runtime) are handled through callback URLs.

Compiling VICE

It’s a bit of an adventure getting VICE running on Amazon Linux. Ubuntu provides a package for VICE (of course, you’re on your own for the ROMs themselves.) With Amazon Linux, you’ll need to download the tarball and compile. Here’s my video recreating that process through to getting the Lambda running:

Compile VICE and Building the Custom Runtime

The steps… fire up an Amazon Linux EC2 instance and:

  • sudo yum update
  • sudo yum install links -y
  • links https://vice-emu.sourceforge.io and download the vice-3.5.tar.gz
  • tar zxvf vice-3.5.tar.gz from the home directory
  • sudo yum install -y gcc gcc-c++ flex bison dos2unix libpng-devel.x86_64
  • links https://www.floodgap.com/retrotech/xa/ and download xa-2.3.11.tar.gz or whatever version is available
  • tar zxvf xa-2.3.11.tar.gz
  • cd xa-2.3.11 and make
  • Add xa to the path with PATH=~/xa-2.3.11:$PATH
  • cd ~
  • cd vice-3.5 and ./configure --disable-pdf-docs --enable-headlessui --without-pulse --without-alsa and make

You should be able to cd ~/vice-3.5/data/C128 and run:

../src/x128 -silent -sound -ntsc -keybuf "10 graphic 1
20 scnclr
30 circle 1,100,100,20
run
" -warp -limitcycles 8000000 -exitscreenshotvicii image.png

and be able to scp the image.png and open it and see a screenshot something like (bump cycles to 10,000,000 if necessary): (Edit: use -warp not +warp to enable warp mode)

Testing x128 for Custom Runtime
C128 Circle

Packaging the Necessary Pieces

Make a vice-package directory or similar and grab the contents of vice-3.5/data/C128 (some files can be excluded, such as build files if you want to trim the package down) and the vice-3.5/src/x128 file and tar the vice-package directory (tar cvf vice-package.tar vice-package/*) and scp it down to your local computer.

Get the Sample Lambda for Custom Runtime

Go to Lambda -> Functions -> Create Function -> Author from Scratch and select the “Provide your own bootstrap on Amazon Linux 2” option:

Custom Runtime provide your own bootstrap
Custom Runtime template

Copy the bootstrap.sample to the root of the lambda package you are going to create and name it bootstrap and the hello.sh.sample as function or whatever the first part of your function.handler name is (see the Runtime settings below the code window):

Runtime settings where your function.handler is named.

Constructing the bootstrap

There are a couple of environment variables (XDG_CACHE_HOME and XDG_CONFIG_HOME) that have to be set to /tmp so that VICE can write to them. Be sure the handler in Runtime settings matches <script_name>.<bash_function_name> or else Lambda won’t be able to find it to invoke (actually the bootstrap below won’t… you can skip using $_HANDLER and hard code, but then the AWS console won’t help you for function configuration.) I disabled the -e option because we’re actually going to exit VICE ungracefully on purpose for simplicity. Be aware that this is the ON ERROR RESUME NEXT or “try with empty catch block” in that your code will ignore all the other potential failures along the way.

#!/bin/sh
# set -euo pipefail
# we're going to exit VICE on clock cycles so -e option would fail in this case
set -uo pipefail

# otherwise vice tries to write to the 'home' directory that isn't a [writeable] thing in Lambda
export XDG_CACHE_HOME=/tmp
export XDG_CONFIG_HOME=/tmp

# Handler format: <script_name>.<bash_function_name>
#
# The script file <script_name>.sh  must be located at the root of your
# function's deployment package, alongside this bootstrap executable.
source $(dirname "$0")/"$(echo $_HANDLER | cut -d. -f1).sh"

while true
do
    # Request the next event from the Lambda runtime
    HEADERS="$(mktemp)"
    EVENT_DATA=$(curl -v -sS -LD "$HEADERS" -X GET "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/next")
    INVOCATION_ID=$(grep -Fi Lambda-Runtime-Aws-Request-Id "$HEADERS" | tr -d '[:space:]' | cut -d: -f2)

    # Execute the handler function from the script
    RESPONSE=$($(echo "$_HANDLER" | cut -d. -f2) $INVOCATION_ID "$EVENT_DATA")

    # Send the response to Lambda runtime
    curl -v -sS -X POST "http://${AWS_LAMBDA_RUNTIME_API}/2018-06-01/runtime/invocation/$INVOCATION_ID/response" -d "$RESPONSE"
done

Constructing the handler

The handler for this setup needs to only output what is intended as a response. I’m redirecting stderr and stdout to /dev/null because there are some messages that pop-up in the current state of the emulator. I am also using the -silent option to suppress all the errors about missing disk drive and other device ROMs that I don’t care about for this case.

function handler () {
  EVENT_DATA=$2

  cd vice-package

  ./x128 -silent -sound -ntsc -keybuf "10 graphic 1
    20 scnclr
    30 circle 1,100,100,20
    run
  " +warp -limitcycles 8000000 -exitscreenshotvicii /tmp/$1.png 2>&1 >/dev/null
  cd ..


  RESPONSE="{\"isBase64Encoded\": true, \"headers\": {\"Content-type\": \"image/png\", \"content-disposition\":\"attachment; filename=$1.png\"}, \"statusCode\":200, \"body\":\"`base64 /tmp/$1.png`\"}"

  echo $RESPONSE
}

The response

The above response is intended to output JSON in preparation for API Gateway Lambda integration. statusCode is required, and to convert the image back to an image isBase64Encoded and the headers Content-type is needed. The content-disposition is to tell it to download. All this gets POSTed back by the bootstrap script to the invocation response callback. The body is the base64 encoded png file, but in the current invocation in Amazon Linux, I’m getting newlines in the output, so that’s a problem to debug before attaching to API Gateway.

One more missing piece

We also need to pull libpng from our EC2 instance and place in the root of the lambda function at the same level as the bootstrap file. Just scp ec2-user@ec2IPaddress:/usr/lib64/libpng\* . for that.

Structure of the zip file and Deploy

zip up the following pieces into your lambda.zip (zip file name doesn’t really matter, just the organization of the contents):

  • bootstrap
  • function.sh
  • libpng*
  • vice-package/*

Once uploaded, you should be able to [Test] the function and check the logs. Add a set -x to your bootstrap if things aren’t behaving. You may need to chmod +x your bootstrap if you haven’t tried to run it locally for testing.

Running a Commodore VICE emulator on a remote Ubuntu Linux machine with Xfvb

The Challenge

I want to have a Commodore 128 VICE emulator start up, run some arbitrary BASIC code, and get a snapshot of the output. There are a few settings configurable from the command line to accomplish this:

  • +sound (without this option you will get and error "pa_simple_new(): Connection refused" because you’re *probably* not going to have a PulseAudio option for your remote linux box)
  • -limitcycles 10000000 (intentionally timeout the machine after 10 million cycles… ~10 seconds)
  • -exitscreenscreenshotvicii – this is just -exitscreenshot for non-128 emulators
  • -keybuf so that you can “type” in your program to the BASIC emulator

Installing and Running VICE

In Ubuntu the VICE package can be installed with sudo apt install vice. You still have a couple of issues: First you have nothing to send your display to. I remedied this with Xfvb (the package is lowercase x, but the executable is uppercase)

sudo apt install vice xfvb 
Xfvb :1 & # if you exit your session, you'll have to kill this off or point to it again
export DISPLAY=:1 # use Xfvb for your "display"

At this point, if you try to run on Ubuntu, you’ll be missing ROMs for the various components (basic and the kernal are two of them). They don’t install with the vice package because they’re not appropriately licensed (understatement) for the Ubuntu distro. If you try to run the emulator without them, you’ll get something like the following:

*** VICE Version 3.4 ***

Welcome to x128, the free portable C128 Emulator.

Current VICE team members:
Marco van den Heuvel, Fabrizio Gennari, Groepaz, Errol Smith, Olaf Seibert,
Marcus Sutton, Kajtar Zsolt, AreaScout, Bas Wassink, Michael C. Martin,
David Hogan.

This is free software with ABSOLUTELY NO WARRANTY.
See the "About VICE" command for more info.

C128MEM: Error - Couldn't load kernal ROM `kernal'.
Error - Machine initialization failed.

Exiting...
Segmentation fault

Getting and installing the ROMs

You can download the ROMs from the release source file on the project page. I used the vice-3.4 source. Download/upload the file to your Ubuntu machine and then untar and copy the rom files from the vice-{version}/data directory to /usb/lib/vice:

tar -zxvf vice-3.4.tar.gz
cd vice-3.4/data
sudo ls **/* | grep  -v '\.' | sudo xargs -I {} cp -Rp --parents {} /usr/lib/vice

Run a test script

The following code should have the emulator draw a circle and then capture to the horribly named haha.png:

x128 -keybuf "10 graphic 1
20 scnclr
30 circle 1,100,100,30
run
" -sound -limitcycles 10000000 -exitscreenshotvicii haha.png

Next Steps

I don’t know… Hook up a lambda? Write a crude server that listens on a COM port? One thing I’m happy about discovering is the -keybuf argument, because I know now that I can inject BASIC (keystrokes to enter BASIC) into an emulator from a source code file without having to worry about the disk or tape image formats.