Lua API
PCSX-Redux features a Lua API that's available through either a direct Lua console, or a Lua editor, both available through the Debug menu. The Lua VM runs on the main thread, the same one as the UI and the emulated MIPS CPU. As a result, care must be taken to not stall for too long, or the UI will become unresponsive. Using coroutines to handle long-running tasks is recommended, yielding periodically to let the UI perform some work too. The UI is probably going to run at 60FPS or so, which gives a ballpark of 15ms per frame.
Lua engine
The Lua engine that's being used is LuaJIT 2.1.0-beta3 compiled in Lua 5.2 compatibility mode. The Lua 5.1 user manual and LuaJIT user manual are recommended reads. In particular, the bindings heavily make use of LuaJIT's FFI capabilities, which allows for direct memory access within the emulator's process. This means there is little protection against dramatic crashes the LuaJIT FFI engine can cause into the emulator's process, and the user must pay extra attention while manipulating FFI objects. Despite that, the code tries as much as possible to sandbox what the Lua code does, and will prevent crashes on any recoverable exception, including OpenGL and ImGui exceptions.
Lua console
All of the messages coming from Lua should display into the Lua console directly. The input text there is a single line execution, so the user can type one-liner Lua statements and get an immediate result.
Lua editor
The editor allows for more complex, multi-line statements to be written, such as complete functions. The editor will by default auto save its contents on the disc under the filename pcsx.lua
, which can potentially be a problem if the last statement typed crashed the emulator, as it'll be reloaded on the next startup. It might become necessary to either edit the file externally, or simply delete it to recover from this state.
The auto-execution of the editor permits for rapid development loop, with immediate feedback of what's done.
For complex projects however, it is recommended to split your work into sub-modules, and use the loadfile
function to load them in your main code. This implies working on your project using an external editor.
API
Basic Lua
The LuaJIT extensions are fully loaded, and can be used globally. The standard Lua libraries are loaded, and are usable. The require
function exists, but isn't recommended as the loading of external DLLs might be difficult to properly accomplish. Loading pure Lua files is fine. The ffi
table is loaded globally, there is no need to require
it, but it'll work nonetheless. As a side-effect of Luv, Lua-compat-5.3 is loaded.
Dear ImGui
A good portion of ImGui is bound to the Lua environment, and it's possible for the Lua code to emit arbitrary widgets through ImGui. It is advised to consult the user manual of ImGui in order to properly understand how to make use of it. The list of current bindings can be found within the source code. Some usage examples will be provided within the case studies.
OpenGL
OpenGL is bound directly to the Lua API through FFI bindings, loosely inspired and adapted from LuaJIT-OpenCL. Some usage examples can be seen in the CRT-Lottes shader configuration page.
Luv
For network access and interaction, PCSX-Redux uses libuv internally, and is exposed to the Lua API through Luv, tho its loop is tied to the main thread one, meaning it'll run only once per frame. There is another layer of network API available through the File API, which is more convenient and faster for simple tasks.
Zlib
The Zlib C-API is exposed through FFI bindings. There is another layer of Zlib API available through the File API, which is more convenient and faster for simple tasks.
FFI-Reflect
The FFI-Reflect library is loaded globally as the reflect
symbol. It's able to generate reflection objects for the LuaJIT FFI module.
PPrint
The PPrint library is loaded globally as the pprint
symbol. It's a more powerful print
function than the one provided by Lua, and can be used to print tables in a more readable way.
Lua-Protobuf
The Lua-Protobuf library is available, but not loaded by default. All of its documented API should be usable straight with no additional work. It has been slightly modified, but nothing that should be visible to the user. There is some limited glue between its API and PCSX's.
PCSX-Redux
Settings
All of the settings are exposed to Lua via the PCSX.settings
table. It contains pseudo-tables that are reflections of the internal objects, and can be used to read and write the settings. The exact list of settings can vary quickly over time, so making a full list here would be fruitless. It is possible however to traverse the settings using pprint
for example. The semantic of the settings is the same as from within the GUI, with the same caveats. For example, disabling the dynamic recompiler requires a reboot of the emulator.
ImGui interaction
PCSX-Redux will periodically try to call the Lua function DrawImguiFrame
to allow the Lua code to draw some widgets on screen. The function will be called exactly once per actual UI frame draw, which, when the emulator is running, will correspond to the emulated GPU's vsync. If the function throws an exception however, it will be disabled until recompiled with new code.
Events Engine interaction & Execution Contexts
LuaJIT C callbacks aren't called from a safe execution context that can allow for coroutine resuming, and luv's execution context doesn't have any error handling.
It is possible to defer executing code to the main loop of PCSX-Redux, which can (a) resume coroutines and (b) execute code in a safe context. The function PCSX.nextTick(func)
will execute the given function in the next main loop iteration. Here's some examples of how to use it:
1 2 3 4 5 6 7 8 9 10 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
|
Of course, this can also delay processing significantly, as the main loop is usually bound to the speed of the UI, which can mean up to 20ms of delay.
File API
While the normal Lua io API is loaded, there's a more powerful API that's more tightly integrated with the rest of the PCSX-Redux File handling code. It's an abstraction class that allows seamless manipulation of various objects using a common API.
The File objects have different properties depending on how they are created and their intention. But generally speaking, the following rules apply:
- Files are reference counted. They will be deleted when the reference count reaches zero. The Lua garbage collector will only decrease the reference count.
- Whenever possible, writes are deferred to an asynchronous thread, making writes return basically instantly. This speed up comes at the trade off of data integrity, which means writes aren't guaranteed to be flushed to the disk yet when the function returns. Data will always have integrity internally within PCSX-Redux however, and when exiting normally, all data will be flushed to the disk.
- Some File objects can be cached. When caching, reads and writes will be done transparently, and the cache will be used instead of the actual file. This will make reads return basically instantly too.
- The Read and Write APIs can haul LuaBuffer objects. These are Lua objects that can be used to read and write data to the file. You can construct one using the
Support.NewLuaBuffer(size)
function. They can be cast to strings, and can be used as a table for reading and writing bytes off of it, in a 0-based fashion. The length operator will return the size of the buffer. The methods:maxsize()
and:resize(size)
are available. They also have a.pbSlice
property that implicitly converts them to a Lua-Protobuf'spb.slice
, which can then be passed topb.decode
. - The Read and Write APIs can also function using Lua-Protobuf's buffers and slices respectively.
- If the file isn't closed when the file object is destroyed, it'll be closed then, but letting the garbage collector do the closing is not recommended. This is because the garbage collector will only run when the memory pressure is high enough, and the file handle will be held for a long time.
- When using streamed functions, unlike POSIX files handles, there's two distinct seeking pointers: one for reading and one for writing.
All File objects have the following API attached to them as methods:
Closes and frees any associated resources. Better to call this manually than letting the garbage collector do it:
1 |
|
Reads from the File object and advances the read pointer accordingly. The return value depends on the variant used.
1 2 3 4 |
|
Reads from the File object at the specified position. No pointers are modified. The return value depends on the variant used, just like the non-At variants above.
1 2 3 4 |
|
Writes to the File object. The non-At variants will advances the write pointer accordingly. The At variants will not modify the write pointer, and simply write at the requested location. Returns the number of bytes written. The string
variants will in fact take any object that can be transformed to a string using tostring()
.
1 2 3 4 5 6 7 8 9 10 |
|
Note that in this context, pb_slice
and pb_buffer
refer to Lua-Protobuf's pb.slice
and pb.buffer
objects respectively.
Some APIs may return a Slice
object, which is an opaque buffer coming from C++. The write
and writeAt
methods can take a Slice
. It is possible to write a slice to a file in a zero-copy manner, which will be more efficient:
1 2 |
|
After which, the slice will be consumed and not reusable. The Slice
object is convertible to a string using tostring()
, and also has two members: data
, which is a const void*
, and size
. Once consumed by the MoveSlice
variants, the size of a slice will go down to zero.
Finally, it is possible to convert a Slice
object to a pb.slice
one using the Support.sliceToPBSlice
function. However, the same caveats as for normal pb.slice
objects apply: it is fragile, and will be invalidated if the underlying Slice is moved or destroyed, so it is recommended to use it as a temporary object, such as an argument to pb.decode
. Still, it is a much faster alternative to calling tostring()
which will make a copy of the underlying slice.
The following methods manipulate the read and write pointers. All of them return their corresponding pointer. The wheel
argument can be of the values 'SEEK_SET'
, 'SEEK_CUR'
, and 'SEEK_END'
, and will default to 'SEEK_SET'
.
1 2 3 4 |
|
These will query the corresponding File object.
1 2 3 4 5 6 7 8 |
|
If applicable, this will start caching the corresponding file in memory.
1 |
|
Same as above, but will suspend the current coroutine until the caching is done. Cannot be used with the main thread.
1 |
|
Duplicates the File object. This will re-open the file, and possibly duplicate all ressources associated with it.
1 |
|
Creates a read-only view of the file starting at the specified position, spanning the specified length. The view will be a new File object, and will be a view of the same underlying file. The default values of start and length are 0 and -1 respectively, which will effectively create a view of the entire file. The view may have less features than the underlying file, but will always be seekable, and keep its seeking position independent of the underlying file. The view will hold a reference to the underlying file.
1 |
|
In addition to the above methods, the File API has these helpers, that'll read or write binary values off their corresponding stream position for the non-At variants, or at the indicated position for the At variants. All the values will be read or stored in Little Endian, regardless of the host's endianness.
1 2 3 4 5 6 7 8 |
|
The Lua VM can create File objects in different ways:
1 2 3 4 5 |
|
The open
function will function on filesystem and network URLs, while the buffer
function will generate a memory-only File object that's fully readable, writable, and seekable. The type
argument of the open
function will determine what happens exactly. It's a string that can have the following values:
READ
: Opens the file for reading only. Will fail if the file does not exist. This is the default type.TRUNCATE
: Opens the file for reading and writing. If the file does not exist, it will be created. If it does exist, it will be truncated to 0 size.CREATE
: Opens the file for reading and writing. If the file does not exist, it will be created. If it does exist, it will be left untouched.READWRITE
: Opens the file for reading and writing. Will fail if the file does not exist.DOWNLOAD_URL
: Opens the file for reading only. Will immediately start downloading the file from the network. Thefilename
argument will be treated as a URL. The curl is the backend for this feature, and its url schemes are supported. The progress of the download can be monitored with the:cacheProgress()
method.DOWNLOAD_URL_AND_WAIT
: As above, but suspends the current coroutine until the download is done. Cannot be used with the main thread.
When calling .buffer()
with no argument, this will create an empty read-write buffer. When calling it with a cdata pointer and a size, this will have the following behavior, depending on type:
READWRITE
(or no type): The memory passed as an argument will be copied first.READ
: The memory passed as an argument will be referenced, and the lifespan of said memory needs to outlast the File object. The File object will be read-only.ACQUIRE
: It will acquire the pointer passed as an argument, and free it later usingfree()
, meaning it needs to have been allocated usingmalloc()
in the first place.
The uvFifo
function will create a File object that will read from and write to the specified TCP address and port after connecting to it. The :failed()
method will return true in case of a connection failure. The address is a string, and must be a strict IP address, no hostnames allowed. The port is a number between 1 and 65535 inclusive. As the name suggests, this object is a FIFO, meaning that incoming bytes will be consumed by any read operation. The :size()
method will return the number of bytes in the FIFO. Writes will be immediately sent over. There are no reception guarantees, as the other side might have disconnected at any point. The :eof()
method will return true when the opposite end of the stream has been disconnected and there's no more bytes in the FIFO. In addition to the normal File
API, a uvFifo
has a method called :isConnecting()
, which returns a boolean indicating the fifo is still connecting, meaning it's possible to verify if the fifo has successfully connected using the boolean expression not fifo:isConnecting() and not fifo:failed()
.
The zReader
function will create a read-only File object which decompresses the data from the specified File object. The file
argument is a File object, and the size
argument is an optional number that will be used to determine the size of the decompressed data. If not specified, the resulting file won't be seekable, and its :size()
method won't work, but the file will be readable until :eof()
returns true. The raw
argument is an optional string that needs to be equal to 'RAW'
, and will determine whether the data is compressed using the raw deflate format, or the zlib format. Any other string means the zlib format will be used.
Iso files
There is some limited API for working with ISO files.
PCSX.getCurrentIso()
will return anIso
object representing the currently loaded ISO file by the emulator. The following methods are available:
1 2 3 |
|
The :open
method has some magic built-in. The size argument is optional, and if missing, the code will attempt to guess the size of the underlying file within the Iso. This can only work on MODE2 FORM1 or FORM2 sectors, and will result in a failed File object otherwise. The mode argument is optional, and can be one of the following:
'GUESS'
: will attempt to guess the mode of the file. This is the default.'RAW'
: the returned File object will read 2352 bytes per sector.'M1'
: the returned File object will read 2048 bytes per sector.'M2_RAW'
: the returned File object will read 2336 bytes per sector. This can't be guessed. This is useful for extracting STR files that require the subheaders to be present.'M2_FORM1'
: the returned File object will read 2048 bytes per sector.'M2_FORM2'
: the returned File object will read 2324 bytes per sector.
The resulting File object will cache a single full sector in memory, meaning that small sequential reads won't read the same sector over and over from the disk.
The ISOReader object has the following methods:
1 |
|
Events
The Lua code can listen for events broadcasted from within the emulator. The following function is available to register a callback to be called when certain events happen:
1 |
|
Important: the return value of this function will be an object that represents the listener itself. If this object gets garbage collected, the corresponding listener will be removed. Thus it is important to store it somewhere that won't get garbage collected right away. The listener object has a :remove
method to stop the listener before its garbage collection time.
The callback function will be called from an unsecured environment, and it is advised to delegate anything complex or risky enough to PCSX.nextTick
.
The eventName
argument is a string that can have the following values:
Quitting
: The emulator is about to quit. The callback will be called with no arguments. This is where you'd need to close libuv objects held by Lua through luv in order to allow the emulator to quit gracefully. Otherwise you may soft lock the application where it'll wait for libuv objects to close.IsoMounted
: A new ISO file has been mounted into the virtual CDRom drive. The callback will be called with no arguments.GPU::Vsync
: The emulated GPU has just completed a vertical blanking interval. The callback will be called with no arguments.ExecutionFlow::ShellReached
: The emulation execution has reached the beginning of the BIOS' shell. The callback will be called with no arguments. This is the moment where the kernel is properly set up and ready to execute any arbitrary binary. The emulator may use this event to side load binaries, or signal gdb that the kernel is ready.ExecutionFlow::Run
: The emulator resumed execution. The callback will be called with no arguments. This event will fire when callingPCSX.resumeEmulator()
, when the user presses Start, or other potential interactions.ExecutionFlow::Pause
: The emulator paused execution. The callback will be called with a table that contains a boolean namedexception
, indicating if the pause is the result of an execution exception within the emulated CPU. This event will fire on breakpoints too, so if breakpoints have Lua callbacks attached on them, they will be executed too.ExecutionFlow::Reset
: The emulator is resetting the emulated machine. The callback will be called with a table that contains a boolean namedhard
, indicating if the reset is a hard reset or a soft reset. This event will fire when callingPCSX.resetEmulator()
, when the user presses Reset, or other potential interactions.ExecutionFlow::SaveStateLoaded
: The emulator just loaded a savestate. The callback will be called with no arguments. This event will fire when callingPCSX.loadSaveState()
, when the user loads a savestate, or other potential interactions. This is useful to listen to in case some internal state needs to be reset within the Lua logic.GUI::JumpToPC
: The UI is being asked to move the assembly view cursor to the specified address. The callback will be called with a table that contains a number namedpc
, indicating the address to jump to.GUI::JumpToMemory
: The UI is being asked to move the memory view cursor to the specified address. The callback will be called with a table that contains a number namedaddress
, indicating the address to jump to, andsize
, indicating the number of bytes to highlight.Keyboard
: The emulator is dispatching keyboard events. The callback will be called with a table containing four numbers:key
,scancode
,action
, andmods
. They are the same values as the glfw callback set byglfwSetKeyCallback
.
Memory and registers
The Lua code can access the emulated memory and registers directly through some FFI bindings:
PCSX.getMemPtr()
will return acdata[uint8_t*]
representing up to 8MB of emulated memory. This can be written to, but careful about the emulated i-cache in case code is being written to.PCSX.getRomPtr()
will return acdata[uint8_t*]
representing up to 512kB of the BIOS memory space. This can be written to.PCSX.getScratchPtr()
will return acdata[uint8_t*]
representing up to 1kB for the scratchpad memory space.PCSX.getRegisters()
will return a structured cdata representing all the registers present in the CPU:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
|
Constants
The table PCSX.CONSTS
contains numerical constants used throughout the rest of the API. Keeping an up to date list here is too exhausting, and it's simpler to print them using pprint(PCSX.CONSTS)
.
Pads
You can access the pads API through PCSX.SIO0.slots[s].pads[p]
where s
is the slot number and p
is the pad number, both indexed from 1, Lua-style. So PCSX.SIO0.slots[1].pads[1]
accesses the first pad, and PCSX.SIO0.slots[2].pads[1]
accesses the second pad.
Each Pad table has the following functions:
1 2 3 |
|
The button constants can be found in PCSX.CONSTS.PAD.BUTTON
.
You can for instance press the button Down on the first pad using the following code:
1 |
|
Execution flow
The Lua code has the following API functions available to it in order to control the execution flow of the emulator:
PCSX.pauseEmulator()
PCSX.resumeEmulator()
PCSX.softResetEmulator()
PCSX.hardResetEmulator()
It's also possible to manipulate savestates using the following functions:
PCSX.createSaveState() -- returns a slice representing the savestate
PCSX.loadSaveState(slice)
PCSX.loadSaveState(file)
Additionally, the following function returns a string containing the .proto file used to serialize the savestate:
PCSX.getSaveStateProtoSchema()
Note that the actual savestates made from the UI are gzip-compressed, but the functions above don't compress or decompress the data, so if trying to reload a savestate made from the UI, it'll need to be decompressed first, possibly through the zReader File object.
Overall, this means the following is possible:
1 2 3 4 5 6 7 8 |
|
Messages
The globals print
and printError
are available, and will display logs in the Lua Console. You can also use PCSX.log
to display a line in the general Log window. All three functions should behave the way you'd expect from the normal print
function in mainstream Lua.
GUI
You can move the cursor within the assembly window and the first memory view using the following functions:
PCSX.GUI.jumpToPC(pc)
PCSX.GUI.jumpToMemory(address[, width])
GPU
You can take a screenshot of the current view of the emulated display using the following:
PCSX.GPU.takeScreenShot()
This will return a struct that has the following fields:
1 2 3 4 5 |
|
The Slice
will contain the raw bytes of the screenshot data. It's meant to be written out using the :writeMoveSlice()
method. The width
and height
will be the width and height of the screenshot, in pixels. The bpp
will be either BPP_16
or BPP_24
, depending on the color depth of the screenshot. The size of the data
Slice will be height * width
multiplied by the number of bytes per pixel, depending on the bpp
.
Breakpoints
If the debugger is activated, and while using the interpreter, the Lua code can insert powerful breakpoints using the following API:
1 |
|
Important: the return value of this function will be an object that represents the breakpoint itself. If this object gets garbage collected, the corresponding breakpoint will be removed. Thus it is important to store it somewhere that won't get garbage collected right away.
The only mandatory argument is address
, which will by default place an execution breakpoint at the corresponding address. The second argument type
is an enum which can be represented by one of the 3 following strings: 'Exec'
, 'Read'
, 'Write'
, and will set the breakpoint type accordingly. The third argument width
is the width of the breakpoint, which indicates how many bytes should intersect from the base address with operations done by the emulated CPU in order to actually trigger the breakpoint. The fourth argument cause
is a string that will be displayed in the logs about why the breakpoint triggered. It will also be displayed in the Breakpoint Debug UI. And the fifth and final argument invoker
is a Lua function that will be called whenever the breakpoint is triggered. By default, this will simply call PCSX.pauseEmulator()
. If the invoker returns false
, the breakpoint will be permanently removed, permitting temporary breakpoints for example. The signature of the invoker callback is:
1 2 3 |
|
The address
parameter will contain the address that triggered the breakpoint. For 'Exec'
breakpoints, this is going to be the same as the current pc
, but for 'Read'
and 'Write'
, it's going to be the actual accessed address. The width
parameter will contain the width of the access that triggered the breakpoint, which can be different from what the breakpoint is monitoring. And the cause
parameter will contain a string describing the reason for the breakpoint; the latter may or may not be the same as what was passed to the addBreakpoint
function. Note that you don't need to strictly adhere to the signature, and have zero, one, two, or three arguments for your invoker callback. The return value of the invoker callback is also optional.
For example, these two examples are well formed and perfectly valid:
1 2 3 4 5 6 7 8 9 10 |
|
The returned breakpoint object will have a few methods attached to it:
:disable()
:enable()
:isEnabled()
:remove()
A removed breakpoint will no longer have any effect whatsoever, and none of its methods will do anything. Remember it is possible for the user to still manually remove a breakpoint from the UI.
Case studies
Spyro: Year of the Dragon
By looking up some of the gameshark codes for this game, we can determine the following memory addresses:
0x8007582c
is the number of lives.0x80078bbc
is the health of Spyro.0x80075860
is the number of unspent jewels available to the player.0x80075750
is the number of dragons Spyro released so far.
With this, we can build a small UI to visualize and manipulate these values in real time:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
|
You can see this code in action in this demo video.
Crash Bandicoot
Using exactly the same as above, we can repeat the same sort of cheats for Crash Bandicoot. Note that when the CPU is being emulated, the DrawImguiFrame
function will be called at least when the emulation is issuing a vsync event. This means that cheat codes that regularly write to memory during vsync can be applied naturally.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
Crash Bandicoot - Using Conditional BreakPoints
This example will showcase using the BreakPoints and Assembly UI, as well as using the Lua console to manipulate breakpoints.
Crash Bandicoot 1 has several modes of execution. These modes tell the game what to do, such as which level to load into, or to load back into the map. These modes are passed to the main game loop routine as an argument. Due to this, manually manipulating memory at the right time with the correct value to can be tricky to ensure the desired result.
The game modes are listed here.
In Crash 1, there is a level that was included in the game but cut from the final level selection due to difficulty, 'Stormy Ascent'. This level can be accessed only by manipulating the game mode value that is passed to the main game routine. There is a gameshark code that points us to the memory location and value that needs to be written in order to set the game mode to the Story Ascent level.
30011DB0 0022
- This is telling us to write the value 0x0022 at memory location0x8001db0
0x0022 is the value of the Stormy Ascent level we want to play.
The issue is that GameShark uses a hook to achieve setting this value at the correct time. We will set up a breakpoint to see where the main game routine is.
Setting the breakpoint can be done through the Breakpoint UI or in the Lua console. There is a link to a video at the bottom of the article showing the entire procedure.
Breakpoints can alternatively be set through the Lua console. In PCSX-Redux top menu, click Debug → Show Lua Console
We are going to add a breakpoint to pause execution when memory address 0x8001db0 is read. This will show where the main game loop is located in memory.
In the Lua console, paste the following hit enter.
1 |
|
You should see where the breakpoint was added in the Lua console, as well as in the Breakpoints UI. Note that we need to assign the result of the function to a variable to avoid garbage collection.
Now open Debug → Show Assembly
Start the emulator with Crash Bandicoot 1 SCUS94900
Right before the BIOS screen ends, the emulator should pause. In the assembly window we can see a yellow arrow pointing to 0x80042068
. We can see this is a lw
instruction that is reading a value from 0x8001db0
. This is the main game loop reading the game mode value from memory!
Now that we know where the main game loop is located in memory, we can set a conditional breakpoint to properly set the game mode value when the main game routine is executed.
This breakpoint will be triggered when the main game loop at 0x80042068
is executed, and ensure the value at 0x80011db0
is set to 0x0022
In the Lua console, paste the following and hit enter.
1 2 3 |
|
We can now disable/remove our Read breakpoint using the Breakpoints UI, and restart the game. Emulation → Hard Reset
If the Emulator status shows Idle, click Emulation → Start
Once the game starts, instead of loading into the main menu, you should load directly into the Stormy Ascent level.
You can see this in action in this demo video.