This is implemented by inserting special opcodes into the game's code.
When the breakpoint handler is finished, it returns the original opcode
so that execute_instruction can continue executing normally.
E.g.
* start xsystem4 with --debug option to drop into debugger immediately
* run (bp "game_main")
* ^d or \exit to close debugger REPL
* ...drop into a debugger REPL again at the game_main function
global config in $XSYSTEM4_HOME/.xsys4rc (default ~/.xsys4rc)
game-specific config in <game-dir>/.xsys4rc
Supported options so far are "font-mincho" and "font-gothic". Syntax is
the same as System40.ini. E.g.
font-mincho = "/home/user/.fonts/msmincho.ttc"
Even though this game is still ain v14, it makes an incompatible change
to the file format: struct definitions now contain a list of functions
at the end (I believe this is a listing of the struct's vtable).
There is now a system for tracking the minor version of the ain file
format. This is a made up value, determined by the file name of the ain
file.
This commit also fixes an issue where the empty string was being added
as a new string to the string table when rebuilding ain files with
ainedit.
Fixes#2.
The problem here was that init_member_functions assumed ASCII or SJIS
encoding of function/struct names in the ain file. Now it will convert
to UTF-8 when analyzing member functions.
This commit also removes the --ain-encoding option to ainedit.
--output-encoding should always match the encoding of the input ain
file. --transcode can be used to change the encoding before running
running further ainedit commands.
This is mainly for testing purposes. Defaults to creating an ain v4
file, which is the same version produced by the SDK compiler.
Also various small improvements so that output matches the SDK compiler:
* add "0" function (currently does nothing)
* add "NULL" function
* compile null return at the end of every function
* set main function index in ain file
Initial support for compiling .jaf (System 4 language) files. So far
this is limited to adding struct definitions and declaring new globals.
The actual bytecode compilation is not yet implemented.
This Bison grammar is based on a C grammar and should cover 95% of the
System 4 language syntax already, though many AST nodes are not yet
implemented.
* Can now extract files from .afa version 3 archives
* Can now extract files from .flat files
* Includes recursively extracting .flat files contained in other
archive formats
* Added option to extract images only
* Now converts images to PNG format by default
* WebP format also supported
Because the table of contents is encrypted in .afa version 3 archives,
the support is a bit limited. Only known file types can be extracted and
they do not retain their original filenames (they are given numbered
filenames instead).
Evenicle2EX.ex uses the previously unknown field data in a different way
that what was assumed. Thanks to that, the unknown data is now more or
less understood:
* uk0 indicates whether there is a value associated with the field
* uk1 indicates whether the field is an index
* uk2 is the value associated with the field, which can be an integer,
float or string (so far, at least...)
Previously it was assumed that uk2 was a single 32-bit integer, but it
can actually store strings as well. Evenicle 2 is the first game to use
a field with a string value other than the empty string, which breaks
this assumption (the empty string is a 32-bit zero, which is why the old
approach worked).
There is now a new syntax for specifying this data. E.g., a field may
now look like this:
indexed string s = ""
where it would previously have looked like this:
string s[1,1,0]
The old syntax is still supported in ainbuild for backwards
compatibility, but is no longer emitted by aindump.
This commit also adds the tool "excmp" which compares two ex files to
see if they are equivalent.
Support for Evenicle 2 and Haha Ranman.
There are a few changes in v14:
* a number of new instructions (semantics yet unknown)
* HLL function arguments and return values can now be complex types
(e.g. arrays, enums)
* support for nested arrays (this isn't necessarily new, but it doesn't
occur in Rance X at all)
Rance03.ex has a broken table in which a field type does not match the
type of the data in the corresponding column. So rather than aborting
with an error, exdump will now print a warning when dumping this file.
Same idea as LOCALREF/GLOBALREF, etc. except for accessing struct
members inside of a member function. This is a bit trickier since it
requires knowing which functions belong to which structs, which isn't
directly encoded in the AIN format.
Enough to get text on the screen, but it isn't pretty. No antialiasing,
kerning, vertical placement isn't correct, probably the size isn't even
quite correct, etc.
Add tool for extracting bitmap fonts from .fnl files (font archives).
This is mainly for testing the underlying .fnl reading code, which is
needed for proper font rendering in the MangaGamer version of Sengoku
Rance. In the future this tool could be updated to support rebuilding
font archives, which would allow adding special glyphs, using condensed
fonts, etc.
Credit to CookieWukk for reverse engineering the .fnl file format.
For the MangaGamer version of Sengoku Rance. This makes libwebp a
dependency.
These webp files have a non-standard metadata block appended to them
which specifies the base CG that should be used for unfilled pixels,
where "unfilled" means any pixels with a pure magenta color (255,0,255).
This is not really a standard ini file, so a custom parser is needed to
access all of the information stored in it (specifically, lists like
"LoadDLL").
This commit also makes flex/bison a hard dependency.
Very incomplete implementation. Capable of reading the file table and
not much else.
Also add the "alice-ar" utility for testing. Eventually this tool should
support modifying ald and afa archives. For now only supports dumping
the afa file table.
Add tool for dumping the contents of .ex files, which contain various
structured data used by the game (tables, lists, trees).
The dump format uses a C-like syntax which should be relatively easy to
parse back in.
Add a pseudo instruction to represent switch case labels, along with the
infrastructure to support pseudo instructions in general. .CASE takes
two arguments: a switch:case index (e.g. "12:8") and the value to match
against.
Also, add a --no-strings option to both programs to allow modifying the
code without changing the string/message tables. This is needed for now
since rebuilding the string/message tables (as currently implemented)
causes serious bugs in Rance X (and probably other games).
Running the following command should produce an ain file that is almost
bit-for-bit identical to the input file (the only difference being minor
drift in floating point constants):
aindump -c --no-strings in.ain | ainedit -c - --no-strings -o
out.ain in.ain
The ainedit utility is a tool capable of modifying AIN files up to
version 12 (Rance X). As far as I know, this is the first publicly
released tool that can handle AIN files above version 7.
The basic mode of operation is as follows:
* dump the CODE section with aindump
* edit the disassembled bytecode
* assemble & insert the edited bytebode back into the original AIN file
using ainedit
E.g.
aindump -c -o out.jam /path/to/Rance10.ain
$EDITOR out.jam
ainedit -c out.jam -o out.ain /path/to/Rance10.ain
mv /path/to/Rance10.ain /path/to/Rance10.ain.backup
mv out.ain /path/to/Rance10.ain
ainedit is also capable of editing declarations (structures, functions,
etc.) using the JSON (-j) output from aindump.
There is currently one major deficiency in this tool: it does not
rebuild the AIN file's switch table (SWI0 section). This means that you
can't insert or delete instructions, as it would render the addresses
stored in the switch table invalid. I hope to fix this soon.
A number of changes have also been made to the aindump utility, since
these tools need to work together.
This requires scanning through the code for a particular method for each
enum type. Not a great solution, but there's not much else that can be
done here.
Also, use khash.h (from klib) for hash tables.
In ain v11+, the first and only argument to CALLMETHOD is the number of
arguments that are being passed to the method. The calling convention
is:
PUSH <struct page>
PUSH <function index>
PUSH <arg 0>
...
PUSH <arg n>
CALLMETHOD <n>
Type 82
-------
I am assuming this is an iterator type because it always appears as a
local variable alongside "dummy" variables with "foreach" in their name,
and the naming of the variables suggests an iterator. Probably there is
a new syntax like:
foreach (iter_var in array_var) { ... }
Type 89
-------
An instance of an interface. This is a 2-valued type (similar to ref
int, etc.) where the first value is the index of a struct page, and the
second value is an index into the struct's vtable.
Any struct that implements an interface gets an array<int> named
"<vtable>" as its first member. This array stores indices into the
global function table. CALLMETHOD now looks up methods in the vtable
rather than the global function table.
Types 92, 93
------------
These are enum types. Type 92 is the normal enum type, and type 93 is
the reference-to-enum type. The "struct type" for these types stores an
index into the enum table.
Types 86, 91
------------
Type 86 only seems to be used as the return value from EnumType::Parse
methods. In the ain file, it has an extra type descriptor, similar to an
array type. This extra type descriptor always has data type 91, and its
struct type is an index into the enum table. Type 91 is simlar to types
92 & 93: it stores an enum table index in its struct type.
Type 86 is a 2-valued type. The first value is the value of the enum (an
integer between 0 and n). The second value seems to be either 0 or -1,
depending on whether EnumType::Parse was able to produce a valid enum
value.
It is possible that type 86 is actually some kind of generic maybe-error
type that only happens to be used by the enum code. But that doesn't
explain why it is paired with type 91 and not type 92.
Print the new v11+ array types properly. Also, print the array rank for
old array types.
The code for reading v11+ ain data is simplified a lot here as well.
I've figured out what most of the extra fields are for. Mostly they are
for:
* a new array type, in which the contained value type is stored in the
variable object rather than encoded in the type number
* argument initvals (I guess the VM is now responsible for these?)
* interfaces
Use the correct argument types for all instructions, with the exception
of SH_REF_LOCAL_ASSIGN_STRUCTREF2 which I could find an example of.
List all system calls.
v11 ain files have 3 new instructions, and 3 modified instructions. It
looks like they decided to break compatibility with this release by
adding arguments to some instructions.
v12 adds one instruction.
Nunuhara Cabbage