Initial support for compiling .jaf (System 4 language) files. So far
this is limited to adding struct definitions and declaring new globals.
The actual bytecode compilation is not yet implemented.
This Bison grammar is based on a C grammar and should cover 95% of the
System 4 language syntax already, though many AST nodes are not yet
implemented.
* Can now extract files from .afa version 3 archives
* Can now extract files from .flat files
* Includes recursively extracting .flat files contained in other
archive formats
* Added option to extract images only
* Now converts images to PNG format by default
* WebP format also supported
Because the table of contents is encrypted in .afa version 3 archives,
the support is a bit limited. Only known file types can be extracted and
they do not retain their original filenames (they are given numbered
filenames instead).
Evenicle2EX.ex uses the previously unknown field data in a different way
that what was assumed. Thanks to that, the unknown data is now more or
less understood:
* uk0 indicates whether there is a value associated with the field
* uk1 indicates whether the field is an index
* uk2 is the value associated with the field, which can be an integer,
float or string (so far, at least...)
Previously it was assumed that uk2 was a single 32-bit integer, but it
can actually store strings as well. Evenicle 2 is the first game to use
a field with a string value other than the empty string, which breaks
this assumption (the empty string is a 32-bit zero, which is why the old
approach worked).
There is now a new syntax for specifying this data. E.g., a field may
now look like this:
indexed string s = ""
where it would previously have looked like this:
string s[1,1,0]
The old syntax is still supported in ainbuild for backwards
compatibility, but is no longer emitted by aindump.
This commit also adds the tool "excmp" which compares two ex files to
see if they are equivalent.
Support for Evenicle 2 and Haha Ranman.
There are a few changes in v14:
* a number of new instructions (semantics yet unknown)
* HLL function arguments and return values can now be complex types
(e.g. arrays, enums)
* support for nested arrays (this isn't necessarily new, but it doesn't
occur in Rance X at all)
Rance03.ex has a broken table in which a field type does not match the
type of the data in the corresponding column. So rather than aborting
with an error, exdump will now print a warning when dumping this file.
Same idea as LOCALREF/GLOBALREF, etc. except for accessing struct
members inside of a member function. This is a bit trickier since it
requires knowing which functions belong to which structs, which isn't
directly encoded in the AIN format.
Enough to get text on the screen, but it isn't pretty. No antialiasing,
kerning, vertical placement isn't correct, probably the size isn't even
quite correct, etc.
Add tool for extracting bitmap fonts from .fnl files (font archives).
This is mainly for testing the underlying .fnl reading code, which is
needed for proper font rendering in the MangaGamer version of Sengoku
Rance. In the future this tool could be updated to support rebuilding
font archives, which would allow adding special glyphs, using condensed
fonts, etc.
Credit to CookieWukk for reverse engineering the .fnl file format.
For the MangaGamer version of Sengoku Rance. This makes libwebp a
dependency.
These webp files have a non-standard metadata block appended to them
which specifies the base CG that should be used for unfilled pixels,
where "unfilled" means any pixels with a pure magenta color (255,0,255).
This is not really a standard ini file, so a custom parser is needed to
access all of the information stored in it (specifically, lists like
"LoadDLL").
This commit also makes flex/bison a hard dependency.
Very incomplete implementation. Capable of reading the file table and
not much else.
Also add the "alice-ar" utility for testing. Eventually this tool should
support modifying ald and afa archives. For now only supports dumping
the afa file table.
Tool for reading and rebuilding dumped .ex files. Currently this only
reads a file on stdin then redumps it to stdout.
Also, move tools code into src/tools/
Add tool for dumping the contents of .ex files, which contain various
structured data used by the game (tables, lists, trees).
The dump format uses a C-like syntax which should be relatively easy to
parse back in.
If the filenames section isn't present, try to guess the filenames from
the functions defined in the file. This works pretty well for most of
the core engine files but gives pretty crappy results for scenario
files.
This feature is primarily intended to be used when dumping the code
section to multiple files (not yet implemented).
Add a pseudo instruction to represent switch case labels, along with the
infrastructure to support pseudo instructions in general. .CASE takes
two arguments: a switch:case index (e.g. "12:8") and the value to match
against.
Also, add a --no-strings option to both programs to allow modifying the
code without changing the string/message tables. This is needed for now
since rebuilding the string/message tables (as currently implemented)
causes serious bugs in Rance X (and probably other games).
Running the following command should produce an ain file that is almost
bit-for-bit identical to the input file (the only difference being minor
drift in floating point constants):
aindump -c --no-strings in.ain | ainedit -c - --no-strings -o
out.ain in.ain
The ainedit utility is a tool capable of modifying AIN files up to
version 12 (Rance X). As far as I know, this is the first publicly
released tool that can handle AIN files above version 7.
The basic mode of operation is as follows:
* dump the CODE section with aindump
* edit the disassembled bytecode
* assemble & insert the edited bytebode back into the original AIN file
using ainedit
E.g.
aindump -c -o out.jam /path/to/Rance10.ain
$EDITOR out.jam
ainedit -c out.jam -o out.ain /path/to/Rance10.ain
mv /path/to/Rance10.ain /path/to/Rance10.ain.backup
mv out.ain /path/to/Rance10.ain
ainedit is also capable of editing declarations (structures, functions,
etc.) using the JSON (-j) output from aindump.
There is currently one major deficiency in this tool: it does not
rebuild the AIN file's switch table (SWI0 section). This means that you
can't insert or delete instructions, as it would render the addresses
stored in the switch table invalid. I hope to fix this soon.
A number of changes have also been made to the aindump utility, since
these tools need to work together.
Dump everything except the code to a JSON file.
With both the disassembled code and this JSON file in hand, it should be
possible to reconstruct the original AIN file.
This requires scanning through the code for a particular method for each
enum type. Not a great solution, but there's not much else that can be
done here.
Also, use khash.h (from klib) for hash tables.
In ain v11+, the first and only argument to CALLMETHOD is the number of
arguments that are being passed to the method. The calling convention
is:
PUSH <struct page>
PUSH <function index>
PUSH <arg 0>
...
PUSH <arg n>
CALLMETHOD <n>
Type 82
-------
I am assuming this is an iterator type because it always appears as a
local variable alongside "dummy" variables with "foreach" in their name,
and the naming of the variables suggests an iterator. Probably there is
a new syntax like:
foreach (iter_var in array_var) { ... }
Type 89
-------
An instance of an interface. This is a 2-valued type (similar to ref
int, etc.) where the first value is the index of a struct page, and the
second value is an index into the struct's vtable.
Any struct that implements an interface gets an array<int> named
"<vtable>" as its first member. This array stores indices into the
global function table. CALLMETHOD now looks up methods in the vtable
rather than the global function table.
Types 92, 93
------------
These are enum types. Type 92 is the normal enum type, and type 93 is
the reference-to-enum type. The "struct type" for these types stores an
index into the enum table.
Types 86, 91
------------
Type 86 only seems to be used as the return value from EnumType::Parse
methods. In the ain file, it has an extra type descriptor, similar to an
array type. This extra type descriptor always has data type 91, and its
struct type is an index into the enum table. Type 91 is simlar to types
92 & 93: it stores an enum table index in its struct type.
Type 86 is a 2-valued type. The first value is the value of the enum (an
integer between 0 and n). The second value seems to be either 0 or -1,
depending on whether EnumType::Parse was able to produce a valid enum
value.
It is possible that type 86 is actually some kind of generic maybe-error
type that only happens to be used by the enum code. But that doesn't
explain why it is paired with type 91 and not type 92.
Print the new v11+ array types properly. Also, print the array rank for
old array types.
The code for reading v11+ ain data is simplified a lot here as well.
I've figured out what most of the extra fields are for. Mostly they are
for:
* a new array type, in which the contained value type is stored in the
variable object rather than encoded in the type number
* argument initvals (I guess the VM is now responsible for these?)
* interfaces
Use the correct argument types for all instructions, with the exception
of SH_REF_LOCAL_ASSIGN_STRUCTREF2 which I could find an example of.
List all system calls.
v11 ain files have 3 new instructions, and 3 modified instructions. It
looks like they decided to break compatibility with this release by
adding arguments to some instructions.
v12 adds one instruction.
The 'lint' type in system 4 is NOT a 64-bit integer type. It is actually
a signed 32-bit integer type that is clamped between 0 and INT32_MAX.
Negative numbers can be assigned to a lint using SH_LOCALASSIGN (or at
least, the unofficial compiler emits such code) but LI_* instructions
always produce a positive result.
I'm guessing System40.exe converts lints to 64-bit integers for doing
arithmetic, and then clamps the result to the 32-bit integer range
before writing back to a variable.
This fixes a bug in Sengoku Rance where units would become invincible in
commander battles after their HP fell below zero.
Replace many instruction arguments with names/values (strings/messages,
local/global variables, structs, functions, etc.) and add labels to jump
targets.
- Create separate directories for source and header files.
- Build code shared between xsystem4 and aindump as a static library.
- Create header dirs gfx/ system4/ and vm/
- Make CG.c/h usable without a running VM
Nunuhara Cabbage