To provide a unified backend for all games, no
matter if they have AVS 2 available or now, use the
property (node) abstraction for any means of
configuration. This is somewhat annoying and a pain
because the property API isn't amazing. However,
the past has shown that having different means of
providing and handling (structured) configuration
data isn't great either.
Thus, have a non AVS implementation that allows
AVS independent applications/old games to use
the same core bemanitools API.
Summary:
Test Plan:
Summary:
Test Plan:
There is no need for different implementations
for this depending on the property (node) backend.
Thus, have these as ext(ension) functions that
already use parts of the actual API to implement
logging property (node) structures.
The previous implementation just went along with how AVS
property models and deals with attributes: attributes are just
another type of node that follow the same rules as standard
xml nodes/elements.
This design already showed several flaws in application as
these nodes always had to be referred to with an appended
‘@‘ on the attribute keys. This was mitigated with an “ext”
function that hides the whole details of “find the attribute node
before reading the attribute” steps.
By adding another property_node implementation with mxml,
the AVS style abstract layer showed incompatibilities with
the “an attribute is a node” approach. mxml doesn’t treat them as a type of node and just addresses them directly using their keys. This is lot simpler and aligns with how handling
attribute is done throughout the code thus far.
Refactor the property_node interface and the current AVS
implementation to also adapt this. Hide the detail that AVS
treats attributes as nodes and also the whole “append @“
to the keys notation.
A long awaited solution to address several
long-standing problems:
* Remove the need for log-server band-aid for iidx
versions using ezusb and the modern AVS logging
System, i.e. iidx 19 to 24. Any non AVS thread calling
the AVS log functions will crash due the mutex
implementation in AVS being not compatible with
non AVS threads (see the journal entry in the dev docs
for the whole story).
* Discontinue using the AVS log functions instead of
bemanitools’s own ones. There was never a technical
reason to do that, actually. It was neat to share one common
interface, but that was about it.
* Fixes performance issues on several games such as stuttering
brief slowdowns and de-syncs
Why not make this an integral part of the logger? There are
situations where async logging is not desired, e.g. ensure
predictiable logging output during debugging/development
tasks, and performance doesn’t matter at this point.
With the decision to implement this as a sink, the async
component can be easily composed arbitrarily with the other
existing log sinks.
Make this configurable. Normally the buffer size
doesn’t need to be 64k, that’s only required for
very verbose debug/development logging output.
Furthermore, the parameter needs to shared and
aligned with the async logger which has it’s own
buffer size. This avoids tight coupling between
the async sink and the logger.
This is required to ensure colored log messages
are executed as an atomic operation. Otherwise,
the color code information written to the terminal
could interleave with concurrent log messages
which results in “randomly” colored messages or
parts of them.
**THIS IS A HIGHLY WORK/DEVELOPMENT IN PROGRESS VERSION**
**THINGS ARE BROKEN AND EVERYTHING IS SUBJECT TO CHANGE**
First cut after massive refactoring with most critical and fundamental changes implemented.
We need to start somewhere, and this might be as good as anything else, so we can get started
with testing, bug fixing and iterating for the next releases.
The following list is non-exhaustive, does not guarantee anything does work, yet, and is supposed
to give a high level idea of what all of this is about. Updated documentation will reflect all of
this at some later point in time in more detail.
* A common "core" now abstracts logging, thread, property and configuration infrastructure and
provides a common interface. This is used by bemanitools internally as well as all tools, hooks
and APIs provided and don't depend on the game, version of the game or AVS version available
anymore
* New bemanitools (public) API
* Versioned API allowing for handling incremental API changes as well as breaking changes by
providing a new/different version when necessary
* Unified interfaces for bemanitools core API, i.e. logging, threads, configuration
* SDK with examples (TBD)
* Dogfooding approach: Bemanitools uses its own (public) API to implement and provide fundamental
features like configurable keyboard implementations for IO or hooks for different games and
versions
* All bemanitools hooks and IO libraries have been or are about to be re-worked to use the new APIs
* New hook API allows for more fine grained runtime control when stages of the hook are to be
executed, i.e. pre AVS, before main game, iat hooking instead of relying purely on DllMain
(which is still a compatible option though)
* launcher as a replacement for bootstrap: Bring it significantly closer to the original bootstrap
by supporting completely vanilla data and bootstrap.xml configurations to run the games. Note
that bemanitools does not include any code or means to run DRM'd data, only decrypted
* inject is also being reworked to use as much of the same "infrastructure" as launcher to provide
a more seamless bootstrapping process for games that keeps pre-eapki data as vanilla as possible
Summary:
Test Plan:
Summary:
Test Plan:
Summary:
Test Plan:
Summary:
Test Plan:
This module contains the "core" (API) of
bemanitools which includes an abstraction
layer for threads and logging at this time.
The threads API is very close to what
util/thread already was with some structural
enhancements which make it easier to understand
and work with the API, I hope. Some additional
helpers (*-ext module) support in doing common
tasks, e.g. setting up the thread API with other
modules.
The log(ging) part receives a major overhaul to
address known limitations and issues with the
util/log module:
- Cleaner API layer
- Separate sinks from actual logging engine
- Sinks are composable
- Improved and cleaner compatibility layer
with AVS logging API
Additional "extensions" (*-ext modules) add
various helper functions for common tasks like
setting up the logging engine with a file and stdout
sink.
The sinks also improved significantly with the file
sink now supporting proper appending and log rotation.
Logging to stdout/stderr supports coloring of log
messages which works across logging engines.
Overall, this refactored foundation is expected to
support future developments and removes known
limitations at the current scale of bemanitools such as:
- Reducing boiler plate code across hooks
- Interop of bemanitools and AVS (and setting the foundation
for addressing currently missing interop, e.g. for
dealing with property structures without AVS)
- Addressing performance issues in the logging engine
due to incorrect interop with AVS
icex2