Abstract: The External Development System provides external developers with a simple, easy-to-use method of storing runtime data. All of the loading, saving, and serialization is handled by the API, leaving the developer free to focus on their application.
Introduction
Externals have to store a lot of data: preferences, high scores, user statistics, etc. Classic externals (those written in Pascal or C and compiled as 68k code resources) had to come up with their own way of storing this data. Every developer managed their data a bit differently and, as a result, the SysOp had to deal with data files appearing all over their hard drive.
Upgrading externals from one version to another often involved running some sort of data migration utility. In some cases, upgrading the data would be far too complex and the SysOp had to reinitialize the external.
When I set out to design the new, Python-based Hermes API, my goal was to eliminate all of this tedious data juggling. Instead of using data files, I decided to provide a property-based, hierarchical data storage system. This system would take care of the loading, saving, marshaling, and unmarshaling of external data. External developers would not need to concern themselves with these tasks. Instead, developers were free to focus on the fun part: writing externals.
This concept greatly simplifies the development of an external and lets an external author build complex externals with much less code than would have been necessary using the classic API. The same Python-based API is also used to access BBS-related data such as global statistics, preferences, and user information.
Other than resource files (which are read-only), externals are not allowed to read and write to the file system. The properties API is the only form of mutable storage available to Python externals. This restriction allows SysOps to run untrusted externals without being concerned about the security and safety of the data on their computer.
Property Types
Properties in the Hermes API are simple key-value pairs. Each property is identified by a unique key and can store any one of the following Python types:
- int and string: Basic integers and strings.
- tuple, list, and dict: Python tuples, lists, and dictionaries can be stored in a property, but all of the contained elements must be primitive Python types. The Hermes API enforces this recursively, so a list-of-a-list-of-a-list-of-objects would not be allowed.
To clarify, you cannot store Python objects in a data property. This restriction allows the Hermes API to easily migrate data between flat files, a database, etc. It also lets us sidestep the complex issues related to class versioning.
The system properties are maintained by the BBS itself and are stored in the BBS’ private data files. Properties created by an external are stored separately. You cannot access this data store from Python other than through the properties interface.
Property Groups
There are five major groups of properties in the Hermes API. These groups represent various components of the BBS as well as the expected lifetime of the data. Each of the groups is described below:
- bbs: The bbs group contains BBS-wide information and statistics. The name of the BBS, the number of nodes on the BBS, etc. would be stored here. An external developer might use the bbs group to store a global high-score list or other statistics about the external. This is not the place for preference information, that kind of data is stored in the prefs group.
- prefs: Preferences-related information about the BBS and its externals are stored in the prefs group. Externals should store SysOp-editable preferences here such as turns-per-day, unlimited SysOp play, etc.
- user: Information about each user is stored in the user group. This is the place for externals to store per-user information needed for the external. If a user is deleted, the data in each external’s user property tree is deleted as well.
- instance: The instance group is unique in that it is not stored on disk. Instead, this group is created each time a user enters the external and is destroyed when the user exits the external. You can think of the instance group as a place to store global variables. You could use real, Python globals instead of the instance group, but using instance allows you to manipulate the data in the Hermes Python Console as well as any development environments that may be produced for the Hermes API.
- external: This group provides generated properties that are useful for gathering information about the external itself. For example, the external group contains a property that lists all of the users that have accessed the current external. Although you can store persistent information in this group, I have yet to find a property that would not be better stored in one of the other groups. Please contact me if you believe that you have found a use for the external group other than for accessing its predefined properties.
Accessing Properties
The Hermes API presents each of the property groups as a tree of objects stored in the hermes module. If you import the entire hermes module into your namespace (which is the recommended thing to do), then the property groups will be accessible as global variables in your code.
The base properties in each group are managed by the BBS itself. bbs.name returns the BBS name, user.realName returns the user’s real name, etc. Read and write access to these properties is controlled by the Hermes Python Runtime. Externals cannot read a user’s password, for example.
Each of the global property objects contains a special data object that stores the data for the current external. Externals can create any number of properties inside of the data object by using the Python assignment operator:
# Create some properties.
user.data.foo = 'bar'
user.data.baz = [1, 2, 3]
user.data.mapping = { 'one': 1, 'two': 2, 'three': 3 }
New properties are automatically created by the Hermes API on their first assignment. Just like all Python properties, uppercase and lowercase are significant.
Externals cannot create new properties outside of the data object. Attempting to assign a value to one of the global property objects will throw an exception.
The data object does not currently support additional levels of hierarchy. In other words, you cannot do something like this:
# This will fail:
user.data.foo.bar.baz = 'abc'
Tips and Tricks
There are a number of common actions that most externals will need to perform at one time or another. The properties API provides an easy way to perform these actions. Sample code for these actions is listed below.
Checking to see if this is the first time that your external has been run. Externals that store data in the bbs group may need to initialize some of their global data structures the first time that the external is run on a new BBS. You can test for the existence of the bbs.data property to see if your external has been run before. For example:
# Initialize our BBS data if this is the first run.
if not bbs.data:
bbs.data.highScores = []
bbs.data.lastWinner = "No one!"
# etc.
# .. continue with your external ..Do not assume that missing BBS data means that your preferences need to be initialized. The SysOp could have specifically cleared your external’s BBS data and expects their custom preferences to be preserved.
Checking to see if your initial preferences need to be created. The same trick used above can be used to initial your preferences on the first run of your external:
# Initialize our preferences if this is the first run.
if not prefs.data:
prefs.data.playsPerDay = 3
prefs.data.unlimitedSysopPlay = True
# etc.
# .. continue with your external ..Do not assume that missing preferences data means that your BBS data needs to be initialized. The SysOp could have specifically cleared your external’s preferences data and expects the BBS data to be preserved.
Checking to see if this is the first time a user has entered your external. You can check the external.users list to see if a user has entered your external before.
Note that a user will only be in this list if you have stored at least one property value for that user. If you do not store anything in the user.data properties tree for a user, then the user will not be in the external.users list. This allows you to prompt the user to register with the external before initializing their properties.
Here is an example of querying a new user to join the external and initializing their properties if they decide to register:
# Create an account for this user if necessary.
if user not in external.users:
# See if they want to join the game.
if not yesNoPrompt('Do you want to join the game? '):
return
# They want to play; initialize their properties.
user.data.playsToday = 0
user.data.score = 0
# etc.
# .. continue with your external ..Clearing user data. You can use the user.deleteData() method to remove a user from your external’s data store. After this method has been called, the user.data object will be cleared and the user will be removed from the external.users list. In other words, the user will appear to be a new user the next time they enter your external.
Note that the user.data object will immediately be reset by this call. You will no longer be able to access any properties that were configured before user.deleteData() was called. If you have cached a reference to the user.data object, then your cached copy will still be valid. Any changes you make to this copy will not be stored in the user’s property tree.
# Delete the user's properties for this external.
user.deleteData()
Sample Code
Leech 2000 makes extensive use of properties to store all of its data. The usage of these properties is detailed on the main Leech 2000 page. You can also browse the Leech 2000 source code to see how the properties are accessed and stored. Leech 2000’s main.py file contains all of the initialization code for the bbs.data, prefs.data, and user.data objects.