ftype

An ftype is an abstraction of a raw C type. It is usually applied to describe struct fields, which is where the name comes from (field type), but it represents any C type in our struct API. A struct definition itself, for example, is again an ftype, as it, too, can be used as the base type for another struct field.

The basic ftypes (corresponding to basic C types) are:

struct.int8     -- represents a int8_t
struct.int16    -- represents a int16_t
struct.int32    -- represents a int32_t
struct.int64    -- represents a int64_t
struct.uint8    -- represents a uint8_t
struct.uint16   -- represents a uint16_t
struct.uint32   -- represents a uint32_t
struct.uint64   -- represents a uint64_t
struct.float    -- represents a float
struct.double   -- represents a double
struct.bool     -- represents a bool
struct.bit      -- represents a bit-packed value
struct.ptr      -- represents a pointer value

These types are trivially employed and simply translate to the native C types they represent when formulating a struct. As for struct.bit, it takes an underlying type and translates that to raw bit fields in C.

Usage

They are usually used as struct fields:

struct.struct({
    int_field           = {struct.int32},
    float_field         = {struct.float},
    bool_field          = {struct.bool},
})

Since SE uses a lot of bit-packed values, especially in older packets, we provide a struct.bit type to define bit fields. It can only be used by specifying an underlying base type:

struct.struct({
    bottom10            = {struct.bit(struct.uint16, 10)},
    top6                = {struct.bit(struct.uint16, 6), offset = 10},
})

Note that the offset is defined outside of the inner ftype. It is metadata of the field description, as described on the struct definition page.

Further we can use struct.ptr to define pointers to a base type:

local entry_type = struct.struct({
    int_field           = {struct.int32},
    float_field         = {struct.float},
})

struct.struct({
    ptr                 = {struct.ptr(entry_type)},
})

This can now be used to describe pointer-based structs. Especially useful for modeling existing memory structs.

Array types

Basic types can be arranged in an array with struct.array:

struct.struct({
    arr_field           = {struct.array(struct.int32, 10)},
})

Since this happens fairly frequently a shortcut was introduced, by just indexing any ftype with a count:

struct.struct({
    arr_field           = {struct.int32[10]},
})

This can now be used to define string types. In C you cannot easily vary the length of strings without resorting to pointers and that introduces both new boilerplate to read/write the data as well as potential bugs by the need to allocate and deallocate memory. So strings in structs are usually given a fixed length. Doing that we can use a char array:

struct.struct({
    string_field        = {struct.int8[10]},
})

We do not provide a particular wrapper around char, so int8_t will have to do. Since we would need to read it via ffi.string it may as well be int8_t, since that function does not discriminate between the underlying type.

However, this means we have to use ffi.string to read and ffi.copy to write to the struct, and we have to take the length into account, including null terminators and overflow protection. Since this still happens very often, especially in packets, we can use the converter mechanism to automate this process.

Converter types

A converter type can be based on any underlying type (including struct and array types) and provides converters to read and write the values from and to memory.

This brings us to struct.string, which is a function that takes a size (in bytes) and returns an ftype of the corresponding size. struct.string(size) creates a char[size] field in the underlying C data, but it also provides converter functions between Lua and C. On read it reads a null-terminated string for at most size characters. On write it writes at most size - 1 characters and then a null byte (or sooner, if the string ends sooner).

Following is a list of types with custom converters and short descriptions:

struct.string(size)

Represents a readable string of at most size - 1 bytes.

struct.data(size)

Represents arbitrary data. Returns it in a string format, of exactly size bytes. Used for binary data.

struct.bitfield(size)

Represents a bit field. Returns a table with 8 * size boolean entries. Used for flags and large bit fields (like key items).

struct.time

Represents a time. Returns a timestamp adjusted for local time.

Note: This implementation may change.

struct.packed_string(size, lookup)

Represents a bit-packed string taking up size bytes. Fixed to 6-bit packing, as that seems to be the only type of bit-packed strings SE uses, so struct.packed_string(size, lookup) will result in a readable string of 4 / 3 * size. This also means that size needs to be divisible by 3.

SE also use different lookup strings for different situations (item signatures, LS names in extdata, LS names in /lsmes, etc.). Examples of those can be found in the types.lua file.

struct.boolbit

Represents a boolean bit. Based on struct.bit, its converter simply converts 1 to true and 0 to false.

Info

Just like individual fields can have metadata, whole ftypes can also have metadata. This is most commonly used for signatures, as in the memory library:

types.party = struct({signature = '6A0E8BCE89442414E8????????8B0D'}, {
    members                 = {0x2C, party_member[18]},
})

The packet type definition file also uses it often to define the size of a struct explicitly:

local equipset_entry = struct({size = 4}, {
    bag_index           = {0x00, uint8},
    slot_id             = {0x01, slot},
    bag_id              = {0x02, bag},
})

If not provided the size is determined automatically. However, if the type is used in an array the exact size needs to be given or the array will be misaligned.

Another common field

Like field metadata this can be used to hold metadata for an ftype to use later. And like field metadata certain keys are used by our libraries and should not be used by developers:

• signature, static_offsets, offsets used for determining memory locations of structs
• size to set an explicit size
• cache to determine what key to cache packets by