-
Notifications
You must be signed in to change notification settings - Fork 4
Interaction with Blockchain
This page describes the protocol, between the Scilla Run-time Library and the C++ Blockchain code (BC), for fetching and updating individual state variables of a contract.
SRTL, and all of the Scilla Virtual Machine (SVM) is designed to be independent of BC, and hence can be built as a shared library whose functions can be called from BC. To facilitate queries in the other direction, i.e., from SRTL to BC, we adopt the convention that BC registers certain mandatory functions with SRTL so that SRTL can callback through them. More on this later.
-
BVec. The fundamental type representing a sequence of bytes. - Map[k1][k2][...] ->
BVec. A possibly nested map, each of whose keys ki is aBVecvalue and the final value is alsoBVec.
BVec values are encoded as std::vector<unsigned char>. It is the responsibility of SRTL to serialize values of Scilla's primitive types and ADTs into BVec values.
Map typed values require extra discussion because of the requirement of serializing whole Maps. This requirement arises when the Scilla program accesses a map variable with lesser indices (including 0) than the depth of the Map variable. In other words, Scilla programs can operate on entire Maps, treating them as immutable values, and we need a way to encode them. Map values are encoded using protobuf (see next).
When all indices are provided, it is assumed that each access has a cost that is constant factor of the number of indices, summed up with the size of the fetched value itself (and gas charged accordingly). When Map values are communicated as a whole, the gas charged will be linear in the cost of the size of the Map value (which will be the sum of sizes of keys and values).
The following protobuf code shows encoding of Scilla's queries to BC for fetching and updating state variables and how the variables' values are encoded.
syntax = "proto2";
/*
* Communication b/w Scilla and the blockchain for fetching
* and updating state variables involve transfer of values.
*
* Scilla values are either sequence of bytes, or a Map.
*
* All non-Map Scilla variables are encoded as byte sequences
* by Scilla and the blockchain is unaware of it.
*/
message ScillaVal {
oneof ValType {
bytes bval = 1;
Map mval = 2;
}
// Unfortunately, we cannot use Protol Buffer's Maps
// as "bytes" keys are not allowed. So we define our own.
message Map {
message MapEntry {
required bytes key = 1;
required ScillaVal val = 2;
}
repeated MapEntry entries = 1;
}
}
/*
* A value fetch or update query by Scilla to the blockchain will
* will be described by this message.
*
* For value fetch, the retun value will be a ScillaVal while for
* value update, the query will be accompanied by a ScillaVal.
*/
message ScillaQuery {
required string name = 1;
// Max nesting depth of this Scilla state variable (0 for non-maps)
required uint32 mapDepth = 2;
// If indices < mapDepth, then the associated ScillaVal
// will have ScillaVal.ValType = ScillaVal.Map.
repeated bytes indices = 3;
// If ignore is true, then the ScillaVal argument associated with
// ScillaQuery is ignored. This means that:
// 1. For `updateStateValue` the key is deleted (if exists) from db.
// 2. For `fetchStateValue` only `found` is set and used, but the actual
// value itself is ignored by Scilla (and hence can be set to a dummy value).
// assert (mapDepth > 0 && indices > 0)
required bool ignoreval = 4;
}We now detail (via C++ like pseudo-code) out how Scilla states are laid out in the backend Level-DB database and how BC fetches and updates it based on ScillaQuery. No error handling is shown here. Error handling need not be a part of the protocol since errors are left to BC to handle in however it seems fit (including aborting the contract execution).
typedef std::vector<unsigned char> BVec;
// Fetch state variable from level-db and provide to Scilla.
// "found" is set based on whether a value is found or not.
// Return value indicates error, if any.
bool fetchStateValue(const BVec &query, BVec &value, bool &found)
{
ScillaQuery &q = ProtoBuf.deserialize(query);
ScillaValue v;
// Form the base of the key into leveldb.
BVec keyBase = contr_addr + "." + q.name;
BVec keyReady(keyBase);
// Append indices to the base key.
for (unsigned idx = 0; idx < q.indices.size(); idx++) {
keyReady.insert(keyReady.end(), ".");
keyReady.insert(keyReady.end(), q.indices[idx].begin(), q.indices[idx].end());
}
assert (q.indices <= q.mapDepth);
if (q.indices == q.mapDepth) {
// result will not be a map and can be just fetched into the store.
v.ValType = bytes;
// Result will be written to v.bval, assuming ProtoBuf's bytes is compatible
// with leveldb bytes. If not, do conversion b/w the two.
found = db->Get(keyReady, v.bval).ok();
return true;
}
// We're fetching a Map value. Need to iterate level-db lexicographically.
v.ValType = ScillaVal::Map;
leveldb::Iterator* it = db->NewIterator(leveldb::ReadOptions());
it->Seek(keyReady));
if (!isPrefix(keyReady, it->key()) {
// no entry.
found = false;
return true;
}
for (; isPrefix(keyReady, it->key()) && it->Valid(); it->Next()) {
// split key with "." as separator. FIXME: This is risky since a `ByStrX`
// Scilla key may happen to contain "." and we're splitting there !
std::vector<BVec> indices = splitKey(removePrefix(keyReady, it->key()), ".");
unsigned n = indices.size();
if (q.indices.size() + indices.size() < q.mapDepth) {
// We have to fetch a map value for "it". but since we don't store map
// values except for empty maps, this must be a protobuf-encoded empty map value.
assert("it->val() is a protobuf encoded empty map");
// This below mval[][][] syntax is a convenience notation for calling
// `createNestedMap` defined below. In real code, use `createNestedMap`.
v.mval[indices[0]][indices[1]]...[indices[n]].ValType = ScillaVal::Map;
} else {
v.mval[indices[0]][indices[1]]...[indices[n]].ValType = bytes;
}
v.mval[indices[0]][indices[1]]...[indices[n]].mval = it->val();
}
return true;
}
// Create a series of nested maps in @v for @indices
ScillaVal &createdNestedMap(std::vector<BVec> indices, ScillaVal &v)
{
ScillaVal &uv = v;
for (unsigned i = 0; i < indices.size(); i++) {
if (uv.mval.Contains(indices[i]) {
// index already exists, just go forward.
uv = uv.mval.Get(indices[i]);
} else {
// index doesn't exist, create a new submap.
ScillaVal::Map *newMap = new ScillaVal::Map;
uv.mval.insert(indices[i], newMap);
uv = newMap;
}
return uv;
}
// Delete all indices in db that have a prefix @prefix.
void deleteIndex(BVec &prefix)
{
for (; isPrefix(prefix, it->key()) && it->Valid(); it->Next()) {
db->Delete(it->key());
}
}
// Update state variable in level-db to value provided by Scilla
// Return value indicates error.
bool updateStateValue(const BVec &query, const BVec &value)
{
ScillaQuery &q = ProtoBuf.deserialize(query);
ScillaVal &v = ProtoBuf.deserialize(value);
// Form the base of the key into leveldb.
BVec keyBase = contr_addr + "." + q.name;
BVec keyReady(keyBase);
// Append indices to the base key.
for (unsigned idx = 0; idx < q.indices.size(); idx++) {
keyReady.insert(keyReady.end(), ".");
keyReady.insert(keyReady.end(), q.indices[idx].begin(), q.indices[idx].end());
}
assert (q.indices <= q.mapDepth);
if (q.deleteMapKey) {
assert(q.indices >= 1)
BVec indices(keyReady);
for (unsigned i = 0; i < q.indices; i++) {
// indices += q.indices[i]
indices.insert(indices.end(), q.indices[i].begin(), q.indices.end());
}
// delete an entire submap if necessary.
deleteIndex(indices);
}
if (q.indices == q.mapDepth) {
// v is not a map and can be just pushed into the store.
assert(v.ValType == bytes);
db->Put(keyReady, v.bval);
return true;
}
// v is a map variable and indices need to be iteratively pushed to store.
// This is a very expensive operation and hence should be charged very high.
// No entry is made in the store if "v" is an empty map.
// Before we push a map variable to the store, we've to delete whatever entries
// it had earlier. So we do that.
deleteIndex(keyReady);
std::function<void(BVec &, ScillaVal &)> mapHandler = [](BVec &keyAcc, ScillaVal &v)
{
assert(v.ValType == ScillaVal::Map);
if (v.mval.entries.size() == 0) {
// We have an empty map. Insert an entry for keyAcc in
// the store to indicate that the key itself exists.
BVec val = ProtoBuf.serialize(v.mval);
db->Put(keyAcc, val);
return;
}
for (MapEntry entry : v.mval.entries) {
// Insert each key<->val pair to the backend database.
BVec index(keyAcc);
index.insert(index.end(), "." + entry.key);
if (entry.val == ScillaVal.Map) {
// We haven't reached the deepest nesting.
if (!mapHandler(index, entry.val.mval)) {
return false;
}
} else {
db->Put(index, entry.val.bval);
}
}
};
mapHandler(keyReady, v);
return true;
}
While every query to the BC from the interpreter contains the mapDepth field (to enable BC to correctly process the query), this information is not available to BC when processing queries from other sources such as the JavaScript API.
At the time of deployment, the interpreter will, in its output, provide information about the map depth of each field in the contract. This will be parsed by the blockchain.
The map depth information will be serialized into a protobuf table (field name : map depth) and stored by the blockchain in leveldb using the key contr_addr + "." + "_fields_map_depth". Since no field in Scilla may have its name beginning with an _, this key will not be a prefix of any other possible key into the database.
The blockchain can, whenever it needs, get the value in this key, deserialize the protobuf table and use the information. It is suggested that it assert that the map depth for a field as stored in this table match the value that is part of every query by the Scilla interpreter.
Note:
- While the information is made available to the blockchain at the end of a deployment invocation of the interpreter, during that run, the interpreter will make
updateStateValuequeries to the blockchain, to set the initial values of fields. During these queries, the blockchain will not yet have the map depth in it's storage. It has to rely on the information from the query itself. - For fetching external state values (see below), the interpreter does not know the full type of the value being fetched. Hence it cannot provide
mapDepth. In this case, the blockchain uses its stored value for the operation.
BC will provide the following APIs for fetching and updating state variables that can be called during contract execution. As mentioned earlier, these APIs will be registered in SRTL, by BC, as callbacks, enabling SRTL to call them.
typedef std::vector<unsigned char> BVec;
// @query is serialized ScillaQuery while @value is serialized ScillaVal.
// "@found: is set to false if @query not found in database.
// Return value indicates error.
bool fetchStateValue(const BVec &query, BVec &value, bool &found);
bool updateStateValue(const BVec &query, const BVec &value);While the basic operations of these functions are straight-forward, it is to be noted that the value being passed (in either direction) can be a Map (protobuf serialized) when indices.size() is lesser than the nesting depth of the map.
Note: The inter-process-communication b/w Scilla interpreter and BC implements the same (de)serialization, but with the above two APIs being provided in OCaml (because the interpreter is in OCaml) instead of C++.
These functions fail on a malformed query (for example if number of indices is higher than max depth of a field) or some internal error in fetching from the backend database. In particular, field not found is not an error, and only found is set to false.
Scilla contracts allow read only accesses to fields of other contracts. To support this, the blockchain must additionally provide the below service:
bool fetchExternalStateValue(const std::string &addr, const BVec &query, BVec &value, bool &found, BVec &type);The semantics of this function is expected to be identical to that of fetchStateValue, except that it takes an additional input addr conveying the address (as a hexademical string) the contract whose field needs to be fetched. There is also an additional output type which, upon success must be filled by the blockchain to contain the type string of the field. The definition of success and failure is similar to that of fetchStateValue. In particular address or a field not found is not an error, but found is set to false.
To be able support this, the blockchain must store, similar to how it stores "map depth", the type string of fields, and also the version of every deployed contract (in order to avoid fetching the init JSON of the contract and looking that up for version).
A special note on ignoreval field of the query. This may be set for non-map queries too, and in that case, the server need not return any value, but just return the type field.
Considering that for every contract we now need to store not only the map-depth of fields, but also their types, and possibly other metadata (such as annotations for sharding), we accompany each field with a _metadata entry. i.e., instead of a contr_addr + "." + "_fields_map_depth" table as described earlier, we can have a more generic contr_addr + "." + "_metadata" table, which in addition to fields_map_depth will contain other entries such as type, sharding_annotation and contract_version.
For arbitrary queries about the blockchain state, the blockchain will provide the below service, which returns false on failure:
bool fetchBlockchainInfo(const std::string &query_name, const std::string &query_args, std::string &query_result);At present, the following queries are to be supported, with the interface allowing for more to be added later without conflicts.
| query_name | query_args | result |
|---|---|---|
| BLOCKNUMBER | "" (empty string) | Decimal string of current block number |
| CHAINID | "" (empty string) | Decimal string of current chain ID |
| TIMESTAMP | Decimal string of BLOCKNUMBER | Decimal string of timestamp, treated as Uint64 internally |
| REPLICATE_CONTRACT | A JSON (text) array containing the address (JSON string) and the init JSON (in the same format as user specifies for a new contract) to be used | Address of the newly deployed contract |
To enable SRTL to call functions in BC without having build dependence on BC code, it is required that BC register with the following mandatory callback registration APIs. These callback registrations must be done prior to deploying or execution a contract using SVM.
typedef std::function<bool(const BVec &query, BVec &value, bool &found)> FetchStateValueType;
typedef std::function<bool(const BVec &query, const BVec &value)> UpdateStateValueType;
void registerFetchStateValue(FetchStateValueType f);
void registerUpdateStateValue(UpdateStateValueType f);