Specification

The open-source protocol for creating interactive, data-driven blocks

This document is a working draft

This specification is currently in progress. We’ve drafting it in public to gather feedback and improve the final document. If you have any suggestions or improvements you would like to add, feel free to submit a PR on our Github repo.

Block types

A block type should be packaged and distributed in a form which embedding applications can easily insert into a web page, and accompanied by metadata files describing the structure of data that the block accepts. A block package might be made available via a URL, package manager, or catalog of block types.

A block package MUST contain:

  • source file or files (e.g. HTML and JavaScript files), which

    • SHOULD contain valid HTML
    • There are various tools to help in meeting these requirements.

A block package SHOULD contain:

  • a JSON file describing the properties the block accepts using JSON Schema vocabulary – the ‘block schema’, which can be automatically generated from block code, and which:

    • MUST be called block-schema.json
    • MUST specify a required array naming any properties the block will not render or function without
    • SHOULD specify any further constraints or validation requirements which improve the block’s functionality
  • a JSON file containing metadata describing the block, which:

    • MUST be called block-metadata.json

    • MUST specify:

      • a name for the block
      • any libraries the block expects the embedding application to supply it with under externals – i.e. libraries which the block depends on but does not include in its package – using the name of the library as it is usually distributed (e.g. via npm), and the expected version (or version range). For example, a block may rely on React ("externals": [{ "react": "^16.0.0" }]), but assume that the embedding application will provide it, to save loading it multiple times on a page. This is also a useful way of indicating how the block expects to be rendered.
      • the path or URL to the entrypoint source file (e.g. index.html, index.js) under source
      • the version of the block, which SHOULD use semantic versioning
      • protocol: the applicable block protocol version
    • SHOULD specify:

      • default – an object, conforming to the block’s schema, representing the default data that applications should provide when creating a block, unless (a) the block can handle being provided no data when first instantiated or (b) variants is provided (see below).
      • icon – an icon for the block, to be displayed when the user is selecting from available blocks (as well as elsewhere as appropriate, e.g. in a website listing the block)
      • author
      • description
      • license
    • MAY specify:

      • variants – an array of objects, each with a name, description, icon, and properties, which represents different variants of the block that the user can create. As a simple example, a ‘header’ block might have variants called ‘Heading 1’ and ‘Heading 2’, which start with { level: 1 } and { level: 2 } as properties, respectively.
      • image – a preview image of the block for users to see it in action before using it. This would ideally have a 3:2 width:height ratio and be a minimum of 900x1170px.

Data transfer

Data provided to blocks

Blocks MUST use the data properties specified in their schema, if any. They can expect these properties to be made available to them by the embedding application, exactly how depending on rendering context. For example, for a React component block they would be passed as ‘props’. In a block with an HTML entrypoint, appropriately sandboxed by the embedding application, the properties would be in the global scope (of the sandbox).

The data for the block itself will be at the root level of the data made available to it, i.e. a block which had a level property at the root of the properties in its schema would have a level property passed in as a prop (or attached to the global scope of sandboxed HTML blocks).

The block’s data represents an ‘entity’ in the system. Alongside the properties declared in its schema, blocks should expect an entityId and entityTypeId to be provided by the embedding application, representing the application’s identifiers for the particular instance and type of the block. These must be strings. The exact form of these will differ across applications – e.g. some might use human-readable strings for entityTypeId, others might use integers or uuids. In any case, the block need only pass these back when requesting updates to entity data.

The entity for the block might link to other entities. For example, a Kanban board block might have a team property which links to another type of entity, a Team. Blocks should expect the following additional fields related to linked data to be passed to them:

  • a linkedEntities field containing the entities linked from the block’s entity, and entities linked to from those entities, and so on*

  • a linkedAggregations field containing the results of aggregations linked from the block’s entity

  • a linkGroups field which provides the links attached to the block or entities provided in either linkedEntities or linkedAggregations, grouped by entity and path

  • where available, an entityTypes field containing entity type definitions for any entities sent to the blocks, which can be parsed to understand the constraints on user input for each entity (e.g. which fields are editable, what sort of data they accept). Each entry in the array is a JSON schema object with an additional entityTypeId field corresponding to the entityTypeId of the entities it describes the shape of.

* Embedding applications will want to impose limits on the extent to which links between their records are followed for any given query, and not all indirectly linked entities may be provided with the block’s initial data. They can be requested using the functions listed below.

Each entity provided under linkedEntities or linkedAggregations will also have an entityId and entityTypeId to be used as arguments when updating those entities. See linking entities for a discussion of how links are managed.

Entity functions

Subject to the permissions granted to them by the embedding application, blocks can expect functions with the names and signatures listed below to be made available to them, i.e. to be passed in along with the properties defined in their schema, or to be otherwise made available in their scope depending on their implementation:

createEntities<T>(actions: CreateEntitiesAction<T>[]): Promise<T[]>

creates one or more entities

returns: the created entities, i.e. objects inside an array

accepts: a single array of objects (CreateEntitiesAction), each with the following shape:

  • entityTypeId [string]: the type of entity to create.
  • data<T> [object]: the field(s) and value(s) with which to create the entity, i.e. its properties.
  • links?: [object][optional]: any links to create along with this entity. See linking entities.
  • selection? [array of strings][optional]: limit the return to only include these fields on the entity.
  • depth? [integer][optional]: limit the depth to which linked data in an entity will be resolved. See linking entities.
updateEntities<T>(actions: UpdateEntitiesAction<T>[]): Promise<T[]>

updates one or more entities

returns: the updated entities, i.e. objects inside an array

accepts: a single array of objects (UpdateEntitiesAction), each with the following shape:

  • data<T> [object]: the fields and values to update on the entity.
  • entityTypeId [string]: the type of entity to update.
  • entityId [string]: the id of the entity to update.
  • selection? [array of strings][optional]: limit the return to only include these fields on the entity.
  • depth? [integer][optional]: limit the depth to which linked data in an entity will be resolved. See linking entities.
deleteEntities(actions: DeleteEntitiesAction[]): Promise<boolean[]>

deletes one or more entities

returns: an array of boolean indicating the success of each operation.

accepts: a single array of objects (DeleteEntitiesAction), each with the following shape:

  • entityTypeId [string]: the type of entity to delete
  • entityId [string]: the id of the entity to delete.
  • depth? [integer][optional]: limit the depth to which linked data in an entity will be resolved. See linking entities.
getEntities<T>(actions: GetEntitiesAction<T>[]): Promise<T[]>

retrieve one or more entities

returns: the retrieved entities, i.e. objects inside an array.

accepts: a single array of objects (GetEntitiesAction<T>), each with the following shape:

  • entityTypeId [string]: the type of entity to retrieve.
  • entityId [string]: the id of the entity to retrieve.
  • selection<T>? [array of strings][optional]: limit the return to only include these fields on the entity.
  • depth? [integer][optional]: limit the depth to which linked data in an entity will be resolved. See linking entities.
aggregateEntities(payload?: AggregateEntitiesPayload): Promise<AggregateEntitiesResult>

retrieve a subset of entities of a given type

returns: an AggregateEntitiesResult object

  • pageNumber [integer]: the page number returned.
  • itemsPerPage [integer]: the number of results per page, i.e. returned.
  • totalCount [integer]: the total number of records available for this query.
  • nextPage [integer]: the number of the next page, if any (null otherwise).
  • filters [array]: any filters applied (empty if none):
    • field [string]: the field name filtered by.
    • operator [enum]: the filter operator. One of IS, IS_NOT, CONTAINS, DOES_NOT_CONTAIN, STARTS_WITH, ENDS_WITH, IS_EMPTY, IS_NOT_EMPTY.
    • value [string]: the value filtered against.
  • sorts [array]: the sort(s) applied:
    • field [string]: the field sorted on.
    • desc [boolean]: whether the sort was descending.

accepts: an optional object (AggregateEntitiesPayload) with the following shape:

  • entityTypeId [object]: the type of entity to provide aggregated data for
  • selection? [array of strings][optional]: limit the return to only include these fields on the entity.
  • depth? [integer][optional]: limit the depth to which linked data in an entity will be resolved. See linking entities.
  • operation? [object][optional]: a description of the aggregation operation, which contains at least one of the following fields:
  • pageNumber? [integer][optional]: the page number to request.
  • itemsPerPage? [integer][optional]: the number of results to return.
  • filters? [array][optional]: filter entities by a given field value:
    • field [string]: the field name to filter by.
    • operator [enum]: the filter operator. One of IS, IS_NOT, CONTAINS, DOES_NOT_CONTAIN, STARTS_WITH, ENDS_WITH, IS_EMPTY, IS_NOT_EMPTY.
    • value [string]: the value to match against.
      • sorts? [array][optional]: specify how to sort results by providing one or more objects with the following shape:
      • field [string]: the field name to sort on.
      • desc? [boolean][optional]: whether to sort descending.
  • Embedding apps will provide a default aggregation if not provided.

The functions defined above return entity data, but block authors should note that in many implementations the embedding application will re-render a block with new entity data whenever it is updated (whether by the block or some other actor), e.g. the block will automatically get sent new data via props when any entity it has previously received via props is updated.

Entity type functions

Where supported and permitted by the embedding application, blocks may be provided with the following functions to work with entity types, i.e. data models.

createEntityTypes(actions: CreateEntityTypesAction[]): Promise<EntityType[]>

creates one or more entity types.

returns: the created entity types, i.e. objects inside an array. An EntityType is a JSON schema object with an additional entityTypeId field.

accepts: a single array of objects (CreateEntityTypesAction<T>), each with the following shape:

updateEntityTypes(actions: UpdateEntityTypesAction[]): Promise<EntityType[]>

updates one or more entity types.

returns: the updated entity types

accepts: a single array of objects (UpdateEntityTypesAction<T>), each with the following shape:

  • entityTypeId [string]: the id of the entity type to update.
  • schema [object]: the JSON schema for the entity type.
deleteEntityTypes(actions: DeleteEntityTypesAction[]): Promise<boolean[]>

deletes one or more entity types.

returns: an array of boolean indicating the success of each operation.

accepts: a single array of objects (DeleteEntityTypesAction<T>), each with the following shape:

  • entityTypeId [string]: the entity type to delete.
getEntityTypes(actions: GetEntityTypesAction[]): Promise<EntityType[]>

retrieves one or more entity types.

returns: the retrieved entity types, i.e. objects inside an array.

accepts: a single array of objects (GetEntityTypesAction<T>), each with the following shape:

  • entityTypeId [string]: the entity type to retrieve.
aggregateEntityTypes(payload: AggregateEntityTypesPayload): Promise<AggregateEntitiesResult>

retrieve one or more entity types.

returns: an AggregateEntitiesResult

accepts: an optional object (AggregateEntityTypesPayload) with a single key, operation. It has the same shape as for aggregating entities.

Linking entities

Another special set of functions provided to blocks relate to managing links between entities.

When creating or updating an entity’s data, including its own, blocks will often wish to express that a certain property on an entity should be a reference to another entity.

For example, that a Person’s employer field should point to a particular Company entity.

A block may also wish to link one of its properties to an aggregation of entities.

For example, a table block displaying the Top 10 people sorted by some property of Person, will need a way of encoding this aggregation in its data.

In order to create a reference to a separate entity or entities as the desired value of a particular field, blocks should create a Link , which:

  • MUST contain sourceEntityId [string]: the entityId of the source entity.
  • MAY contain sourceEntityVersionId [string]: optionally specify that this link is only from a specific version.
  • MAY contain sourceEntityTypeId: [string]: the entityTypeId of the source entity.
  • MUST contain path [string]: the path to the field on the source entity this link is conceptually made on, expressed as a JSON path.
  • MUST contain EITHER:
    • destinationEntityId [string] – the id of a single entity the link is made to, OR
    • aggregate – an aggregation operation which the embedding application should resolve the link to, following the structure of the operation object described above.
  • if destinationEntityId is defined:
    • MAY contain destinationEntityVersionId to pin the link to a specific version of the destination entity.
    • MAY contain destinationEntityTypeId: [string]: the entityTypeId of the destination entity.
  • MAY contain index [integer]: the position of this link in a list (for where ordering of links is important).

Once created, a Link includes a linkId.

E.g.1. creating a Link with the following data indicates that this particular user should be linked to a company with id company1, and that the link conceptually is made on the user’s employer field – although when delivering data to blocks the resolved data for entity company1 will be provided alongside the user to the block, in the linkedEntities array, rather than injected into the properties of the user, and the link itself available in the links array provided to the block.

{
  "sourceEntityId": "user1",
  "destinationEntityId": "company1",
  "path": "employer"
}

E.g.2. creating a Link with the following data indicates that this particular table should be linked to the top 10 sales by value, and that the link is conceptually made on the table’s rows field – although when delivering the data this data would be provided alongside the table, in the linkedAggregations array

{
  "sourceEntityId": "table1",
  "path": "rows",
  "aggregate": {
    "entityTypeId": "sales",
    "sorts": [{ "field": "value", "desc": true }],
    "itemsPerPage": 10,
    "pageNumber": 1
  }
}

Links will be provided to a block under a linkGroups field, which is an array of objects, each of which specifies a source entity, a path (field name), and the links on that path.

{
  "sourceEntityId": "user1",
  "path": "company",
  "links": [
    {
      "sourceEntityId": "user1",
      "destinationEntityId": "company1",
      "path": "company"
    }
  ]
}

N.B. this data structure has been chosen to allow for later pagination of links on a field.

The entities linked to will be provided under linkedEntities and linkedAggregations (and the entities they link onwards to, depending on the depth the graph is resolved to).

An entry in linkedAggregations follows the shape of the AggregateEntitiesResult object, with the addition of a link field containing the link which generated it (to allow identifying which entity and path the result ‘belongs’ to, where it might otherwise be ambiguous).

To create, update and delete links between entities, blocks should expect the following functions to be made available to them:

createLinks(actions: CreateLinksAction[]): Promise<Link[]>

creates one or more links.

returns: the created links, i.e. objects inside an array (now with linkId)

accepts: a single array of objects (CreateLinksAction<T>). Each object:

  • MUST contain sourceEntityId [string]: the entityId of the source entity.
  • MAY contain sourceEntityVersionId [string]: optionally specify that this link is only from a specific version.
  • MAY contain sourceEntityTypeId: [string]: the entityTypeId of the source entity.
  • MUST contain path [string]: the path to the field on the source entity this link is conceptually made on, expressed as a JSON path.
  • MUST contain EITHER:
    • destinationEntityId [string] – the id of a single entity the link is made to, OR
    • aggregate – an aggregation operation which the embedding application should resolve the link to, following the structure of the operation object described above.
  • if destinationEntityId is defined:
    • MAY contain destinationEntityVersionId [string] to pin the link to a specific version of the destination entity.
    • MAY contain destinationEntityTypeId: [string]: the entityTypeId of the destination entity.
  • MAY contain index [integer]: the position of this link in a list (for where ordering of links is important).
updateLinks(actions: UpdateLinksAction[]): Promise<Link[]>

updates one or more links.

returns: the updated links, i.e. objects inside an array

accepts: a single array of objects (UpdateLinksAction), each with the following shape:

  • linkId [string]: the id of the link to update.
  • data [object]: the Link to overwrite the existing one with.
deleteLinks(actions: DeleteLinksAction[]): Promise<boolean[]>

deletes one or more links.

returns: an array of boolean indicating the success of each operation.

accepts: a single array of objects (DeleteLinksAction), each with the following shape:

  • linkId [string]: the entity type to delete.
getLinks(actions: GetLinksAction[]): Promise<Link[]>

retrieve one or more links.

returns: the retrieved links, i.e. objects inside an array.

accepts: a single array of objects (GetLinksAction), each with the following shape:

  • linkId [string]: the link to retrieve.
Describing links in JSON schema

Where blocks wish to express in their schema – or in the schema of any other entity type – that the value of a field should be a link to another entity, they can use the JSON schema $ref keyword when describing the accepted types for the field. The value of the $ref should be the value of $id in the JSON schema for the target type.

Where blocks wish to express that a property in a schema is the inverse of another property, they can use an inverseOf keyword with a $ref pointing to the relevant schema and property.

E.g. to express that a company’s employees field is the inverse of users’ employer field:

{
  "$id": "https://example.com/schemas/company",
  "type": "object",
  "properties": {
    "employees": {
      "type": "array",
      "items": {
        "type": { "$ref": "https://example.com/schemas/user" }
      },
      "inverseOf": {
        "$ref": "https://example.com/schemas/user#/properties/employer"
      }
    }
  }
}

Embedding applications can use these inverseOf declarations to resolve inverse links without blocks needing to create them in both directions.

Limiting linked data returned

When requesting entity data via a block protocol function, blocks MAY include a depth field which will specify how many levels of linked entity data to resolve, to avoid expensive queries that pull in unneeded data from an extensive entity graph. For example, a depth of 2 on a Person would resolve their linked Employer, and their Employer’s linked Location, but no further. A depth of 0 would resolve no links to other entities.

Field summary

A block can expect the following fields to be made available to it, whether passed in as props or via another method appropriate to their rendering strategy:

// data representing the block entity itself
entityId
entityTypeId
// ...plus any keys declared in the block’s schema

// data representing entities linked from the block and onwards
linkedEntities
linkedAggregations
links

// functions
createEntities
updateEntities
deleteEntities
getEntities
aggregateEntities
createEntityTypes
updateEntityTypes
deleteEntityTypes
getEntityTypes
aggregateEntityTypes
createLinks
updateLinks
deleteLinks
getLinks

Third-party data stores

Where blocks interact with third-party data stores, i.e. they send data for storage outside the embedding application, they SHOULD where possible keep the entity data in the embedding application in sync, for example by:

  • creating entities via the functions described above at the same time as creating records in other data stores

  • reflecting any changes to entities provided by the embedding application if the user takes action to edit them in the block UI, by updating entities via the functions described above

Where changes to relevant entity data can occur in both the embedding application and the third-party data store even when the block is not being used, additional synchronization outside the block will be required to ensure consistent user data.

Tracking user action

🤔

We are considering options for blocks reporting on user actions within them, both to allow the embedding application to track activity, and to be able to indicate user focus to other users where the application implements collaborative/multiplayer editing.

Potential options include:

  • passing a reportUserAction function to blocks, to report on keypresses, drags, etc.

  • passing a usersFocus property to blocks containing an array of focus objects, each indicating where different users are focused, to allow the block to render indicators.

  • handling tracking user focus and rendering focus indicators outside of blocks.

Tracking user actions will also be important for a page-level action history/stack, e.g. for undos.

Edit history

🤔

While embedding applications can handle displaying an interface for reloading blocks at particular earlier versions, we will specify a way of communicating to blocks that (a) an earlier version is being displayed, and (b) the difference with the current version would allow blocks to implement visual diffs and so on.

Comments

🤔

We want to facilitate users leaving comments on elements within blocks.

This could be

  • managed entirely outside the block, e.g. by a wrapper around the block which provides a context menu to users for adding comments on blocks – which avoids blocks having to have any knowledge of commenting, but could interfere with how the block wants to respond to user input,or

  • managed by providing functions to blocks to trigger a comment attached to specific elements in blocks – which allows blocks to have control over how and to what element the user is able to attach comments, but means that blocks have to implement ‘offer comment option’ behavior.

Styling

Blocks SHOULD provide at least basic visual styling to allow them to be embedded and used without modification by any web application.

Blocks SHOULD use the CSS variables listed in Appendix A as property values where appropriate, but SHOULD provide fallback values in case the embedding application does not define them.

Ideally, the embedding application will also provide a styleVariables object alongside block data, which includes keys with the same names as the CSS variables listed in Appendix A and appropriate values.

Add blocks to your app

Anyone with an existing application who wants to embed semantically-rich, reusable blocks in their product can use the protocol. Improve your app’s utility and tap into a world of structured data with no extra effort, for free.

Build your own blocks

Any developer can build and publish blocks to the global registry for other developers to use. Create blocks that solve real-world problems, and contribute to an open source community changing the landscape of interoperable data.