ArmarX Memory System

Table of Contents

See also

Introduction to the Memory System

Distributed Memory

The ArMem memory system is distributed: It consists of several memory servers storing data in a hierarchical structure. A memory server can be defined in any ArmarX package and defines which kind of data it stores (e.g. robot state, object classes and instances, visual data, speech, ...). As a memory server knows which data it stores, it can provide special behaviour, such as custom visualization, aggregation, interpolation or prediction. As each server is a separate component describing its own structure, it is simple to add new memory servers containing special data types, which are defined in ArmarX packages further down in the dependency chain.

All memory servers follow the same hierarchical data structure and self-describing data format. The data structure consists of several levels, storing histories/timelines of different entities stored in different segments (see "Memory Levels" below). The common data format is ArmarX Object Notation (ARON) / Interpretable Data Format (IDF), which is a self-describing, hierarchical data format, allowing extensive introspection as well as general storage. Each item in the memory (i.e. entries in all levels) can be identified with a Memory ID, which contains the keys of each (specified) level. For example, a core segment ID specifies the memory name and core segment name, while an entity instance ID specifies the whole path to the leaf of the data structure.

Technically, each memory server is one ArmarX component implementing the armarx::armem::server::MemoryInterface (comprising a armarx::armem::server::ReadingMemoryInterface and a armarx::armem::server::WritingMemoryInterface). Memory servers register themselves in the Memory Name System (MNS), where they can be found by their (semantic) memory name (which is usually a short, human-readable name such as "Robot", "Object" or "Vision"). Memory servers receive data via commits (via the armarx::armem::server::WritingMemoryInterface), and return data according to queries (via the armarx::armem::server::ReadingMemoryInterface).

Memory Levels

A memory server stores data in a hierarchical structure consisting of several levels:

- Grasping (Memory)
- Grasps (Core Segment)
- KnownObjectGraspPlanner (Provider Segment)
- my_object (Entity)
- t = 2021-03-09 14:24:20.064524 (Entity Snapshot)
- 00 (Entity Instance)
- 01 (Entity Instance)
- 02 (Entity Instance)
Level Key Description Examples Implementation
Memory Name (String) Semantic grouping of one or more core segments. Corresponds to one memory server. Robot, Object, Vision armarx::armem::wm::Memory
Core Segment Name (String) Building block of a memory containing homogeneous data of a specific (ARON) type. Specifies a Core Segment Type which all provided data must comply to. (Robot) Proprioception, (Object) Classes, Known (Object) Instances, ImageRGB armarx::armem::wm::CoreSegment
Provider Segment Name (String) Sub-segment of a core segment which contains the data from a single source (e.g. a camera, a method, a component). Can define a Provider Segment Type which extends the Core Segment Type. Primesense, MyObjectLocalizer, MyGraspPlanner armarx::armem::wm::ProviderSegment
Entity Name (String) A single thing or concept whose properties change over time. An entity has a history / timeline, i.e. a sequence of snapshots storing the entity's properties at a specific point in time. image, some_object, some_object_grasps armarx::armem::wm::Entity
Entity Snapshot Timestamp (armarx::armem::Time aka armarx::DateTime) An entry of an entity's history, i.e. the state of an entity at a specific point in time. Can contain multiple (entitiy) instances. Image, object instance, grasp hypotheses at a time t armarx::armem::wm::EntitySnapshot
Entity Instance Index (int) One instance of the segment's ARON type. left/right stereo image (at time t), object (at time t), grasp hypothesis (one of many at time t) armarx::armem::wm::EntityInstance
Entity Instance Metadata – Metadata stored alongside the instance's data. Further timesteps, confidence armarx::armem::wm::EntityInstanceMetadata

Existing Memory Servers and Segments

This is a list of existing memory servers, their core segments and their ARON data types.

RobotState

Core Segment Name Type Description Example Entities
Description RobotDescription Robot description (e.g. link to XML or URDF file) Description of ARMAR-6
Proprioception Proprioception Robot configuration and internal sensor values. Proprioception of ARMAR-6
Localization Transform Transformations between frames {world,map,robot} to retrieve global robot pose. Transfrom from robot root frame to map frame

Missing:

  • Robot calibration: should be part of Description core segment

Object

Core Segment Name Type Description Example Entities
Class ObjectClass Static description of a known object class. Entity names are (ArmarXObjects) object IDs. KIT/CoffeeFilters, YCB/001_chips_can
Instance ObjectInstance Localized object instance. Entity names are (ArmarXObjects) object IDs. KIT/CoffeeFilters, YCB/001_chips_can/instance_1
Attachments ObjectAttachment and ArticulatedObjectAttachment

Vision

Core Segment Name Type Description Example Entities
ImageRGB ImageRGB RGB images (mono and stereo) of a camera image
ImageDepth ImageDepth Depth images of an RGB-D or stereo camera image

Speech

Core Segment Name Type Description Example Entities
Command SpeechCommand Language commands from speech (topic Speech_Commands). command
TextListenerInterface Text Data from TextListenerInterface topics. (TextToSpeech, RobotAction) text

...

Skills

Core Segment Name Type Description Example Entities
Statechart Transition Transitions in ArmarX state charts. tbd

Example

An example memory server (alongside a matching client) showing how to implement a standard memory server and test the framework during development.

Core Segment Name Type Description Example Entities
ExampleData ExampleData Some example data. example

GeneralPurpose

This memory is meant to allow for a quick-and-dirty integration of your data type in the memory framework. When your development finishes a stable state, consider creating moving the core segment(s) to an existing memory server or creating a new dedicated memory server.

Core Segment Name Type Description Example Entities
None. You can add segments on the fly.