Persistence strategy that writes items (e.g. More...

#include <RobotAPI/libraries/armem/server/ltm/persistence/disk/DiskPersistence.h>

Inheritance diagram for DiskPersistence:

Public Member Functions
bool	containsContainer (const armarx::armem::MemoryID &id, std::string key) override
	Checks if the container is available for the current memory id.

bool	containsItem (const armarx::armem::MemoryID &id, std::string key) override
	Checks if current container contains the item defined by its key.

void	createPropertyDefinitions (PropertyDefinitionsPtr &defs, const std::string &prefix) override

	DiskPersistence ()

	DiskPersistence (const std::filesystem::path &memoryParentPath)

	DiskPersistence (const std::string &identifier, const std::string &exportName, const std::filesystem::path &memoryParentPath)

void	flushBatch ()
	Explicitly flush all pending batch writes to disk.

size_t	getBatchPendingCount () const
	Get the current number of items pending in the batch buffer.

const BatchWriteStatistics &	getBatchStatistics () const
	Get batch write statistics.

int	getCompressionLevel () const

std::size_t	getCompressionMinBytes () const

std::vector< std::string >	getContainerKeys (const armarx::armem::MemoryID &id) override
	Returns all containers for the current id.

std::vector< std::string >	getItemKeys (const armarx::armem::MemoryID &id) override
	Returns all items for the current id.

size_t	getStorageErrorCount () const

bool	isBatchWriteEnabled () const
	Check if batch writing is enabled.

bool	isCompressionEnabled () const

void	resetBatchStatistics ()
	Reset batch write statistics.

void	resetStorageErrorCount ()

std::vector< unsigned char >	retrieveItem (const armarx::armem::MemoryID &id, std::string key) override
	Reads the data of the file with name 'key' at the current location.

void	setBatchSizeThreshold (size_t size)
	Set the maximum number of items to accumulate before auto-flushing.

void	setBatchTimeThresholdMs (size_t ms)
	Set the maximum time to hold items before auto-flushing.

void	setBatchWriteEnabled (bool enable)
	Enable or disable batch writing.

void	setCompressionEnabled (bool enable)
	Enable transparent gzip compression of exported JSON files.

void	setCompressionLevel (int level)
	zlib level, 1 (fastest) to 9 (smallest). Default 1 - see gzipCompress().

void	setCompressionMinBytes (std::size_t bytes)
	Only compress payloads of at least this many bytes.

void	setMinAvailableDiskSpace (const int minDiskSpace)

void	storeItem (const armarx::armem::MemoryID &id, std::string key, std::vector< unsigned char > &data) override
	Create a new file with name 'key' and stores the data inside it.

void	validateStoragePermissions ()

virtual	~DiskPersistence ()
	Destructor - flushes pending batch and stops batch thread.

Public Member Functions inherited from MemoryPersistenceStrategy
virtual void	disable ()

virtual void	enable ()

std::string	getExportName ()

std::string	getIdentifier ()

virtual bool	isEnabled ()

	MemoryPersistenceStrategy ()

	MemoryPersistenceStrategy (const std::string &identifier, const std::string &exportName, bool enabled=true)

std::vector< unsigned char >	retrieveItem (const armarx::armem::MemoryID &memoryId, ItemIdentifier itemIdentifier)

virtual ItemResult	retrieveItemWithOrigin (const armarx::armem::MemoryID &id, std::string &key)
	Retrieves the actual data but also appends the origin which is in this case the strategy identifier.

virtual void	setExportName (const std::string &exportName)

void	setIdentifier (const std::string &identifier)

void	storeItem (const armarx::armem::MemoryID &memoryId, ItemIdentifier itemIdentifer, std::vector< unsigned char > &data)

virtual	~MemoryPersistenceStrategy ()=default

Public Attributes
int	minDiskSpace = 50

Static Public Attributes
static constexpr const char *	GZIP_SUFFIX = ".gz"
	Suffix appended to the logical key when an item is stored compressed.

Static Public Attributes inherited from MemoryPersistenceStrategy
static const constexpr char *	DATA_FILENAME = "data.aron"

static const int	DEPTH_TO_DATA_FILES = 7

static const constexpr char *	MEMORY_EXPORT_SUFFIX = "_"

static const constexpr char *	METADATA_FILENAME = "metadata.aron"

static const constexpr char *	TYPE_FILENAME = "type.aron"

Additional Inherited Members
Protected Attributes inherited from MemoryPersistenceStrategy
bool	enabled_ = true
	If false, the strategy is not writing or reading anything.

std::string	exportName_
	Name of the specific memory export where our items should be stored.

std::string	identifier_
	Name of the strategy.

Detailed Description

Persistence strategy that writes items (e.g.

json files) to a specific container (a directory) Use it to write the data of a WM to disk.

Where are the items written (=:location)? /memoryParentPath/exportName/path(id)/

How is the file of an item named? -> Just they key (e.g. filename = key = "data.aron.json") If you might want a more sophisticated solution fell free to implement your custom ItemIdentifier today!

Constructor & Destructor Documentation

◆ DiskPersistence() [1/3]

DiskPersistence ( )

inline

◆ DiskPersistence() [2/3]

DiskPersistence ( const std::filesystem::path & memoryParentPath )

inline

◆ DiskPersistence() [3/3]

DiskPersistence	(	const std::string &	identifier,
		const std::string &	exportName,
		const std::filesystem::path &	memoryParentPath )

inline

Parameters

identifier basically a unique name for the strategy (important if you use different strategies

See also: RedundantPersistenceStrategy)

Parameters

exportName	identifier for the exported memory. A new directory with name is created beneath the memoryParentPath. Everything is stored inside it.
memoryParentPath	path where the memory should be exported to

◆ ~DiskPersistence()

virtual ~DiskPersistence ( )

inlinevirtual

Destructor - flushes pending batch and stops batch thread.

Member Function Documentation

◆ containsContainer()

bool containsContainer	(	const armarx::armem::MemoryID &	id,
		std::string	key )

overridevirtual

Checks if the container is available for the current memory id.

Returns: true if the current location contains the directory with name 'key'

Implements MemoryPersistenceStrategy.

◆ containsItem()

bool containsItem	(	const armarx::armem::MemoryID &	id,
		std::string	key )

overridevirtual

Checks if current container contains the item defined by its key.

Returns: true if the current location contains a file with the name 'key'

Implements MemoryPersistenceStrategy.

◆ createPropertyDefinitions()

void createPropertyDefinitions	(	PropertyDefinitionsPtr &	defs,
		const std::string &	prefix )

inlineoverridevirtual

Implements MemoryPersistenceStrategy.

◆ flushBatch()

void flushBatch ( )

Explicitly flush all pending batch writes to disk.

This is automatically called when batch thresholds are reached.

◆ getBatchPendingCount()

size_t getBatchPendingCount ( ) const

Get the current number of items pending in the batch buffer.

◆ getBatchStatistics()

const BatchWriteStatistics & getBatchStatistics ( ) const

inline

Get batch write statistics.

◆ getCompressionLevel()

int getCompressionLevel ( ) const

inline

◆ getCompressionMinBytes()

std::size_t getCompressionMinBytes ( ) const

inline

◆ getContainerKeys()

std::vector< std::string > getContainerKeys ( const armarx::armem::MemoryID & id )

overridevirtual

Returns all containers for the current id.

Returns: containers <=> directories at current location (=/memoryParentPath/exportName/path(id)/)

Implements MemoryPersistenceStrategy.

◆ getItemKeys()

std::vector< std::string > getItemKeys ( const armarx::armem::MemoryID & id )

overridevirtual

Returns all items for the current id.

Returns: items <=> files at the current location (=/memoryParentPath/exportName/path(id)/)

Implements MemoryPersistenceStrategy.

◆ getStorageErrorCount()

size_t getStorageErrorCount ( ) const

inline

◆ isBatchWriteEnabled()

bool isBatchWriteEnabled ( ) const

inline

Check if batch writing is enabled.

◆ isCompressionEnabled()

bool isCompressionEnabled ( ) const

inline

◆ resetBatchStatistics()

void resetBatchStatistics ( )

inline

Reset batch write statistics.

◆ resetStorageErrorCount()

void resetStorageErrorCount ( )

inline

◆ retrieveItem()

std::vector< unsigned char > retrieveItem	(	const armarx::armem::MemoryID &	id,
		std::string	key )

overridevirtual

Reads the data of the file with name 'key' at the current location.

Returns: data if a file was found, an empty vector if the file is empty or was not found

Implements MemoryPersistenceStrategy.

◆ setBatchSizeThreshold()

void setBatchSizeThreshold ( size_t size )

inline

Set the maximum number of items to accumulate before auto-flushing.

Parameters

size	Maximum batch size (default: 100)

◆ setBatchTimeThresholdMs()

void setBatchTimeThresholdMs ( size_t ms )

inline

Set the maximum time to hold items before auto-flushing.

Parameters

ms	Maximum time in milliseconds (default: 100ms)

◆ setBatchWriteEnabled()

void setBatchWriteEnabled ( bool enable )

Enable or disable batch writing.

When enabled, writes are accumulated and flushed in batches for better I/O performance.

Parameters

enable True to enable batching, false for immediate writes (default behavior)

◆ setCompressionEnabled()

void setCompressionEnabled ( bool enable )

inline

Enable transparent gzip compression of exported JSON files.

When enabled, items whose key ends in ".json" and whose payload is at least getCompressionMinBytes() large are gzipped and stored under "<key>.gz". Reading is unaffected: containsItem()/retrieveItem() accept the plain (logical) key and transparently fall back to the ".gz" variant, and getItemKeys() reports logical keys. Exports written before this feature existed therefore keep working, and an export may freely mix both.

The compressed file deliberately gets a ".gz" suffix rather than being compressed in place under the original name: a reader that does not know about compression then simply does not find the file, instead of choking on unparsable content.

◆ setCompressionLevel()

void setCompressionLevel ( int level )

inline

zlib level, 1 (fastest) to 9 (smallest). Default 1 - see gzipCompress().

◆ setCompressionMinBytes()

void setCompressionMinBytes ( std::size_t bytes )

inline

Only compress payloads of at least this many bytes.

Defaults to one filesystem block (4096). Compressing anything smaller cannot free a single block, so it would cost compatibility (older readers no longer find the file) for no disk saving at all. In practice this leaves metadata.aron.json and type.aron.json as plain JSON and only compresses the instance data, which is where the size actually is.

◆ setMinAvailableDiskSpace()

void setMinAvailableDiskSpace ( const int minDiskSpace )

inline

◆ storeItem()

void storeItem	(	const armarx::armem::MemoryID &	id,
		std::string	key,
		std::vector< unsigned char > &	data )

overridevirtual

Create a new file with name 'key' and stores the data inside it.

Implements MemoryPersistenceStrategy.

◆ validateStoragePermissions()

void validateStoragePermissions ( )

Member Data Documentation

◆ GZIP_SUFFIX

const char* GZIP_SUFFIX = ".gz"

staticconstexpr

Suffix appended to the logical key when an item is stored compressed.

◆ minDiskSpace

int minDiskSpace = 50

The documentation for this class was generated from the following files:

RobotAPI/libraries/armem/server/ltm/persistence/disk/DiskPersistence.h
RobotAPI/libraries/armem/server/ltm/persistence/disk/DiskPersistence.cpp

Public Member Functions

Public Attributes

Static Public Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ DiskPersistence() [1/3]

◆ DiskPersistence() [2/3]

◆ DiskPersistence() [3/3]

◆ ~DiskPersistence()

Member Function Documentation

◆ containsContainer()

◆ containsItem()

◆ createPropertyDefinitions()

◆ flushBatch()

◆ getBatchPendingCount()

◆ getBatchStatistics()

◆ getCompressionLevel()

◆ getCompressionMinBytes()

◆ getContainerKeys()

◆ getItemKeys()

◆ getStorageErrorCount()

◆ isBatchWriteEnabled()

◆ isCompressionEnabled()

◆ resetBatchStatistics()

◆ resetStorageErrorCount()

◆ retrieveItem()

◆ setBatchSizeThreshold()

◆ setBatchTimeThresholdMs()

◆ setBatchWriteEnabled()

◆ setCompressionEnabled()

◆ setCompressionLevel()

◆ setCompressionMinBytes()

◆ setMinAvailableDiskSpace()

◆ storeItem()

◆ validateStoragePermissions()

Member Data Documentation

◆ GZIP_SUFFIX

◆ minDiskSpace