Persistence strategy that writes items (e.g. More...

#include <RobotAPI/libraries/armem/server/ltm/persistence/disk/DiskPersistence.h>

Inheritance diagram for DiskPersistence:

Public Member Functions
bool	containsContainer (const armarx::armem::MemoryID &id, std::string key) override
	Checks if the container is available for the current memory id.

bool	containsItem (const armarx::armem::MemoryID &id, std::string key) override
	Checks if current container contains the item defined by its key.

void	createPropertyDefinitions (PropertyDefinitionsPtr &defs, const std::string &prefix) override

	DiskPersistence ()

	DiskPersistence (const std::filesystem::path &memoryParentPath)

	DiskPersistence (const std::string &identifier, const std::string &exportName, const std::filesystem::path &memoryParentPath)

void	flushBatch ()
	Explicitly flush all pending batch writes to disk.

size_t	getBatchPendingCount () const
	Get the current number of items pending in the batch buffer.

const BatchWriteStatistics &	getBatchStatistics () const
	Get batch write statistics.

int	getCompressionLevel () const

std::size_t	getCompressionMinBytes () const

std::vector< std::string >	getContainerKeys (const armarx::armem::MemoryID &id) override
	Returns all containers for the current id.

std::vector< std::string >	getItemKeys (const armarx::armem::MemoryID &id) override
	Returns all items for the current id.

size_t	getStorageErrorCount () const

bool	isBatchWriteEnabled () const
	Check if batch writing is enabled.

bool	isCompressionEnabled () const

void	resetBatchStatistics ()
	Reset batch write statistics.

void	resetStorageErrorCount ()

std::vector< unsigned char >	retrieveItem (const armarx::armem::MemoryID &id, std::string key) override
	Reads the data of the file with name 'key' at the current location.

void	setBatchSizeThreshold (size_t size)
	Set the maximum number of items to accumulate before auto-flushing.

void	setBatchTimeThresholdMs (size_t ms)
	Set the maximum time to hold items before auto-flushing.

void	setBatchWriteEnabled (bool enable)
	Enable or disable batch writing.

void	setCompressionEnabled (bool enable)
	Enable transparent gzip compression of exported JSON files.

void	setCompressionLevel (int level)
	zlib level, 1 (fastest) to 9 (smallest). Default 1 - see gzipCompress().

void	setCompressionMinBytes (std::size_t bytes)
	Only compress payloads of at least this many bytes.

void	setMinAvailableDiskSpace (const int minDiskSpace)

void	storeItem (const armarx::armem::MemoryID &id, std::string key, std::vector< unsigned char > &data) override
	Create a new file with name 'key' and stores the data inside it.

void	validateStoragePermissions ()

virtual	~DiskPersistence ()
	Destructor - flushes pending batch and stops batch thread.

Public Member Functions inherited from MemoryPersistenceStrategy
virtual void	disable ()

virtual void	enable ()

std::string	getExportName ()

std::string	getIdentifier ()

virtual bool	isEnabled ()

	MemoryPersistenceStrategy ()

	MemoryPersistenceStrategy (const std::string &identifier, const std::string &exportName, bool enabled=true)

std::vector< unsigned char >	retrieveItem (const armarx::armem::MemoryID &memoryId, ItemIdentifier itemIdentifier)

virtual ItemResult	retrieveItemWithOrigin (const armarx::armem::MemoryID &id, std::string &key)
	Retrieves the actual data but also appends the origin which is in this case the strategy identifier.

virtual void	setExportName (const std::string &exportName)

void	setIdentifier (const std::string &identifier)

void	storeItem (const armarx::armem::MemoryID &memoryId, ItemIdentifier itemIdentifer, std::vector< unsigned char > &data)

virtual	~MemoryPersistenceStrategy ()=default

Public Attributes
int	minDiskSpace = 50

Static Public Attributes
static constexpr const char *	GZIP_SUFFIX = ".gz"
	Suffix appended to the logical key when an item is stored compressed.

Static Public Attributes inherited from MemoryPersistenceStrategy
static const constexpr char *	DATA_FILENAME = "data.aron"

static const int	DEPTH_TO_DATA_FILES = 7

static const constexpr char *	MEMORY_EXPORT_SUFFIX = "_"

static const constexpr char *	METADATA_FILENAME = "metadata.aron"

static const constexpr char *	TYPE_FILENAME = "type.aron"

Additional Inherited Members
Protected Attributes inherited from MemoryPersistenceStrategy
bool	enabled_ = true
	If false, the strategy is not writing or reading anything.

std::string	exportName_
	Name of the specific memory export where our items should be stored.

std::string	identifier_
	Name of the strategy.

Detailed Description

Persistence strategy that writes items (e.g.

json files) to a specific container (a directory) Use it to write the data of a WM to disk.

Where are the items written (=:location)? /memoryParentPath/exportName/path(id)/

How is the file of an item named? -> Just they key (e.g. filename = key = "data.aron.json") If you might want a more sophisticated solution fell free to implement your custom ItemIdentifier today!

Definition at line 106 of file DiskPersistence.h.

Constructor & Destructor Documentation

◆ DiskPersistence() [1/3]

DiskPersistence ( )

inline

Definition at line 109 of file DiskPersistence.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ DiskPersistence() [2/3]

DiskPersistence ( const std::filesystem::path & memoryParentPath )

inline

Definition at line 113 of file DiskPersistence.h.

Here is the call graph for this function:

◆ DiskPersistence() [3/3]

DiskPersistence	(	const std::string &	identifier,
		const std::string &	exportName,
		const std::filesystem::path &	memoryParentPath )

inline

Parameters

identifier basically a unique name for the strategy (important if you use different strategies

See also: RedundantPersistenceStrategy)

Parameters

exportName	identifier for the exported memory. A new directory with name is created beneath the memoryParentPath. Everything is stored inside it.
memoryParentPath	path where the memory should be exported to

Definition at line 123 of file DiskPersistence.h.

Here is the call graph for this function:

◆ ~DiskPersistence()

virtual ~DiskPersistence ( )

inlinevirtual

Destructor - flushes pending batch and stops batch thread.

Definition at line 133 of file DiskPersistence.h.

Member Function Documentation

◆ containsContainer()

bool containsContainer	(	const armarx::armem::MemoryID &	id,
		std::string	key )

overridevirtual

Checks if the container is available for the current memory id.

Returns: true if the current location contains the directory with name 'key'

Implements MemoryPersistenceStrategy.

Definition at line 201 of file DiskPersistence.cpp.

Here is the call graph for this function:

◆ containsItem()

bool containsItem	(	const armarx::armem::MemoryID &	id,
		std::string	key )

overridevirtual

Checks if current container contains the item defined by its key.

Returns: true if the current location contains a file with the name 'key'

Implements MemoryPersistenceStrategy.

Definition at line 217 of file DiskPersistence.cpp.

◆ createPropertyDefinitions()

void createPropertyDefinitions	(	PropertyDefinitionsPtr &	defs,
		const std::string &	prefix )

inlineoverridevirtual

Implements MemoryPersistenceStrategy.

Definition at line 177 of file DiskPersistence.h.

◆ flushBatch()

void flushBatch ( )

Explicitly flush all pending batch writes to disk.

This is automatically called when batch thresholds are reached.

Definition at line 669 of file DiskPersistence.cpp.

Here is the caller graph for this function:

◆ getBatchPendingCount()

size_t getBatchPendingCount ( ) const

Get the current number of items pending in the batch buffer.

Definition at line 675 of file DiskPersistence.cpp.

◆ getBatchStatistics()

const BatchWriteStatistics & getBatchStatistics ( ) const

inline

Get batch write statistics.

Definition at line 276 of file DiskPersistence.h.

◆ getCompressionLevel()

int getCompressionLevel ( ) const

inline

Definition at line 219 of file DiskPersistence.h.

◆ getCompressionMinBytes()

std::size_t getCompressionMinBytes ( ) const

inline

Definition at line 231 of file DiskPersistence.h.

◆ getContainerKeys()

std::vector< std::string > getContainerKeys ( const armarx::armem::MemoryID & id )

overridevirtual

Returns all containers for the current id.

Returns: containers <=> directories at current location (=/memoryParentPath/exportName/path(id)/)

Implements MemoryPersistenceStrategy.

Definition at line 71 of file DiskPersistence.cpp.

Here is the call graph for this function:

◆ getItemKeys()

std::vector< std::string > getItemKeys ( const armarx::armem::MemoryID & id )

overridevirtual

Returns all items for the current id.

Returns: items <=> files at the current location (=/memoryParentPath/exportName/path(id)/)

Implements MemoryPersistenceStrategy.

Definition at line 154 of file DiskPersistence.cpp.

◆ getStorageErrorCount()

size_t getStorageErrorCount ( ) const

inline

Definition at line 285 of file DiskPersistence.h.

◆ isBatchWriteEnabled()

bool isBatchWriteEnabled ( ) const

inline

Check if batch writing is enabled.

Definition at line 248 of file DiskPersistence.h.

◆ isCompressionEnabled()

bool isCompressionEnabled ( ) const

inline

Definition at line 214 of file DiskPersistence.h.

◆ resetBatchStatistics()

void resetBatchStatistics ( )

inline

Reset batch write statistics.

Definition at line 281 of file DiskPersistence.h.

◆ resetStorageErrorCount()

void resetStorageErrorCount ( )

inline

Definition at line 290 of file DiskPersistence.h.

◆ retrieveItem()

std::vector< unsigned char > retrieveItem	(	const armarx::armem::MemoryID &	id,
		std::string	key )

overridevirtual

Reads the data of the file with name 'key' at the current location.

Returns: data if a file was found, an empty vector if the file is empty or was not found

Implements MemoryPersistenceStrategy.

Definition at line 354 of file DiskPersistence.cpp.

Here is the call graph for this function:

◆ setBatchSizeThreshold()

void setBatchSizeThreshold ( size_t size )

inline

Set the maximum number of items to accumulate before auto-flushing.

Parameters

size	Maximum batch size (default: 100)

Definition at line 254 of file DiskPersistence.h.

◆ setBatchTimeThresholdMs()

void setBatchTimeThresholdMs ( size_t ms )

inline

Set the maximum time to hold items before auto-flushing.

Parameters

ms	Maximum time in milliseconds (default: 100ms)

Definition at line 260 of file DiskPersistence.h.

◆ setBatchWriteEnabled()

void setBatchWriteEnabled ( bool enable )

Enable or disable batch writing.

When enabled, writes are accumulated and flushed in batches for better I/O performance.

Parameters

enable True to enable batching, false for immediate writes (default behavior)

Definition at line 611 of file DiskPersistence.cpp.

Here is the call graph for this function:

◆ setCompressionEnabled()

void setCompressionEnabled ( bool enable )

inline

Enable transparent gzip compression of exported JSON files.

When enabled, items whose key ends in ".json" and whose payload is at least getCompressionMinBytes() large are gzipped and stored under "<key>.gz". Reading is unaffected: containsItem()/retrieveItem() accept the plain (logical) key and transparently fall back to the ".gz" variant, and getItemKeys() reports logical keys. Exports written before this feature existed therefore keep working, and an export may freely mix both.

The compressed file deliberately gets a ".gz" suffix rather than being compressed in place under the original name: a reader that does not know about compression then simply does not find the file, instead of choking on unparsable content.

Definition at line 212 of file DiskPersistence.h.

Here is the call graph for this function:

◆ setCompressionLevel()

void setCompressionLevel ( int level )

inline

zlib level, 1 (fastest) to 9 (smallest). Default 1 - see gzipCompress().

Definition at line 217 of file DiskPersistence.h.

◆ setCompressionMinBytes()

void setCompressionMinBytes ( std::size_t bytes )

inline

Only compress payloads of at least this many bytes.

Defaults to one filesystem block (4096). Compressing anything smaller cannot free a single block, so it would cost compatibility (older readers no longer find the file) for no disk saving at all. In practice this leaves metadata.aron.json and type.aron.json as plain JSON and only compresses the instance data, which is where the size actually is.

Definition at line 229 of file DiskPersistence.h.

◆ setMinAvailableDiskSpace()

void setMinAvailableDiskSpace ( const int minDiskSpace )

inline

Definition at line 183 of file DiskPersistence.h.

◆ storeItem()

void storeItem	(	const armarx::armem::MemoryID &	id,
		std::string	key,
		std::vector< unsigned char > &	data )

overridevirtual

Create a new file with name 'key' and stores the data inside it.

Implements MemoryPersistenceStrategy.

Definition at line 230 of file DiskPersistence.cpp.

Here is the call graph for this function:

◆ validateStoragePermissions()

void validateStoragePermissions ( )

Definition at line 588 of file DiskPersistence.cpp.

Here is the call graph for this function:

Member Data Documentation

◆ GZIP_SUFFIX

const char* GZIP_SUFFIX = ".gz"

staticconstexpr

Suffix appended to the logical key when an item is stored compressed.

Definition at line 234 of file DiskPersistence.h.

◆ minDiskSpace

int minDiskSpace = 50

Definition at line 195 of file DiskPersistence.h.

The documentation for this class was generated from the following files:

RobotAPI/libraries/armem/server/ltm/persistence/disk/DiskPersistence.h
RobotAPI/libraries/armem/server/ltm/persistence/disk/DiskPersistence.cpp

Public Member Functions

Public Attributes

Static Public Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ DiskPersistence() [1/3]

◆ DiskPersistence() [2/3]

◆ DiskPersistence() [3/3]

◆ ~DiskPersistence()

Member Function Documentation

◆ containsContainer()

◆ containsItem()

◆ createPropertyDefinitions()

◆ flushBatch()

◆ getBatchPendingCount()

◆ getBatchStatistics()

◆ getCompressionLevel()

◆ getCompressionMinBytes()

◆ getContainerKeys()

◆ getItemKeys()

◆ getStorageErrorCount()

◆ isBatchWriteEnabled()

◆ isCompressionEnabled()

◆ resetBatchStatistics()

◆ resetStorageErrorCount()

◆ retrieveItem()

◆ setBatchSizeThreshold()

◆ setBatchTimeThresholdMs()

◆ setBatchWriteEnabled()

◆ setCompressionEnabled()

◆ setCompressionLevel()

◆ setCompressionMinBytes()

◆ setMinAvailableDiskSpace()

◆ storeItem()

◆ validateStoragePermissions()

Member Data Documentation

◆ GZIP_SUFFIX

◆ minDiskSpace