![]() |
GeMA
The GeMA main application
|
The GeMA state dumping functionality is intended to allow the user to save the current simultion state so that it can later be restarted from the same point on a different simulator run. This is usefull if a long simulation was interrupted by hardware or power problems, programming errors or even if the user desires to change some simulation parameters as the result of inspecting the simualtion progress. It can also be used for extending the simulation time after a succesfully completed simulation.
Saving and recovering states can be implemented in GeMA by usign a StateDump
object. This is a "low level" object intended to be used as a building block to more user friendly options, such as the Dump Library. It is also usefull for saving and recovering some intermediate states while orchestrating coupled simulations with multiple solvers.
A state dump object can be created by a call to StateDump()
and is responsible for storing the contents provided by "state items" to either disk or to another place in memory. The set of state items is fixed and must be configured by calls to dump:
addStateItem()
and/or dump:
addGroupItem()
before a call to dump:
init()
prepares the state dump object to be used. This initial configuration does not save any state data to disk (or memory). It only teaches the state dump object about the data that will be saved / restored.
Supported operations are to either save or load the full set of state items. When saving, the state items are queried to provide the data to be saved, and when loading, the recovered data is sent to the items. There is also support for saving and loading individual state items or groups of "active" items. When specified over a single item, the operation can specify that only a part of the state item data should be saved or loaded.
Keep in mind that when saving data to disk, a state object is tied to a single dump file whose contents will be overwritten at each time the state is saved. If you want to keep several state dumps, you should use several objects.
When loading data from a file saved by a previous execution of GeMA, the set of state items configured on the state dump object must be the same as the one configured when the dump file was saved. In the current implementation, that means that the number of items should be equal and each item should have exactly the same name, type and group. They should also be in the same relative order. As an exception, if the file will be used only for loading the data (restore mode), no updates, it is possible to operate with a partial set of dump items. In that setting, tailored for recovering mesh and boundary condition data before the orchestration, the implementation still expects the partial set to be equivalent and in the same order as part of the dump file configuration. Some basic sanity checks are done when loading a dump file to enforce those restrictions.
The file layout is quite simple and is intended for saving and recovering data on the same machine, with the same version of GeMA. Each state item is saved into a data block. Each data block can be split in several non-continuous data chunks. This allows for a state item to grow without the need of reorganizing the whole file.
A call to the dump:
save()
operation always replaces the complete set of file data, so it never creates blocks with multiple chunks and suffers no consequences if the state item sizes are changed (either growing or shrinking). Calls to dump:
saveItem()
, on the other hand will update the file with the given data saving it on top of the old one (unless at the first call). If the state item size has grown, a new data chunk is added if there is no space available on the last chunk of the state item block. If the size has shrinked, data chunks might be left unused on the file.
To prevent inefficiency due to adding multiple chunks to a state item, the state dump object creation options receives as parameters the A and B coefficients of a linear equation that is used to find the file block size when saving a state item for the first time or when growing is needed: fileBlockSize = A * stateItemSize + B. Another alternative is to use one state dump object per state item. In that case, being the only state item in the file always allows for growing a chunk.
Stored data can be optionally compressed with the zlib library. The desired compression level and the minimum buffer size for which compression is applied are options to the state dump object creation. Important: compression can not be used together with partial state item load / save operations.
Example:
What can be dumped:
GemaMesh
plugin, including support for changes of the mesh geometry along the simulation.dump:
addStateItem()
method for details on what data can be saved.Limitations:
dumpFile
option (see the GemaMesh plugin documentation), when dynamic attributes are saved to the mesh. The same restriction applies for boundary condition properties. Index:
dump:
addStateItem()
dump:
addGroupItem()
dump:
init()
dump:
itemGroup()
dump:
numItems()
dump:
findItem()
dump:
setGroupActive()
dump:
save()
dump:
load()
dump:
saveItem()
dump:
loadItem()
dump:
saveActiveGroups()
dump:
loadActiveGroups()
dump:
flush()
dump:
clear()
StateDump(options) | ||
---|---|---|
Description: | Creates a new state dump object that can de used to control state dumps from Lua. The given options table controls the state dump behaviour. | |
Parameters: | options | A table filled with the desired state dump options. Each field is optional. Missing fields will be ignored and default values applied. Available options: mode - Defines if the dump will be done to file or memory. Accepted values are "file" , "memory" , "restore" (for partial loading of a dump file, without saving possibility - see the dumpFile option on the GemaMesh plugin documentation) or "auto" (file or memory depending on memory availability). If missing, the default will be the value of the defDumpStateMode field from the simulation options. file - The dump file name when dumping to disk. Can include path macros. If missing, the default will be the value of the defDumpStateFile field from the simulation options. saveAndRestore - Meaningfull only for file dumps, this option should be set to true if this state dump will be used to save a simulation state that can be later loaded in another GeMA execution, false if that is not needed. Setting it to false makes the state simpler when the saved accessors support function values but limits the state dump usage for loading and restoring states within a single GeMA execution. If missing, the default will be the value of the defDumpStateSaveAndRestore field from the simulation options. compression - Defines if the state dump should be compressed (using zlib) or not. Valid values are integer numbers between 0 (no compression) and 9 (maximum compression) or -1 (default zlib compression). If missing, the default will be the value of the defDumpStateCompression field from the simulation options. compressionMinSize - When state dump compression is enabled, this option defines the minimum size (in bytes) that a saved data item must have to be compressed. Data items smaller than this value will not be compressed at all. Must be a value greater than zero. If missing, the default will be the value of the defDumpStateCompressionMinSize field from the simulation options. growFactorA - When saving state dumps to file, and dump items can grow in size over time, it might be more efficient to pre-store space for the item growing. This is done by applying a linear expansion to the data item size: filesize = A * datasize + B . This parameter gives the A factor, which should be >= 1.0. If missing, the default will be the value of the defDumpStateGrowFactorA field from the simulation options. growFactorB - When saving state dumps to file, and dump items can grow in size over time, it might be more efficient to pre-store space for the item growing. This is done by applying a linear expansion to the data item size: filesize = A * datasize + B . This parameter gives the B factor. If missing, the default will be the value of the defDumpStateGrowFactorB field from the simulation options. printStructure - When set to true, a detailed summary of the added dump items and loaded dump items (when restoring from an existing file) will be printed. If missing, the default is false . |
Returns: | Returns the created object. |
Example:
dump:addStateItem(accessor, obj, fixed, groupId) dump:addStateItem(matrix, fixed, groupId) dump:addStateItem(globals, locals, localTableName, groupId) dump:addStateItem(file, fileName, tempName, groupId) | ||
---|---|---|
Description: | Adds a new state dump item to the state dump object. Each function signature corresponds to a different kind of data added to the state. The first one can be used to add mesh data associated with nodes, cells or Gauss points identified by an accessor object, or boundary condition data, also identified by an accessor object. The second one stores the contents of a matrix object. The third signature can be used to save the contents of global and local Lua variables. Finaly, the fourth one is used for saving the state (current file position) associated withe a file created by io.open() . Important 1: notice that adding a new state item to the dump state does not saves any data to the state. This operation just informs the state dump about the items that will be included on it, defining its layout. The data will only be saved when a call to dump: save() , dump: saveActiveGroups() or dump: saveItem() is made, with the contents at the moment of those calls. Important 2: After all calls to addStateItem() and/or addGroupItem() , it is necessary to call dump: init() to complete the state dump initialization. | |
Parameters: | accessor | Either a value, cell, Gauss or boundary condition accessor object. This option tells the state that the mesh / boundary condition data reachable by this accessor should be added to the state dump. If the accessor references a specific data history entry, only that entry will be added to the state. If an accessor is added multiple times to the state, the multiple adds will not make any difference (and the returned item id will be equal to the first returned one), but keep in mind that if a mesh or boundary condition is added to the state by a call to dump: addGroupItem() , any accessor to that mesh or boundary condition data can not be added to the state before the mesh / boundary condition. |
obj | A mesh object pointing to the mesh that the given accessor belongs to for node, cell or Gauss accessors. A boundary condition object pointing to the boundary condition that the given accessor belongs to for boundary condition accessors. | |
fixed | An optional parameter that can be used to give a hint to the state dump that the accessor or matrix data added to the state has a fixed size, and so the state can optimize its storage for a no growing data. This hint does not prevent the data from growing. It ony optimizes the data storage for a non-growing case where no room for growing is pre-allocated. If missing, false will be assumed. | |
groupId | Each item is associated to a group, identified by a group id. This optional parameter allows a dump item to be explicitely tied to a given group (a value >= 0). A value of -1 means that the item will be tied to the next available group id. A value of -2 means that the item will be tied to the same group id as the one used by the last added item. If not given, the default of -1 will be assumed. | |
matrix | A matrix object. This option tells the state that the matrix data associated with this object should be added to the state dump. | |
globals | A table with the names of the global Lua variables that should be stored on the state dump. Variables that can be stored must have values that can be properly serialized. That excludes functions and user data objects. Allowed values are: numbers, strings, boolean, nil and tables with number and string keys, whose values are accepted. Tables can not have cycles. | |
locals | A table with the names of the local Lua variables that should be stored on the state dump. Variables that can be stored must have values that can be properly serialized. That excludes functions and user data objects. Allowed values are: numbers, strings, boolean, nil and tables with number and string keys, whose values are accepted. Tables can not have cycles. When storing local values, the given variables must be in scope when the dump: save() or dump: saveItem() operation is called. Upon loading, the values are stored in the global table indicated by localTableName , indexed by the local variable name. They are not restored to locals in scope when the load is done since they are usually not the same. | |
localTableName | The optional name of the global table that will be used to store the loaded local variables upon a dump: load() or dump: loadItem() operation. This table is also cleared and used as an auxiliary table during save operations. If missing, the loadedLocalVariables global table will be used. | |
file | The file handle returned by an io.open() call, whose file pointer position will be stored on the state dump allowing a "rollback" for "append only" files. There are two modes of operation: 1) - The dump is created with a file handle and a single file name. When the state is saved, the current file size is saved. When loaded, the current file is truncated to the saved size. For this to work, the file must not be overwritten (cleared) on the restore operation (must not be opened with a "w" flag). It is also not compatible with multiple restore operations where the second one is a "later" state than the previous. 2) - The dump is created with a file handle and two file names. On this scenario, the first file name is the ultimate file desired by the user. It is used by the dump as a verification option. The second file name is a temporary file where the data is really saved during the simulation and should correspond to the given file handle. When the simulation is done, this file should be copied to the desired name and the temp file kept for dump restoration in a manner simmilar to what is done with the neutral file generation. When the state is saved, the current file size is saved. When loaded, first it is verified if the desired file saved on the dump is the same as the one received. If ok, the saved size is used to copy that many bytes from the temp file used on the dump creation to the current temp file (they should be different), overwritting its contents. | |
fileName | The file name on the single file mode or the first (ultimate) name on the two files option. See the description for the file parameter. | |
tempName | The second (temporary) file name on the two files option. See the description for the file parameter. | |
Returns: | Returns a numeric id that can be used to identify the item in calls to dump: saveItem() or nil on errors. |
Example:
dump:addGroupItem(mesh, groupId) dump:addGroupItem(bc, groupId) dump:addGroupItem(handle, groupId) | ||
---|---|---|
Description: | Adds a set of new state dump items to the state dump object. Each function signature corresponds to a different kind of data added to the state. The first one stores all the data associated with the given mesh, including geometry and additional mesh data. The second one stores all the data associated with the given boundary condition, including application point and additional properties data. Finally, the third stores the contents of the state associated with a handle returned by a process, such as solvers returned by the Fem process. Important 1: notice that adding a new state item to the dump state does not saves any data to the state. This operation just informs the state dump about the items that will be included on it, defining its layout. The data will only be saved when a call to dump: save() , dump: saveActiveGroups() or dump: saveItem() is made, with the contents at the moment of those calls. Important 2: After all calls to addGroupItem() , it is necessary to call dump: init() to complete the state dump initialization. | |
Parameters: | mesh | A mesh object. This option tells the state that all the data associated with this mesh should be added to the state dump, including the mesh geometry and all its associated data. When this option is used, there is no need to add mesh accessors to the state. |
bc | A boundary condition object. This option tells the state that all the data associated with this boundary condition should be added to the state dump, including application points and all its associated property data. When this option is used, there is no need to add boundary condition accessors to the state. | |
handle | A handle object returned by a GeMA process. Handle objects are usualy objects that store some process state and get passed as parameters into process calls, like solvers returned by the Fem process. | |
groupId | Each item is associated to a group, identified by a group id. This optional parameter defines the group (a value >= 0) to wich all dump items added by this call will be explicitely tied to. A value of -1 means that the items will be tied to the next available group id. A value of -2 means that the items will be tied to the same group id as the one used by the last added item. If not given, the default of -1 will be assumed. | |
Returns: | Returns the a numeric id of the group that received the items added by this function or nil on errors. |
Example:
dump:init() | |
---|---|
Description: | Initializes the state dump object. When dumping to a file, the dump file is created. Must be called after all desired items have been added to the state by calls to dump: addStateItem() and/or dump: addGroupItem() . |
Parameters: | None. |
Returns: | Returns true if the initialization was successfull, false otherwise. |
Example:
dump:itemGroup(itemId) | |
---|---|
Description: | Returns the group id of the group that the given item belongs to. |
Parameters: | itemId - The item id returned by a call to dump: addStateItem() . |
Returns: | Returns the group id. |
Example:
dump:numItems() | |
---|---|
Description: | Returns the number of state items added to the state dump object. Keep in mind that this can be different from the number of items added directly to the state due to some calls effectively adding more than one item. |
Parameters: | None. |
Returns: | Returns the number of items. |
Example:
dump:findItem(name, typeMask, typeValue, groupId, start) | ||
---|---|---|
Description: | Finds the item id of an item with the given name, type and belonging to the given group. This is a highly specific function, usefull only for those with some knowledge of how item dumps are named by GeMA and how item types are specified. | |
Parameters: | name | The desired item name. Can be a regular expression pattern. Uses a fairly common syntax as specified by the QRegExp Qt class. |
typeMask, typeValue | Type filter. For an item to be matched by this function, its type is anded with typeMask and the result must be equal to typeVale . In this way, a a mask of 0 with a value of 0 accepts any type (default if parameters are missing). | |
groupId | Group id filter. If the groupId is different from -1, a dump item will be matched by this function if its group id is equal to the given value. A value of -1 means that every group id is acceptable. If not given, a default of -1 will be assumed. | |
start | The start index for the search. Can be used to recover other matches than the first one. If not given, a default value of 0 will be assumed. | |
Returns: | Returns the item id of the first item that matches the given search criteria or nil if the item was not found. |
Example:
dump:setGroupActive(groupId, active) | ||
---|---|---|
Description: | Marks all the items of the given group as active or inactive. This selection, togeter with calls to dump: saveActiveGroups() and dump: loadActiveGroups() can be used to save and/or load data from a group of dump items. | |
Parameters: | groupId | The id of the group that will have its item states marked as active / inactive. A group id of -1 affects all the state items. |
active | The new state of the given item group. Either true for making the items active or false for making them inactive. | |
Returns: | Returns nothing. |
Example:
dump:save() | |
---|---|
Description: | Saves to the state dump the current data associated with all configured items. |
Parameters: | None. |
Returns: | Returns true if the operation was successfull, false otherwise. |
Example:
dump:load() | |
---|---|
Description: | Loads from the state dump the data associated with all configured items, updating them. |
Parameters: | None. |
Returns: | Returns true if the operation was successfull, false otherwise. |
Example:
dump:saveItem(id, size, offset) | ||
---|---|---|
Description: | Saves to the state dump the current data associated with a single dump item. If the given size is different from zero (or nil ), only part of the item data will be saved. Keep in mind that currently, only accessor and matrix items support partial saving. When saving data to disk, consider calling dump: flush() after this call. | |
Parameters: | id | The item id returned by the call to dump: addStateItem() used to add this item to the state. |
size, offset | Optional parameters. When size is different from 0 or nil , they together define the partial data to be saved. Both parameters are given as a number of entries and not bytes. A value of 100, 20 for an accessor item means that 100 data entries will be saved, starting from the 20th value. | |
Returns: | Returns true if the operation was successfull, false otherwise. |
Example:
dump:loadItem(id, size, offset) | ||
---|---|---|
Description: | Loads from the state dump the data associated with a single dump item, updating it. If the given size is different from zero (or nil ), only part of the item data will be loaded. Keep in mind that currently, only accessor and matrix items support partial loading. | |
Parameters: | id | The item id returned by the call to dump: addStateItem() used to add this item to the state. |
size, offset | Optional parameters. When size is different from 0 or nil , they together define the partial data to be loaded. Both parameters are given as a number of entries and not bytes. A value of 100, 20 for an accessor item means that 100 data entries will be loaded, starting from the 20th value. | |
Returns: | Returns true if the operation was successfull, false otherwise. |
Example:
dump:saveActiveGroups() | |
---|---|
Description: | Saves to the state dump the current data associated with all the items belonging to the set of active groups. Item groups are marked as active / inactive by calls to dump: setGroupActive() . |
Parameters: | None. |
Returns: | Returns true if the operation was successfull, false otherwise. |
Example:
dump:loadActiveGroups() | |
---|---|
Description: | Loads from the state dump the data associated with all the items belonging to the set of active groups. Item groups are marked as active / inactive by calls to dump: setGroupActive() . |
Parameters: | None. |
Returns: | Returns true if the operation was successfull, false otherwise. |
Example:
dump:flush() | |
---|---|
Description: | For state dumps saving data to a file, this function makes sure that the file was flushed to disk. Calling this function is only usefull after calls to dump: saveItem() since calls to dump: save() will automatically flush the file. |
Parameters: | None. |
Returns: | Nothing. |
Example:
dump:clear() | |
---|---|
Description: | For state dumps saving data to memory, this function discards the used memory after the state dump is not needed anymore. |
Parameters: | None. |
Returns: | Nothing. |
Example: