Sparksee allows some configuration variables to be set in order to update or monitorize the behavior of the system, as well as to make the deployment of Sparksee-based applications easier.
There are two alternatives for setting-up the configuration:
SparkseeConfig class.Both can even be used at the same time. In case of conflict, the settings from the SparkseeConfig class have a higher priority.
SparkseeConfig classThe SparkseeConfig class defines a setter and a getter
for each of the variables that can be specified for Sparksee
configuration. If not changed, all the variables are loaded with default
values. All these values can be overwritten by the user calling the
corresponding setter or loading a properties file.
In order to set up a configuration the user must create an instance
of the SparkseeConfig class, which will have the default
values. The path to a configuration file can be provided to the
SparkseeConfig constructor. If it’s provided it will be
used to override the default settings and it will be used to store any
changes including the downloaded license key.
The Sparksee license key, by default, will be automatically
downloaded when you try to create a Sparksee class with a
SparkseeConfig that has valid client id and license id
values but no license key already set (or it is expired or close to
expire). And if the key can be successfully downloaded, it will be saved
to the specified config file (if provided). So providing a path to a
config file when creating a SparkseeConfig object is highly recommended
to avoid having to donwload the license key every time.
All the variables in SparkseeConfig may be set if the user needs to change them. We strongly recommend understanding each of the variables prior to changing their default values:
SparkseeConfig cfg = new SparkseeConfig("sparksee.cfg");
cfg.setClientId("Your client identifier");
cfg.setLicenseId("Your license identifier");
cfg.setCacheMaxSize(2048); // 2 GB
cfg.setLogFile("HelloSparksee.log");
...
Sparksee sparksee = new Sparksee(cfg);
Database db = sparksee.create("HelloSparksee.gdb", "HelloSparksee");SparkseeConfig cfg = new SparkseeConfig("sparksee.cfg");
cfg.SetClientId("Your client identifier");
cfg.SetLicenseId("Your license identifier");
cfg.SetCacheMaxSize(2048); // 2 GB
cfg.SetLogFile("HelloSparksee.log");
...
Sparksee sparksee = new Sparksee(cfg);
Database db = sparksee.Create("HelloSparksee.gdb", "HelloSparksee");SparkseeConfig cfg(L"sparksee.cfg");
cfg.SetClientId(L"Your client identifier");
cfg.SetLicenseId(L"Your license identifier");
cfg.SetCacheMaxSize(2048); // 2 GB
cfg.SetLogFile(L"HelloSparksee.log");
...
Sparksee * sparksee = new Sparksee(cfg);
Database * db = sparksee->Create(L"HelloSparksee.gdb", L"HelloSparksee");cfg = sparksee.SparkseeConfig("sparksee.cfg")
cfg.set_client_id("Your client identifier")
cfg.set_license_id("Your license identifier")
cfg.set_cache_max_size(2048) # 2 GB
cfg.set_log_file("Hellosparksee.log")
...
sparks = sparksee.Sparksee(cfg)
db = sparks.create(u"Hellosparksee.gdb", u"HelloSparksee")STSSparkseeConfig *cfg = [[STSSparkseeConfig alloc] initWithPath: @"sparksee.cfg"];
[cfg setClientId: @"Your client identifier"];
[cfg setLicenseId: @"Your license identifier"];
[cfg setCacheMaxSize: 2048]; // 2 GB
[cfg setLogFile: @"HelloSparksee.log"];
...
STSSparksee *sparksee = [[STSSparksee alloc] initWithConfig: cfg];
//[cfg release];
STSDatabase *db = [sparksee create: @"HelloSparksee.gdb" alias: @"HelloSparksee"];As previously explained, as an alternative to
SparkseeConfig methods, the user can load Sparksee
configuration variables from a properties file in order to make
development of their applications using Sparksee easier.
A properties file is a plain-text file where there is one line per
property. Each property is defined by a key and a value as follows:
key=value.
This is an example of a Sparksee configuration properties file:
sparksee.clientId="Your client identifier"
sparksee.licenseId="Your license identifier"
sparksee.io.cache.maxsize=2048
sparksee.log.file="HelloSparksee.log"By default, SparkseeConfig tries to load all the
variables defined in the ./sparksee.cfg file (in the
execution directory). If the user has not created this file, the default
values will be assumed.
The recommended way of loading properties since Sparksee 6.0 is to
provide the configuration file path to the SparkseeConfig
class constructor. This way, the settings in the file will be loaded,
and the file can be saved either manually or automatically when a new
license key is downloaded (when the SparkseeConfig is used
at the Sparksee class constructor without having a valid
license key but with valid client and license ids).
SparkseeConfig cfg = new SparkseeConfig("sparksee/config/dir/mysparksee.cfg");
Sparksee sparksee = new Sparksee(cfg);
Database db = sparksee.create("HelloSparksee.gdb", "HelloSparksee");SparkseeConfig cfg = new SparkseeConfig("sparksee/config/dir/mysparksee.cfg");
Sparksee sparksee = new Sparksee(cfg);
Database db = sparksee.Create("HelloSparksee.gdb", "HelloSparksee");SparkseeConfig cfg(L"sparksee/config/dir/mysparksee.cfg");
Sparksee * sparksee = new Sparksee(cfg);
Database * db = sparksee->Create(L"HelloSparksee.gdb", L"HelloSparksee");cfg = sparksee.SparkseeConfig("sparksee/config/dir/mysparksee.cfg")
sparks = sparksee.Sparksee(cfg)
db = sparks.create(u"Hellosparksee.gdb", u"HelloSparksee")STSSparkseeConfig *cfg = [[STSSparkseeConfig alloc] initWithPath: @"sparksee/config/dir/mysparksee.cfg"];
STSSparksee *sparksee = [[STSSparksee alloc] initWithConfig: cfg];
//[cfg release];
STSDatabase *db = [sparksee create: @"HelloSparksee.gdb" alias: @"HelloSparksee"];The old way of loading a properties file was loading them with the
SparkseeProperties before the SparkseeConfig
class is instantiated. The SparkseeProperties class
contains a method for specifying a different file name and path to be
loaded.
To load Sparksee configuration variables from a different file rather
than the default, the method load must be called before the
SparkseeConfig class is instantiated, as we can see in the
following examples where the mysparksee.cfg file is used to
load Sparksee configuration variables:
SparkseeProperties.load("sparksee/config/dir/mysparksee.cfg");
SparkseeConfig cfg = new SparkseeConfig();
Sparksee sparksee = new Sparksee(cfg);
Database db = sparksee.create("HelloSparksee.gdb", "HelloSparksee");SparkseeProperties.Load("sparksee/config/dir/mysparksee.cfg");
SparkseeConfig cfg = new SparkseeConfig();
Sparksee sparksee = new Sparksee(cfg);
Database db = sparksee.Create("HelloSparksee.gdb", "HelloSparksee");SparkseeProperties::Load(L"sparksee/config/dir/mysparksee.cfg");
SparkseeConfig cfg;
Sparksee * sparksee = new Sparksee(cfg);
Database * db = sparksee->Create(L"HelloSparksee.gdb", L"HelloSparksee");sparksee.SparkseeProperties.load("sparksee/config/dir/mysparksee.cfg");
cfg = sparksee.SparkseeConfig()
sparks = sparksee.Sparksee(cfg)
db = sparks.create(u"Hellosparksee.gdb", u"HelloSparksee")[STSSparkseeProperties load: @"sparksee/config/dir/mysparksee.cfg"];
STSSparkseeConfig *cfg = [[STSSparkseeConfig alloc] init];
STSSparksee *sparksee = [[STSSparksee alloc] initWithConfig: cfg];
//[cfg release];
STSDatabase *db = [sparksee create: @"HelloSparksee.gdb" alias: @"HelloSparksee"];Here is the list of the existing Sparksee configuration variables
divided by categories, with an identifier name, a description, the valid
format for the value, the default value and the corresponding method to
set each variable in the SparkseeConfig class. Please note
that the identifier name is the key to be used in the properties
file.
Valid formats for the variables values are:
A string, optionally quoted.
A number.
A time unit: <X>[D|H|M|S|s|m|u]
where:
<X> is a numberD for days, H for hours, M for
minutes, S or s for seconds, m
for milliseconds and u for microseconds. If no unit
character is given, seconds are assumed.sparksee.clientId: Client
identifier. It’s your client identifier that Sparsity should have sent
you after you appplied for a license.
Value format: String. Default value: “”
This variable can be set by calling
SparkseeConfig#setClientId.
This is a mandatory setting since Sparksee 6.0.sparksee.licenseId: License
identifier. It’s the identifier of the license that Sparsity should have
sent you after you appplied for a license.
Value format: String. Default value: “”
This variable can be set by calling
SparkseeConfig#setLicenseId.
This is a mandatory setting since Sparksee 6.0.sparksee.license: License key code.
This setting is expected to be automatically set for you using the
client and license identifiers since Sparksee 6.0. You should only set
it yourself on Sparksee versions < 6.0 or on special cases and you
may need to contact Sparsity first to get it.
Value format: String. Default value: “”
This variable can be set by calling
SparkseeConfig#setLicense
Details about logging capabilities & value information are given in the ‘Logging’ section of the ‘Maintenance and monitoring’ chapter.
sparksee.log.level: logging level.
Allowed values are (case insensitive): off, severe, warning, info,
config, fine.
Value format: String. Default value: info.
This variable can be set by calling
SparkseeConfig#setLogLevel
sparksee.log.file: log file. This
is ignored when the log level is ‘off’.
Value format: String. Default value: “sparksee.log”
This variable can be set by calling
SparkseeConfig#setLogFile
Details about the use of transactions in Sparksee are given in the ‘Transactions’ section of the ‘Graph Database’ chapter.
sparksee.io.rollback: enable
(“true”) or disable (“false”) the rollback functionality.
Value format: String. Default value: “true”
This variable can be set by calling
SparkseeConfig#setRollbackEnabled
Details about the recovery module are given in the ‘Recovery’ section of the ‘Maintenance and monitoring’ chapter.
sparksee.io.recovery: enable
(“true”) or disable (“false”) the recovery functionality. If disabled,
all the other related variables are ignored.
Value format: String. Default value: “false”
This variable can be set by calling
SparkseeConfig#setRecoveryEnabled
sparksee.io.recovery.logfile:
recovery log file. Empty string means that the recovery log file will be
the same as the database path with the format “.log”. For example, if
the database is at “database.gdb” then the recovery log file will be
“database.gdb.log”.
Value format: String. Default value: “”
This variable can be set by calling
SparkseeConfig#setRecoveryLogFile
sparksee.io.recovery.cachesize:
maximum size for the recovery cache in extents.
Value format: number. Default value: 256 (for the default extent size)
This variable can be set by calling
SparkseeConfig#setRecoveryCacheMaxSize
sparksee.io.recovery.checkpointTime:
checkpoint frequency for the recovery cache.
Value format: time unit. Values must be greater than or equal to 1 second. Default value: 60
This variable can be set by calling
SparkseeConfig#setRecoveryCheckpointTime
sparksee.storage.extentsize: extent
size in KB. An extent is the unit for I/O operations. It must be the
same for the whole life of the database. Therefore, it can be set when
the database is created and it cannot be updated subsequently.
Recommended sizes are: 4, 8, 16, 32 or 64. Note large extent sizes will
have a penalty performance if recovery is enabled.
Value format: number. Default value: 4
This variable can be set by calling
SparkseeConfig#setExtentSize
sparksee.storage.extentpages:
number of pages per extent. A page is the logical unit of allocation
size for the internal data structures of the system. Therefore, it can
be set when the database is created and it cannot be updated
subsequently.
Value format: number. Default value: 1
This variable can be set by calling
SparkseeConfig#setExtentPages
sparksee.storage.checksum: whether
to create new databases with checksums support enabled or not. Trying to
open an existing image with a value of this variable different from that
when the database was created, will throw an error.
Value format: Boolean. Default value: true
This variable can be set by calling
SparkseeConfig#setChecksumEnabled
sparksee.storage.aesKey: The AES
encryption key used to encrypt/decrypt the database. Trying to open an
existing image with a value of this variable different from that when
the database was created, will throw an error. Supported key sizes are 8
bytes (64 bits), 16 bytes (128 bits) and 32 bytes (256 bits).
Setting this option by means of the configuration file is
discouraged for being insecure. Use the API instead.
Value format: String. Default value: “”
This variable can be set by calling
SparkseeConfig#setAESEncryptionEnabled
sparksee.storage.aesIV: The AES
encryption Initialization Vector (IV) used to encrypt/decrypt the
database. Trying to open an existing image with a value of this variable
different from that when the database was created, will throw an error.
The length of the IV is 16 bytes (128 bits). Setting this option
by means of the configuration file is discouraged for being insecure.
Use the API instead.
Value format: String. Default value: “”
This variable can be set by calling
SparkseeConfig#setAESEncryptionEnabled
Additionally, AES key and IV can be automatically generated with
SparkseeConfig#setAESEncryptionEnabled overload that takes
the desired size of the key as a parameter. In such a case, the randomly
generated key and IV must be retrieved and stored by means of the
SparkseeConfig#getAESKey and
SparkseeConfig#getAESIV methods, respectively.
The cache is split into different areas, the most important being for the persistent data whilst the rest are for the running user’s sessions. The size of these areas is automatic, dynamic and adaptive depending on the requirements of each of the parts. Thus, there is constant negotiation between them. Although default values might be the best configuration, the user can adapt them for very specific requirements, taking into account the fact that negotiation between areas only happens when the pool is using memory in the range [minimum, maximum]. Therefore pools are always guaranteed to have at least the minimum memory of the configuration and never be larger than the maximum.
sparksee.io.cache.maxsize: maximum
size for all the cache pools in MBs (persistent pools as well as all
session & other temporary pools). A value for this variable of zero
means unlimited, which is in fact all the available memory (when the
system starts) minus 512 MB (which is a naive estimate of the minimum
memory to be used by the OS).
Value format: number. Default value: 0
This variable can be set by calling
SparkseeConfig#setCacheMaxSize
sparksee.io.pool.frame.size: number
of extents per frame. Whereas the extent size determines the I/O unit,
the frame size is the allocation/deallocation unit in number of extents.
Thus, values higher than 1 allow the pre-allocation of extents in bulk
which might make sense when loading large data sets, especially if
working with a small extent size.
Value format: number. Default value: 1
This variable can be set by calling
SparkseeConfig#setPoolFrameSize
sparksee.io.pool.persistent.minsize:
minimum size in number of frames for the persistent pool where zero
means unlimited.
Value format: number. Default value: 64
This variable can be set by calling
SparkseeConfig#setPoolPersistentMinSize
sparksee.io.pool.persistent.maxsize:
maximum size in number of frames for the persistent pool where zero
means unlimited.
Value format: number. Default value: 0
This variable can be set by calling
SparkseeConfig#setPoolPersistentMaxSize
sparksee.io.pool.temporal.minsize:
minimum size in number of frames for the session/temporal pools where
zero means unlimited. It makes sense to have a larger minimum size in
case of highly memory-consuming sessions.
Value format: number. Default value: 16
This variable can be set by calling
SparkseeConfig#setPoolTemporaryMinSize
sparksee.io.pool.temporal.maxsize:
maximum size in number of frames for the session/temporal pool where
zero means unlimited.
Value format: number. Default value: 0
This variable can be set by calling
SparkseeConfig#setPoolTemporaryMaxSize
sparksee.io.pool.partitions: The
number of partitions the pools are partitioned into. Increasing this
value reduces the amount of contention to the pool in applications with
multiple threads querying the database. On the contrary, larger values
require larger values of sparksee.io.cache.maxsize.
Value format: number. Default value: 16 (Linux, Windows and Mac), 1 (Android and IOS)
This variable can be set by calling
SparkseeConfig#setPoolPartitions
For development and debugging purposes it may be useful to enable the cache statistics. More details about this can be found in the ‘Statistics’ section of the ‘Maintenance and monitoring’ chapter.
sparksee.cache.statistics: enable
(“true”) or disable (“false”) storing of cache statistics.
Value format: String. Default value: false
This variable can be set by calling
SparkseeConfig#setCacheStatisticsEnabled
sparksee.cache.statisticsFile: file
where cache statistics are stored. This is ignored when the cache
statistics are disabled.
Value format: String. Default value: statistics.log
This variable can be set by calling
SparkseeConfig#setCacheStatisticsFile
sparksee.cache.statisticsSnapshotTime:
frequency for storing the cache statistics. This is ignored when the
cache statistics are disabled.
Value format: time unit. Default value: 1000
This variable can be set by calling
SparkseeConfig#setCacheStatisticsSnapshotTime
High availability configuration is explained in detail in the ‘High availability’ chapter.
sparksee.ha: enables or disables HA
mode. If disabled, all other HA variables are ignored.
Value format: String. Default value: false
This variable can be set by calling
SparkseeConfig#setHighAvailabilityEnabled
sparksee.ha.ip: IP address and port
for the instance. It must be given as ip:port
Value format: String. Default value: localhost:7777
This variable can be set by calling
SparkseeConfig#setHighAvailabilityIP
sparksee.ha.coordinators:
comma-separated list of the ZooKeeper instances. For each instance, the
IP address and the port must be given as: ip:port.
Moreover, the port must correspond to the given clientPort
in the ZooKeeper configuration file.
Value format: String. Default value: “”
This variable can be set by calling
SparkseeConfig#setHighAvailabilityCoordinators
sparksee.ha.sync: synchronization
polling time. If 0, polling is disabled and synchronization is only
performed when the slave receives a write request, otherwise the
parameter fixes the frequency the slaves poll the master asking for
writes. The polling timer is reset if the slaves receive a write
request, as it synchronizes then.
Value format: time unit. Default value: 0.
This variable can be set by calling
SparkseeConfig#setHighAvailabilitySynchronization
sparksee.ha.master.history:
master’s history log. The history log is limited to a certain period of
time, so writes occurring beyond that period of time will be removed
from it and the master will not accept requests from those deleted
Sparksee slaves. For example, in case of 12H, the master
will store all write operations during the last 12 hours in the history
log. It will reject requests from a slave which has not been up dated in
the last 12 hours.
Value format: time unit. Default value: 1D
This variable can be set by calling
SparkseeConfig#setHighAvailabilityMasterHistory
In most cases default values are the best option. In fact, non-advanced users should consider setting only the following variables for a basic configuration:
sparksee.clientId and
sparksee.licenseId).sparksee.io.cache.maxsize).sparksee.io.recovery,
sparksee.io.recovery.logfile).sparksee.log.level and
sparksee.log.file).