sparksee.TypeLoader Class Reference

Base TypeLoader class. More...

Inheritance diagram for sparksee.TypeLoader:

Inheritance graph

List of all members.

Public Member Functions

def run_two_phases
 Run the loader for two phases loading.
def run_n_phases
 Run the loader for N phases loading.
def set_frequency
 Sets the frequency of listener notification.
def set_log_off
 Truns off all the error reporting.
def run
 Run the loader.
def set_log_error
 Sets a log error file.
def set_type
 Sets the type to be loaded.
def set_graph
 Sets the graph where the data will be loaded.
def set_row_reader
 Sets the input data source.
def register
 Registers a new listener.
def set_attribute_positions
 Sets the list of attribute positions.
def set_locale
 Sets the locale that will be used to read the data.
def set_timestamp_format
 Sets a specific timestamp format.
def set_attributes
 Sets the list of Attributes.


Detailed Description

Base TypeLoader class.

Base class to load a node or edge type from a graph using a RowReader.

TypeLoaderListener can be registered to receive information about the progress of the load process by means of TypeLoaderEvent. The default frequency of notification to listeners is 100000.

Check out the 'Data import' section in the SPARKSEE User Manual for more details on this.

Author:
Sparsity Technologies http://www.sparsity-technologies.com

Member Function Documentation

def sparksee.TypeLoader.run_two_phases (   self  ) 

Run the loader for two phases loading.

Firstly load all objects (and create them if necessary) and secondly loads all the attributes.

Working on this mode it is necessary to build a temporary file.

Exceptions:
IOError null
RuntimeError null

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.run_n_phases (   self,
  partitions 
)

Run the loader for N phases loading.

Firstly load all objects (and create them if necessary) and secondly loads all the attributes. But in this case, attributes are loaded one by one. This way, if there are three attributes, then 4 traverses are necessary.

Working on this mode it is necessary to build a temporary file.

Parameters:
partitions [in] Number of horizontal partitions to perform the load.
Exceptions:
IOError null
RuntimeError null

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_frequency (   self,
  freq 
)

Sets the frequency of listener notification.

freq [in] Frequency in number of rows managed to notify progress to all listeners

Parameters:
freq null

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_log_off (   self  ) 

Truns off all the error reporting.

The log file will not be created and no exceptions for invalid data will be thrown. If you just want to turn off the logs, but abort at the first error what you should do is not call this method and not set a logError file.

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.run (   self  ) 

Run the loader.

Exceptions:
IOError null
RuntimeError null

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_log_error (   self,
  path 
)

Sets a log error file.

By default errors are thrown as a exception and the load process ends. If a log file is set, errors are logged there and the load process does not stop.

Parameters:
path null
Exceptions:
IOError If bad things happen opening the file.

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_type (   self,
  t 
)

Sets the type to be loaded.

Parameters:
t [in] Type identifier.

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_graph (   self,
  g 
)

Sets the graph where the data will be loaded.

Parameters:
g [in] Graph.

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_row_reader (   self,
  rr 
)

Sets the input data source.

Parameters:
rr [in] Input RowReader.

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.register (   self,
  tel 
)

Registers a new listener.

Parameters:
tel TypeLoaderListener to be registered.

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_attribute_positions (   self,
  attrsPos 
)

Sets the list of attribute positions.

attrsPos [in] Attribute positions (column index >=0).

Parameters:
attrsPos null

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_locale (   self,
  localeStr 
)

Sets the locale that will be used to read the data.

It should match the locale used in the rowreader.

Parameters:
localeStr [in] The locale string for the read data. See CSVReader.

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_timestamp_format (   self,
  timestampFormat 
)

Sets a specific timestamp format.

Parameters:
timestampFormat null

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.

def sparksee.TypeLoader.set_attributes (   self,
  attrs 
)

Sets the list of Attributes.

Parameters:
attrs [in] Attribute identifiers to be loaded

Reimplemented in sparksee.NodeTypeLoader, and sparksee.EdgeTypeLoader.


Generated on Fri Feb 7 13:22:20 2014 for SparkseePython by  doxygen 1.5.6