sparksee.TypeLoader Class Reference

Base TypeLoader class. More...

Inheritance diagram for sparksee.TypeLoader:

Public Member Functions
	run (self)
	Run the loader.
	run_two_phases (self)
	Run the loader for two phases loading.
	set_log_error (self, path)
	Sets a log error file.
	set_type (self, type)
	Sets the type to be loaded.
	set_locale (self, locale_str)
	Sets the locale that will be used to read the data.
	set_timestamp_format (self, timestamp_format)
	Sets a specific timestamp format.
	set_attributes (self, attrs)
	Sets the list of Attributes.
	register (self, tel)
	Registers a new listener.
	set_frequency (self, freq)
	Sets the frequency of listener notification.
	set_row_reader (self, rr)
	Sets the input data source.
	set_attribute_positions (self, attrs_pos)
	Sets the list of attribute positions.
	run_n_phases (self, partitions)
	Run the loader for N phases loading.
	set_graph (self, graph)
	Sets the graph where the data will be loaded.
	set_log_off (self)
	Truns off all the error reporting.

Detailed Description

Base TypeLoader class.

Base class to load a node or edge type from a graph using a RowReader.

TypeLoaderListener can be registered to receive information about the progress of the load process by means of TypeLoaderEvent. The default frequency of notification to listeners is 100000.

Check out the 'Data import' section in the SPARKSEE User Manual for more details on this.

Author

Sparsity Technologies http://www.sparsity-technologies.com

Member Function Documentation

register()

sparksee.TypeLoader.register	(		self,
			tel )

Registers a new listener.

Parameters

tel	TypeLoaderListener to be registered.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

run()

sparksee.TypeLoader.run

(

self

)

Run the loader.

ErrorFor other issues

Exceptions

RuntimeError	null
IOError	If bad things happen using to the RowReader.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

run_n_phases()

sparksee.TypeLoader.run_n_phases	(		self,
			partitions )

Run the loader for N phases loading.

Firstly load all objects (and create them if necessary) and secondly loads all the attributes. But in this case, attributes are loaded one by one. This way, if there are three attributes, then 4 traverses are necessary.

Working on this mode it is necessary to build a temporary file. ErrorFor other issues

Parameters

partitions

[in] Number of horizontal partitions to perform the load.

Exceptions

RuntimeError	null
IOError	If bad things happen using to the RowReader.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

run_two_phases()

sparksee.TypeLoader.run_two_phases

(

self

)

Run the loader for two phases loading.

Firstly load all objects (and create them if necessary) and secondly loads all the attributes.

Working on this mode it is necessary to build a temporary file. ErrorFor other issues

Exceptions

RuntimeError	null
IOError	If bad things happen using to the RowReader.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_attribute_positions()

sparksee.TypeLoader.set_attribute_positions	(		self,
			attrs_pos )

Sets the list of attribute positions.

Parameters

attrs_pos

[in] Attribute positions (column index >=0).

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_attributes()

sparksee.TypeLoader.set_attributes	(		self,
			attrs )

Sets the list of Attributes.

Parameters

attrs

[in] Attribute identifiers to be loaded

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_frequency()

sparksee.TypeLoader.set_frequency	(		self,
			freq )

Sets the frequency of listener notification.

Parameters

freq	[in] Frequency in number of rows managed to notify progress to all listeners

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_graph()

sparksee.TypeLoader.set_graph	(		self,
			graph )

Sets the graph where the data will be loaded.

Parameters

graph

[in] Graph.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_locale()

sparksee.TypeLoader.set_locale	(		self,
			locale_str )

Sets the locale that will be used to read the data.

It should match the locale used in the rowreader.

Parameters

locale_str

[in] The locale string for the read data. See CSVReader.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_log_error()

sparksee.TypeLoader.set_log_error	(		self,
			path )

Sets a log error file.

By default errors are thrown as a exception and the load process ends. If a log file is set, errors are logged there and the load process does not stop.

Parameters

path	[in] The path to the error log file.

Exceptions

IOError

If bad things happen opening the file.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_log_off()

sparksee.TypeLoader.set_log_off

(

self

)

Truns off all the error reporting.

The log file will not be created and no exceptions for invalid data will be thrown. If you just want to turn off the logs, but abort at the first error what you should do is not call this method and not set a logError file.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_row_reader()

sparksee.TypeLoader.set_row_reader	(		self,
			rr )

Sets the input data source.

Parameters

rr	[in] Input RowReader.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_timestamp_format()

sparksee.TypeLoader.set_timestamp_format	(		self,
			timestamp_format )

Sets a specific timestamp format.

Parameters

timestamp_format

[in] A string with the timestamp format definition.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.

set_type()

sparksee.TypeLoader.set_type	(		self,
			type )

Sets the type to be loaded.

Parameters

type	[in] Type identifier.

Reimplemented in sparksee.EdgeTypeLoader, and sparksee.NodeTypeLoader.