Base TypeLoader class. More...
Public Member Functions | |
| def | run (self) |
| Run the loader. More... | |
| def | run_two_phases (self) |
| Run the loader for two phases loading. More... | |
| def | set_log_error (self, path) |
| Sets a log error file. More... | |
| def | set_type (self, type) |
| Sets the type to be loaded. More... | |
| def | set_locale (self, locale_str) |
| Sets the locale that will be used to read the data. More... | |
| def | set_timestamp_format (self, timestamp_format) |
| Sets a specific timestamp format. More... | |
| def | set_attributes (self, attrs) |
| Sets the list of Attributes. More... | |
| def | register (self, tel) |
| Registers a new listener. More... | |
| def | set_frequency (self, freq) |
| Sets the frequency of listener notification. More... | |
| def | set_row_reader (self, rr) |
| Sets the input data source. More... | |
| def | set_attribute_positions (self, attrs_pos) |
| Sets the list of attribute positions. More... | |
| def | run_n_phases (self, partitions) |
| Run the loader for N phases loading. More... | |
| def | set_graph (self, graph) |
| Sets the graph where the data will be loaded. More... | |
| def | set_log_off (self) |
| Truns off all the error reporting. More... | |
Detailed Description
Base TypeLoader class.
Base class to load a node or edge type from a graph using a RowReader.
TypeLoaderListener can be registered to receive information about the progress of the load process by means of TypeLoaderEvent. The default frequency of notification to listeners is 100000.
Check out the 'Data import' section in the SPARKSEE User Manual for more details on this.
Member Function Documentation
| def sparksee.TypeLoader.register | ( | self, | |
| tel | |||
| ) |
Registers a new listener.
- Parameters
-
tel TypeLoaderListener to be registered.
| def sparksee.TypeLoader.run | ( | self | ) |
Run the loader.
- Exceptions
-
RuntimeError null IOError null
| def sparksee.TypeLoader.run_n_phases | ( | self, | |
| partitions | |||
| ) |
Run the loader for N phases loading.
Firstly load all objects (and create them if necessary) and secondly loads all the attributes. But in this case, attributes are loaded one by one. This way, if there are three attributes, then 4 traverses are necessary.
Working on this mode it is necessary to build a temporary file.
- Parameters
-
partitions [in] Number of horizontal partitions to perform the load.
- Exceptions
-
RuntimeError null IOError null
| def sparksee.TypeLoader.run_two_phases | ( | self | ) |
Run the loader for two phases loading.
Firstly load all objects (and create them if necessary) and secondly loads all the attributes.
Working on this mode it is necessary to build a temporary file.
- Exceptions
-
RuntimeError null IOError null
| def sparksee.TypeLoader.set_attribute_positions | ( | self, | |
| attrs_pos | |||
| ) |
Sets the list of attribute positions.
- Parameters
-
attrs_pos [in] Attribute positions (column index >=0).
| def sparksee.TypeLoader.set_attributes | ( | self, | |
| attrs | |||
| ) |
Sets the list of Attributes.
- Parameters
-
attrs [in] Attribute identifiers to be loaded
| def sparksee.TypeLoader.set_frequency | ( | self, | |
| freq | |||
| ) |
Sets the frequency of listener notification.
- Parameters
-
freq [in] Frequency in number of rows managed to notify progress to all listeners
| def sparksee.TypeLoader.set_graph | ( | self, | |
| graph | |||
| ) |
Sets the graph where the data will be loaded.
- Parameters
-
graph [in] Graph.
| def sparksee.TypeLoader.set_locale | ( | self, | |
| locale_str | |||
| ) |
Sets the locale that will be used to read the data.
It should match the locale used in the rowreader.
- Parameters
-
locale_str [in] The locale string for the read data. See CSVReader.
| def sparksee.TypeLoader.set_log_error | ( | self, | |
| path | |||
| ) |
Sets a log error file.
By default errors are thrown as a exception and the load process ends. If a log file is set, errors are logged there and the load process does not stop.
- Parameters
-
path [in] The path to the error log file.
- Exceptions
-
IOError If bad things happen opening the file.
| def sparksee.TypeLoader.set_log_off | ( | self | ) |
Truns off all the error reporting.
The log file will not be created and no exceptions for invalid data will be thrown. If you just want to turn off the logs, but abort at the first error what you should do is not call this method and not set a logError file.
| def sparksee.TypeLoader.set_row_reader | ( | self, | |
| rr | |||
| ) |
Sets the input data source.
- Parameters
-
rr [in] Input RowReader.
| def sparksee.TypeLoader.set_timestamp_format | ( | self, | |
| timestamp_format | |||
| ) |
Sets a specific timestamp format.
- Parameters
-
timestamp_format [in] A string with the timestamp format definition.
| def sparksee.TypeLoader.set_type | ( | self, | |
| type | |||
| ) |
Sets the type to be loaded.
- Parameters
-
type [in] Type identifier.
