public abstract class AbstractConnector extends Object implements Connector
Modifier and Type | Field and Description |
---|---|
static int |
DATE_LENGTH_UNRESTRICTED |
protected String |
projectId
Project id
|
protected SourceSchema |
schema
The LDM schema of the data source
|
Modifier | Constructor and Description |
---|---|
protected |
AbstractConnector()
Default constructor
|
Modifier and Type | Method and Description |
---|---|
protected void |
checkLoadingStatus(String taskUri,
String tmpDir,
CliParams p,
ProcessingContext ctx)
Checks the status of data integration process in the GoodData platform
|
protected int |
copyAndTransform(CSVReader cr,
CSVWriter cw,
boolean transform,
int dateLength)
Copies the extracted data and transform them
|
void |
deploy(SLI sli,
List<Column> columns,
String dir,
String archiveName)
Create the GoodData data package with the ALL snapshots data
|
void |
dump(String file)
Dumps the source data CSV to a file
|
protected void |
dumpData(Command c,
CliParams p,
ProcessingContext ctx)
Dumps the data to CSV
|
static void |
expandDates(SourceSchema s) |
void |
extract(String dir)
Extracts the source data CSV to the database where it is going to be transformed
|
abstract void |
extract(String file,
boolean transform)
Extract rows
|
void |
extractAndTransfer(Command c,
String pid,
Connector cc,
boolean waitForFinish,
CliParams p,
ProcessingContext ctx)
Extract data from the internal database and transfer them to a GoodData project
|
String |
generateMaqlCreate()
Generates the MAQL for the data source
|
String |
getProjectId() |
Map<String,String> |
getResult() |
SourceSchema |
getSchema()
LDM schema getter
|
protected void |
initSchema(String configFileName)
Initializes the source and PDM schemas from the config file
|
static List<Column> |
populateColumnsFromSchema(SourceSchema schema) |
boolean |
processCommand(Command c,
CliParams cli,
ProcessingContext ctx)
Processes single command
|
protected void |
setIncremental(List<Column> cols)
Sets the incremental loading status for a part
|
protected void |
setProjectId(ProcessingContext ctx)
Sets the project id from context
|
void |
setProjectId(String projectId) |
void |
setSchema(SourceSchema schema)
LDM schema setter
|
protected void |
transferData(Command c,
CliParams p,
ProcessingContext ctx)
Transfers the data to GoodData project
|
protected SourceSchema schema
protected String projectId
public static final int DATE_LENGTH_UNRESTRICTED
public SourceSchema getSchema()
public void setSchema(SourceSchema schema)
public void extract(String dir) throws IOException
extract
in interface Connector
dir
- target directory where the data package will be storedIOException
- in case of IO issuespublic void dump(String file) throws IOException
dump
in interface Connector
file
- target fileIOException
- in case of IO issuesprotected int copyAndTransform(CSVReader cr, CSVWriter cw, boolean transform, int dateLength) throws IOException
cr
- - readercw
- - writertransform
- - perform transformations?dateLength
- - cuts the fate to first dateLength charsIOException
public abstract void extract(String file, boolean transform) throws IOException
file
- name of the target filetransform
- perform transformationsIOException
public void deploy(SLI sli, List<Column> columns, String dir, String archiveName) throws IOException
deploy
in interface Connector
sli
- the SLI interfacecolumns
- the SLI columnsdir
- target directory where the data package will be storedarchiveName
- the name of the target ZIP archiveIOException
- IO issuesprotected void initSchema(String configFileName) throws IOException
configFileName
- the config fileIOException
- in cas the config file doesn't existspublic static void expandDates(SourceSchema s)
public String generateMaqlCreate()
Connector
generateMaqlCreate
in interface Connector
public boolean processCommand(Command c, CliParams cli, ProcessingContext ctx) throws ProcessingException
processCommand
in interface Executor
c
- command to be processedcli
- parameters (commandline params)ctx
- processing contextProcessingException
- in case of issues during the command processingprotected void transferData(Command c, CliParams p, ProcessingContext ctx) throws IOException, InterruptedException
c
- commandp
- cli parametersctx
- current contextIOException
- IO issuesInterruptedException
- internal problem with making file writableprotected void dumpData(Command c, CliParams p, ProcessingContext ctx) throws IOException, InterruptedException
c
- commandp
- cli parametersctx
- current contextIOException
- IO issuesInterruptedException
- internal problem with making file writablepublic static List<Column> populateColumnsFromSchema(SourceSchema schema)
public void extractAndTransfer(Command c, String pid, Connector cc, boolean waitForFinish, CliParams p, ProcessingContext ctx) throws IOException, InterruptedException
extractAndTransfer
in interface Connector
c
- commandpid
- project idcc
- connectorwaitForFinish
- synchronous execution flagp
- cli parametersctx
- current contextIOException
- IO issuesInterruptedException
- internal problem with making file writableprotected void setIncremental(List<Column> cols)
cols
- SLI columnsprotected void checkLoadingStatus(String taskUri, String tmpDir, CliParams p, ProcessingContext ctx) throws InterruptedException, IOException
taskUri
- the uri where the task status is determinedtmpDir
- temporary dir where the temporary data reside. This directory will be deleted.p
- cli parametersctx
- current contextIOException
- IO issuesInterruptedException
- internal problem with making file writablepublic String getProjectId()
public void setProjectId(String projectId)
protected void setProjectId(ProcessingContext ctx) throws InvalidParameterException
ctx
- process contextInvalidParameterException
- if the project id isn't initializedCopyright © 2014. All rights reserved.