Package org.openrefine.importers

Class ImporterUtilities

java.lang.Object

org.openrefine.importers.ImporterUtilities

public class ImporterUtilities
extends Object

Constructor Summary

Constructors

Constructor	Description
ImporterUtilities()

Method Summary

Modifier and Type	Method	Description
static void	appendColumnName(List<String> columnNames, int index, String name)
static com.fasterxml.jackson.databind.node.ArrayNode	convertErrorsToJsonArray(List<Exception> exceptions)
static MultiFileReadingProgress	createMultiFileReadingProgress(ImportingJob job, List<ImportingFileRecord> fileRecords)	A proxy for reporting the progress of reading individual files back into the global progress of an importing job.
static ProgressReporter	createProgressReporterForProjectSave(ImportingJob job)	Creates a progress reporter for the second phase of project import: saving the project in the workspace.
static ColumnModel	expandColumnModelIfNeeded(ColumnModel columnModel, int c)	Adds columns to a column model if it misses one column to store a cell at a given index.
static boolean	getBooleanOption(String name, Properties options, boolean def)
static int	getIntegerOption(String name, Properties options, int def)
static Reader	getReaderFromStream(InputStream inputStream, ImportingFileRecord fileRecord, String commonEncoding)
protected static Grid	mergeGrids(List<Grid> grids)	Given two grids with potentially different columns, unify the two into a single grid by adding columns of the second grid which are not present in the first at the end.
static String	mostCommonFormat(List<ImportingFileRecord> records)	Given a list of importing file records, return the most common format in them, or null if none of the records contain a format.
static InputStream	openAndTrackFile(String fileSource, File file, MultiFileReadingProgress progress)
static Serializable	parseCellValue(String text)
static ColumnModel	setupColumns(List<String> columnNames)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ImporterUtilities

public ImporterUtilities()

Method Detail

parseCellValue

public static Serializable parseCellValue(String text)

getIntegerOption

public static int getIntegerOption(String name,
                                   Properties options,
                                   int def)

getBooleanOption

public static boolean getBooleanOption(String name,
                                       Properties options,
                                       boolean def)

appendColumnName

public static void appendColumnName(List<String> columnNames,
                                    int index,
                                    String name)

setupColumns

public static ColumnModel setupColumns(List<String> columnNames)

createMultiFileReadingProgress

public static MultiFileReadingProgress createMultiFileReadingProgress(ImportingJob job,
                                                                      List<ImportingFileRecord> fileRecords)

A proxy for reporting the progress of reading individual files back into the global progress of an importing job. The importing process is split into two stages:

• the initial reading of the files required to create the corresponding Grid;
• the saving of the concatenated grids into the workspace.

We consider that the first phase accounts for the first 50% of progress and the saving phase accounts for the second 50% of the global progress. In the first phase, progress is measured by bytes read from the original files. In the second phase, progress is measured by proportion of rows saved (over the total number of rows).

createProgressReporterForProjectSave

public static ProgressReporter createProgressReporterForProjectSave(ImportingJob job)

Creates a progress reporter for the second phase of project import: saving the project in the workspace.

openAndTrackFile

public static InputStream openAndTrackFile(String fileSource,
                                           File file,
                                           MultiFileReadingProgress progress)
                                    throws FileNotFoundException

Throws:

FileNotFoundException

expandColumnModelIfNeeded

public static ColumnModel expandColumnModelIfNeeded(ColumnModel columnModel,
                                                    int c)

Adds columns to a column model if it misses one column to store a cell at a given index.

getReaderFromStream

public static Reader getReaderFromStream(InputStream inputStream,
                                         ImportingFileRecord fileRecord,
                                         String commonEncoding)

mostCommonFormat

public static String mostCommonFormat(List<ImportingFileRecord> records)

Given a list of importing file records, return the most common format in them, or null if none of the records contain a format.

Parameters:

records - the importing file records to read formats from

Returns:

the most common format, or null

convertErrorsToJsonArray

public static com.fasterxml.jackson.databind.node.ArrayNode convertErrorsToJsonArray(List<Exception> exceptions)

mergeGrids

protected static Grid mergeGrids(List<Grid> grids)

Given two grids with potentially different columns, unify the two into a single grid by adding columns of the second grid which are not present in the first at the end.