Using this reference
The catalogue contains 36 components: 6 Sources, 17 Transform, 8 Quality and 5 Targets. Visible names match Dova; identifiers and JSON keys are not translated. A component available in the editor does not establish that a connection is configured or that every worker can execute it.
Each example is a partial, synthetic configuration. Complete required fields for your case, authorized connections and runtime bindings before validation. Never place passwords inside the Mapping.
Sources
Flat File input (source_csv)
Reads delimited CSV, TSV or TXT. Headers determine columns; headerMappings projects, renames and converts only mapped fields. Without a header, columns assigns positional names.
Settings: format, delimiter, hasHeader, headerRowNumber, columns, headerMappings, encoding.
format: csv, tsv, txt; encoding: utf-8, utf-8-bom.
Synthetic example:
{
"format": "csv",
"hasHeader": true,
"delimiter": ","
}Limits: Use a one-character delimiter and UTF-8. With hasHeader=true, columns is not a field-selection switch. Set the header row before reviewing data.
JSON input (source_json)
Reads objects, JSON arrays or JSON Lines as records. recordPath locates records; fieldMappings selects source paths and output names/types. Without mappings it retains observed top-level fields and serializes nested objects or arrays as JSON text.
Settings: format, recordPath, fieldMappings, encoding.
format: json, jsonl; encoding: utf-8, utf-8-bom.
Synthetic example:
{
"format": "json",
"recordPath": "items"
}Limits: Paths use dotted names, not general JSONPath expressions. A missing nonnullable mapped field causes an error.
Excel input (source_excel)
Reads workbook sheets and ranges. Select sheets, headers and row/column bounds; headerMappings projects, renames and converts fields. includeSheetName can add sheet provenance.
Settings: workbookFormat, sheetMode, sheetName, sheetNames, sheetPattern, sheetNameRegex, includeHiddenSheets, hasHeader, headerRowNumber, dataStartRowNumber, startColumn, endColumn, columns, headerMappings, skipEmptyRows, includeSheetName, sheetNameField, passwordProtected.
workbookFormat: auto, xlsx, xlsm, xls; sheetMode: single, list, pattern, regex, all.
Synthetic example:
{
"sheetMode": "single",
"sheetName": "Sheet1",
"hasHeader": true,
"headerRowNumber": 1,
"dataStartRowNumber": 2
}Limits: Reader formats are xlsx, xlsm and xls. Sheets and ranges must exist. Supply a workbook password only for the authorized operation; do not store it in Mapping configuration.
Database input (source_database)
Reads a table/view or a read-only query through a compatible worker. In table mode, selectedColumns projects fields and schema/objectName identify the relation. In query mode, output columns come from sqlQuery.
Settings: connectionId, sourceMode, schema, objectType, objectName, sqlQuery, selectedColumns, fetchSize, queryTimeoutSeconds.
sourceMode: table, query; objectType: table, view.
Synthetic example:
{
"sourceMode": "table",
"objectName": "customers",
"selectedColumns": [
"name",
"amount"
]
}Limits: selectedColumns does not rewrite arbitrary SQL. Generic connector streaming supports table/view mode; queries depend on the connector and runtime. Selecting a connection type does not configure credentials.
API input (source_api)
Reads records with GET through a configured connection. responsePath locates records and fieldMappings projects nested paths into typed logical fields. Pagination can be none, page, offset or cursor.
Settings: connectionId, method, resourcePath, responsePath, fieldMappings, paginationMode, pageSize, retries, timeoutSeconds.
method: GET; paginationMode: none, page, offset, cursor.
Synthetic example:
{
"method": "GET",
"resourcePath": "/customers",
"paginationMode": "none"
}Limits: Actual support depends on the provider. SaaS connectors use their own resources/entities; they do not all accept generic REST pagination options.
Object storage input (source_object)
Reads CSV or JSONL from object storage. resource identifies the object, selectedColumns projects fields and delimiter applies to CSV.
Settings: connectionId, resource, format, selectedColumns, delimiter.
format: csv, jsonl.
Synthetic example:
{
"resource": "input/customers.csv",
"format": "csv",
"selectedColumns": [
"name",
"amount"
],
"delimiter": ","
}Limits: Requires a compatible connection, read permissions and a stable tabular schema. Displaying the component does not establish storage access.
Transform
Row filter (filter)
Keeps rows matching one field condition and preserves all their columns and values.
Settings: field, operator, value.
operator: equals, not_equals, contains, starts_with, ends_with, gt, gte, lt, lte, not_empty.
Synthetic example:
{
"field": "status",
"operator": "equals",
"value": "active"
}Limits: equals and not_equals compare case-sensitive text. contains, starts_with and ends_with ignore case. gt/gte/lt/lte convert to numbers. not_empty excludes null and blank text.
Field selector (select)
Projects each row and its output schema onto only the configured existing fields, in configured order. Drops all other fields while keeping row count.
Settings: fields.
Synthetic example:
{
"fields": [
"First_name",
"Last_name"
]
}Limits: Does not transform values, rename fields or filter rows. At least one selected field must exist. Unexpected final fields require checking the executed revision, connected branch and downstream components.
Field mapper (rename)
Renames fields listed in mapping while preserving their values, unmapped fields and all rows.
Settings: mapping.
Synthetic example:
{
"mapping": {
"old_name": "new_name"
}
}Limits: Output names must remain unique. It does not replace values or drop unmapped fields; add Field selector when projection is needed.
Calculated field (derive)
Adds or replaces one field using literal text and ${field} or ${param.name} interpolation. Copies all other fields and rows.
Settings: field, template.
Synthetic example:
{
"field": "label",
"template": "ID-${id}"
}Limits: It is not an expression engine: no arithmetic, LEFT, SUBSTR, slicing or initial capitalization. Output is text bounded to 4,000 characters.
Convert type (convert)
Converts a field to string, integer, number, boolean, date or datetime. An empty outputField replaces the source; a named output keeps the source and writes the converted result there.
Settings: field, targetType, outputField, onError, defaultValue.
targetType: string, integer, number, boolean, date, datetime; onError: fail, null, keep, default.
Synthetic example:
{
"field": "amount",
"targetType": "number",
"outputField": "amount_number",
"onError": "fail"
}Limits: date requires YYYY-MM-DD; datetime uses ISO 8601. There is no custom date format. onError accepts fail, null, keep or default; numbers must be finite.
Clean text (clean)
Applies operations in the configured order only to fields. Preserves all rows and columns; unselected fields keep their values and types, and null stays null.
Settings: fields, operations. Type at least one letter in Operations to see available matches and choose an operation.
| Operation | Effect |
|---|---|
trim | Removes whitespace from both ends. |
ltrim | Removes whitespace from the left end. |
rtrim | Removes whitespace from the right end. |
collapse_whitespace | Reduces whitespace groups to one space. |
lower | Converts text to lowercase. |
upper | Converts text to uppercase. |
capitalize | Uppercases the first character and lowercases the remainder. |
title | Capitalizes each word; whitespace and punctuation separate words. |
remove_accents | Removes accent marks. |
remove_line_breaks | Replaces each group of line breaks with one space, preserving word separation. |
To transform jORGE into Jorge, configure:
{
"fields": ["first_name"],
"operations": ["trim", "capitalize"]
}Order matters: trim removes surrounding whitespace before capitalization. Selected nonnull values become text; avoid capitalize when the remainder of a name must retain its original case.
Replace values (replace)
Replaces literal text in one field while preserving other columns and all rows. It is the direct choice for changing exactly jorge to Jorge.
Settings: field, find, replacement, matchMode, caseSensitive.
matchMode: contains, exact, starts_with, ends_with.
Synthetic example:
{
"field": "name",
"find": "jorge",
"replacement": "Jorge",
"matchMode": "exact",
"caseSensitive": true
}Limits: It does not use regular expressions. matchMode accepts exact, contains, starts_with, ends_with; caseSensitive controls case matching. This specific replacement does not capitalize arbitrary names.
Split field (split)
Splits text by a literal delimiter into outputFields. keepSource controls retaining the original field; trimParts trims pieces.
Settings: field, delimiter, outputFields, keepSource, trimParts.
Synthetic example:
{
"field": "full_name",
"delimiter": " ",
"outputFields": [
"first_name",
"last_name"
],
"keepSource": true,
"trimParts": true
}Limits: It does not extract characters by position. Splitting occurs at most one fewer time than the output count, leaving the remainder in the last field. Missing pieces become empty strings.
Concatenate fields (concat)
Joins values as text into outputField using separator; preserves source fields and all rows.
Settings: fields, outputField, separator, skipEmpty.
Synthetic example:
{
"fields": [
"first_name",
"last_name"
],
"outputField": "full_name",
"separator": " ",
"skipEmpty": true
}Limits: It does not add numbers. skipEmpty skips blank or whitespace-only pieces; null becomes empty text.
Join (join)
Combines exactly two immediate inputs on equality keys. Supports inner, left, right, full; emits both schemas and uses rightPrefix for right-side name collisions.
Settings: joinType, leftInputId, rightInputId, leftKey, rightKey, conditions, keyType, strategy, caseSensitive, trimStrings, nullsEqual, rightPrefix.
joinType: inner, left, right, full; keyType: auto, string, integer, number, boolean, date, datetime; strategy: auto, hash, sorted.
Synthetic example:
{
"joinType": "left",
"leftInputId": "customers",
"rightInputId": "orders",
"leftKey": "id",
"rightKey": "customer_id",
"keyType": "auto",
"strategy": "auto",
"caseSensitive": true,
"trimStrings": true,
"nullsEqual": false,
"rightPrefix": "right_"
}Limits: Set valid leftInputId, rightInputId and keys. Multiple conditions use AND. Many-to-many matches multiply rows. Nulls do not match by default. Expansion limits and auto, hash, sorted strategies apply.
Limit rows (limit)
Keeps a contiguous window of rows after offset, preserving columns and values.
Settings: count, offset.
Synthetic example:
{
"count": 10,
"offset": 0
}Limits: count: 1–250,000; offset: 0–250,000. It does not sort: add Row sort first for a top-N by a field.
Row sort (sort)
Sorts rows by one field in asc or desc direction, preserving all columns and rows.
Settings: field, direction.
direction: asc, desc.
Synthetic example:
{
"field": "amount",
"direction": "desc"
}Limits: Numerically convertible values sort numerically; other text ignores case. This is not multi-key sorting. Streaming requires a staging step.
Group aggregate (aggregate)
Produces one row per groupBy group, containing only grouping fields and the metric output. count counts rows; sum and avg aggregate one field.
Settings: groupBy, metric, field, outputField.
metric: count, sum, avg.
Synthetic example:
{
"groupBy": [
"region"
],
"metric": "sum",
"field": "amount",
"outputField": "total"
}Limits: It does not calculate between columns in the same row and has no min/max. sum/avg ignore values that cannot become finite numbers. Metric output is text. Empty groupBy groups all nonempty input.
Unpivot (unpivot)
Turns each column in fields into a row: keyField receives its original name and valueField its value. Keeps keepFields; when empty, retains the other input fields.
Settings: fields, keepFields, keyField, valueField, dropNulls.
Synthetic example:
{
"fields": [
"jan",
"feb"
],
"keepFields": [
"id"
],
"keyField": "month",
"valueField": "amount",
"dropNulls": false
}Limits: Expands rows. Output names cannot overwrite existing columns. dropNulls removes only null, keeping zero, false and empty text. This is not pivot.
Run sequence (sequence)
Adds an integer sequence start + row index × step, preserving rows, fields and order.
Settings: outputField, start, step.
Synthetic example:
{
"outputField": "sequence",
"start": 1,
"step": 1
}Limits: Restarts each run; it is not a persistent or globally unique identity. step cannot be zero and output cannot overwrite an existing field. Safe integer bounds apply.
Field fingerprint (hash_fields)
Adds a SHA-256 hexadecimal fingerprint over selected field names and canonical typed JSON values. Preserves original data.
Settings: fields, outputField.
Synthetic example:
{
"fields": [
"id",
"name"
],
"outputField": "fingerprint"
}Limits: Null and empty text differ. This is neither encryption nor anonymization. Input fields must exist and output cannot overwrite them; JSON depth is limited to 20 and numbers must be finite.
Partition rank (rank)
Adds a numeric position within partitionBy groups while retaining original row order. dense shares consecutive ranks; rank leaves gaps; row_number assigns unique positions.
Settings: partitionBy, field, direction, mode, outputField.
direction: asc, desc; mode: dense, rank, row_number.
Synthetic example:
{
"partitionBy": [
"region"
],
"field": "amount",
"direction": "desc",
"mode": "dense",
"outputField": "rank"
}Limits: The ordered field must be finite numeric on every row; null, blank and boolean fail. Output cannot overwrite existing columns. Empty partitionBy treats all input as one group. Final output is not sorted.
Quality
Default value (defaults)
Fills a field when null, blank or both according to when; preserves present values, other columns and all rows. It may add a missing field.
Settings: field, value, when.
when: null_or_empty, null, empty.
Synthetic example:
{
"field": "name",
"value": "Unknown",
"when": "null_or_empty"
}Limits: Zero and false are retained. It does not replace nonempty literal values; use Replace values for that goal.
Deduplicate (dedupe)
Keeps the first row of each key combination with all its fields. Empty fields compares all input columns.
Settings: fields.
Synthetic example:
{
"fields": [
"customer_id"
]
}Limits: Drops duplicates but does not merge values or automatically choose the latest record. Sort first if the retained first row matters. Comparison uses typed JSON values.
Quality check (validate)
Checks one field for empty values. Fails execution when their percentage exceeds maxNullPercent; otherwise passes every row and field unchanged.
Settings: field, maxNullPercent.
Synthetic example:
{
"field": "name",
"maxNullPercent": 0
}Limits: Null and whitespace-only text are empty; zero and false are not. The threshold is 0–100. It does not fill or remove values.
Required fields (quality_required)
Marks a row invalid when any required field is null or blank. Execution fails if the percentage exceeds maxInvalidPercent.
Settings: fields, maxInvalidPercent.
Synthetic example:
{
"fields": [
"id",
"name"
],
"maxInvalidPercent": 0
}Limits: On threshold success, even invalid rows remain. It does not fill data or discard records.
Type check (quality_type)
Checks whether values can convert to expectedType without converting them. Preserves rows and values when the threshold is not exceeded.
Settings: field, expectedType, allowEmpty, maxInvalidPercent.
expectedType: string, integer, number, boolean, date, datetime.
Synthetic example:
{
"field": "amount",
"expectedType": "number",
"allowEmpty": false,
"maxInvalidPercent": 0
}Limits: Use Convert type for transformation. allowEmpty controls missing values. Numeric text may pass and remain text.
Pattern check (quality_pattern)
Checks full-field matching against a safe regular expression. Preserves data when maxInvalidPercent is not exceeded.
Settings: field, pattern, allowEmpty, maxInvalidPercent.
Synthetic example:
{
"field": "code",
"pattern": "[A-Z]{3}",
"allowEmpty": false,
"maxInvalidPercent": 0
}Limits: Safety limits apply to pattern. It does not extract, replace or filter values. allowEmpty controls missing values separately.
Range check (quality_range)
Checks finite numbers against minimum and/or maximum, with inclusive or exclusive bounds. Keeps data on threshold success.
Settings: field, minimum, maximum, inclusive, allowEmpty, maxInvalidPercent.
Synthetic example:
{
"field": "age",
"minimum": "18",
"maximum": "100",
"inclusive": true,
"allowEmpty": false,
"maxInvalidPercent": 0
}Limits: At least one bound is required. It does not clamp or replace out-of-range values. allowEmpty controls missing values.
Allowed values (quality_domain)
Checks exact textual membership in allowedValues, respecting caseSensitive. Preserves data on threshold success.
Settings: field, allowedValues, caseSensitive, allowEmpty, maxInvalidPercent.
Synthetic example:
{
"field": "status",
"allowedValues": [
"active",
"inactive"
],
"caseSensitive": true,
"allowEmpty": false,
"maxInvalidPercent": 0
}Limits: The list cannot be empty. It does not map or replace values; use Replace values to transform them. allowEmpty controls missing values.
Targets
Flat File output (target_csv)
Writes rows as CSV or TXT using upstream output columns and their order. Configure header, delimiter, quoting, nulls, encoding and line endings.
Settings: format, fileName, delimiter, includeHeader, textQualifier, quoteMode, escapeCharacter, lineEnding, encoding, nullValue.
format: csv, txt; textQualifier: double, single, none; quoteMode: minimal, all; lineEnding: lf, crlf; encoding: utf-8, utf-8-bom.
Synthetic example:
{
"fileName": "output.csv",
"format": "csv",
"delimiter": ",",
"includeHeader": true
}Limits: It has no fields/columns selector. Use Field selector before the target to remove fields. fileName controls output naming, not projection.
JSON output (target_json)
Writes a JSON array or JSON Lines using only upstream output columns. includeNulls=false omits null-valued fields.
Settings: format, fileName, pretty, includeNulls, lineEnding, encoding.
format: json, jsonl; lineEnding: lf, crlf; encoding: utf-8, utf-8-bom.
Synthetic example:
{
"fileName": "output.jsonl",
"format": "jsonl",
"includeNulls": true
}Limits: It does not select arbitrary fields: use Field selector. pretty controls JSON array formatting.
Database output (target_database)
Writes upstream fields to a table through a compatible worker. append adds rows; replace and upsert depend on destination capabilities. keyColumns defines upsert keys.
Settings: connectionId, schema, objectName, writeMode, batchSize, commitIntervalRows, keyColumns, createIfMissing, queryTimeoutSeconds.
writeMode: append, replace, upsert.
Synthetic example:
{
"objectName": "customers",
"writeMode": "append",
"batchSize": 5000
}Limits: replace can replace existing data. Do not assume universal support for createIfMissing, write modes or query timeouts. Verify retries and keys before production.
API output (target_api)
Sends records with POST, PUT or PATCH. Supports append/upsert with success codes, batches, retries and timeout where supported by the REST runtime.
Settings: connectionId, method, resourcePath, writeMode, keyColumns, uniqueKeysEnforced, batchSize, successCodes, retries, timeoutSeconds.
method: POST, PUT, PATCH; writeMode: append, upsert.
Synthetic example:
{
"method": "POST",
"resourcePath": "/customers",
"writeMode": "append",
"batchSize": 100
}Limits: Upsert requires exactly one key. Idempotence depends on the API. uniqueKeysEnforced describes provider enforcement; it does not create a uniqueness constraint.
Object storage output (target_object)
Writes upstream fields as CSV/JSONL to object storage using conditional replacement. Requires idempotencyKey and exactly one precondition.
Settings: connectionId, resource, format, writeMode, delimiter, idempotencyKey, ifAbsent, expectedEtag, expectedGeneration.
format: csv, jsonl; writeMode: replace.
Synthetic example:
{
"resource": "output/customers.csv",
"format": "csv",
"writeMode": "replace",
"idempotencyKey": "example-run",
"ifAbsent": true
}Limits: Use ifAbsent, expectedEtag or expectedGeneration, never several. With ETag/generation set ifAbsent=false. There is no unconditional append mode; verify connection and permissions.
Execution compatibility
Generic streaming requires one linear path with a compatible source and target. Join and multi-input graphs need an execution path supporting that topology; do not change the algorithm merely to hide a compatibility error. Row sort, Group aggregate, Partition rank and quality gates may stage data before emitting results. Streaming does not mean unlimited memory or immediate output for every component.
Verify the result
Validate the graph, save and publish deliberately. Run a synthetic sample through authorized runtime and compare schema, rows, ordering and values. If CSV has extra fields after Field selector, check its saved settings, the Mapping revision used by the Session and downstream components: the selector does remove fields from both schema and rows.