////
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at
  http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
////
:documentationPath: /pipeline/transforms/
:language: en_US
:description: The Microsoft Excel Writer transform writes incoming rows from Hop out to an MS Excel file. It supports both the .xls and .xlsx file formats.

= image:transforms/icons/excelwriter.svg[Excel writer transform Icon, role="image-doc-icon"] Excel writer

[%noheader,cols="3a,1a", role="table-no-borders" ]
|===
|
== Description

The Microsoft Excel Writer transform writes incoming rows from Hop out to an MS Excel file. It supports both the .xls and .xlsx file formats.

The .xls files use a binary format which is better suited for simple content, while the .xlsx files use the Open XML format which works well with templates since it can better preserve charts and miscellaneous objects.

|
== Supported Engines
[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
!===
!Hop Engine! image:check_mark.svg[Supported, 24]
!Spark! image:check_mark.svg[Supported, 24]
!Flink! image:check_mark.svg[Supported, 24]
!Dataflow! image:check_mark.svg[Supported, 24]
!===
|===

== Options

=== File & sheet tab

*File section*

[options="header"]
|===
|Option|Description
|Stream XLSX data|Check this option when writing large XLSX files.
It uses internally a streaming API and is able to write large files without any memory restrictions (of course not exceeding Excel's limit of 1,048,575 rows and 16,384 columns).
|Create parent folder|Enable to create the parent folder
|If output file exists|Check this option when writing large XLSX files.
It uses internally a streaming API and is able to write large files without any memory restrictions (of course not exceeding Excel's limit of 1,048,575 rows and 16,384 columns).
|Add filename(s) to result|Check to have the filename added to the result filenames
|Wait for first row before creating file|Checking this option makes the transform create the file only after it has seen a row.
If this is disabled the output file is always created, regardless of whether rows are actually written to the file.
|===

*Sheet section*

[options="header"]
|===
|Option|Description
|Sheet Name|The sheet name the transform will write rows to.
|Make this the active sheet|If checked the Excel file will by default open on the above sheet when opened in MS Excel.
|If sheet exists in output file|The output file already has this sheet (for example when using a template, or writing to existing files), you can choose to write to the existing sheet, or replace it.
|Protect Sheet|The XLS file format allows to protect an entire sheet from changes.
If checked you need to provide a password.
Excel will indicate that the sheet was protected by the user you provide here.
|===

*Template section*

When creating new files (when existing files are replaced, or completely fresh files are created) you may choose to create a copy of an existing template file instead.
Please make sure that the template file is of the same type as the output file (both must be xls or xlsx respectively).

When creating new sheets, the transform may copy a sheet from the current document (the template or an otherwise existing file the transform is writing to).
A new sheet is created if the target sheet is not present, or the existing one shall be replaced as per configuration above.

=== Content tab

*Content options section*

[options="header"]
|===
|Option|Description
|Start writing at cell|This is the cell to start writing to in Excel notation (letter column, number row)
|When writing rows|The transform may overwrite existing cells (fast), or shift existing cells down (append new rows at the top of sheet)
|Write Header|If checked the first line written will contain the field names
|Write Footer|If checked the last line written will contains the field names
|Auto Size Columns|If checked the transform tries to automatically size the columns to fit their content.
Since this is not a feature the xls(x) file formats support directly, results may vary.
|Force formula recalculation a|If checked, the transform tries to make sure all formula fields in the output file are updated.

* The xls file format supports a "dirty" flag that the transform sets.
The formulas are recalculated as soon as the file is opened in MS Excel.
* For the xlsx file format, the transform must try to recalculate the formula fields itself.
Since the underlying POI library does not support the full set of Excel formulas yet, this may give errors.
The transform will throw errors if it cannot recalculate the formulas.
|Leave styles of existing cells unchanged|If checked, the transform will not try to set the style of existing cells it is writing to.
This is useful when writing to pre-styled template sheets.
|===

*When writing to existing sheet section*

[options="header"]
|===
|Option|Description
|Start writing at end of sheet|The transform will try to find the last line of the sheet, and start writing from there.
|Offset by ... rows|Any non-0 number will cause the transform to move this amount of rows down (positive numbers) or up (negative numbers) before writing rows.
Negative numbers may be useful if you need to append to a sheet, but still preserve a pre-styled footer.
|Begin by writing ... empty lines|The transform will try to find the last line of the sheet, and start writing from there.
|Omit Header|Any non-0 number will cause the transform to move this amount of rows down (positive numbers) or up (negative numbers) before writing rows.
Negative numbers may be useful if you need to append to a sheet, but still preserve a pre-styled footer.
|===

*Fields section*

The fields section is for specifying the fields that must be written to the Excel file. You can choose to use a xref:metadata-types/static-schema-definition.adoc[Schema Definition] or to define the required fields' layout manually.

If you decide to define the fields layout by using a xref:metadata-types/static-schema-definition.adoc[Schema Definition], use the xref:pipeline/transforms/schemamapping.adoc[Schema Mapping] transform to adjust the incoming stream according to the choosen xref:metadata-types/static-schema-definition.adoc[Schema Definition]

[options="header"]
|===
|Option|Description
|Schema Definition|Name of the xref:metadata-types/static-schema-definition.adoc[Schema Definition] that we want to reference.
|Name|The field to write
|Type|The type of data
|Format|The Excel format to use in the sheet.
Please consult the Excel manual for valid formats.
There are some online references as well.
|Style from cell|A cell (i.e. A1, B3 etc.) to copy the styling from for this column (usually some pre-styled cell in a template)
|Field Title|If set, this is used for the Header/Footer instead of the Hop field name
|Header/Footer style from cell|A cell to copy the styling from for headers/footers (usually some pre-styled cell in a template)
|Field Contains Formula|Set to Yes, if the field contains an Excel formula (no leading '=')
|Hyperlink|A field, that contains the target to link to.
The supported targets are Link to other cells, http, ftp, email, and local documents
|Cell Comment / Cell Author|The xlsx format allows to put comments on cells.
If you'd like to generate comments, you may specify fields holding the comment and author for a given column.
|===
