////
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at
  http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
////
:documentationPath: /pipeline/transforms/
:language: en_US
:description: The Avro File Output transform can write Apache Avro messages to a file or field

= image:transforms/icons/avro_output.svg[Avro File Output Icon, role="image-doc-icon"] Avro File Output

[%noheader,cols="3a,1a", role="table-no-borders" ]
|===
|
== Description

The Avro File Output transform can write Apache Avro messages to a file or field
|
== Supported Engines
[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
!===
!Hop Engine! image:check_mark.svg[Supported, 24]
!Spark! image:check_mark.svg[Supported, 24]
!Flink! image:check_mark.svg[Supported, 24]
!Dataflow! image:check_mark.svg[Supported, 24]
!===
|===

== Options

[options="header"]
|===

|Option|Description

|Transform name
|Name of the transform.
Note: This name has to be unique in a single pipeline.

|Output Type
|The type of output you want, this can be binary file, binary message or a JSON message. When selecting message it will be sent to the Output Field.

|Filename
|The filepath where to save the avro data file.

|Output Field
|When not writing to a file this is the fieldname used to output the message to.

|Automatically create schema
|You can either select an existing schema or create a schema based on the fields in the fields tab.

|Write schema to file
|Select this to write the schema to a separate file.

|Avro namespace
|Namespace used in the Avro file.

|Avro record name
|Name used for the Avro record.

|Avro documentation
|

|Schema filename
|Filepath to write or read the schema from.

|Create parent folder
|When creating a file, select this if you want to create the parent folder.

|Compression codec
|Compression used for the avro messages when writing to a file.

|Include transform nr in filename
|Add the Transform instance number to the filename.

|Include partition nr in filename
|Add the Partition number to the filename.

|Include date in filename
|Include the date when the file was created in the filename

|Include time in filename
|Include the time when the file was created in the filename

|Specify date format
|Specify a custom date and format to attach to the filename

|Date time format
|Date time specification used for the formatting

|Add filenames to result
|Add the resulting filenames that are created to the filename results

|===

Fields tab:

[options="header"]
|===
|Option|Description

|Name
|The source field in the stream.

|Avro Schema Path
|The destination field, when an existing schema is specified this is populated with the fields from the schema to map the input fields to.

|Avro Type
|The Avro type of the field in the Avro destination file.

|Nullable
|Is the field nullable?

|===
== Metadata Injection Support

To inject the Avro field type use following codes

AVRO_TYPE_NONE = 0 +
AVRO_TYPE_BOOLEAN = 1 +
AVRO_TYPE_DOUBLE = 2 +
AVRO_TYPE_FLOAT = 3 +
AVRO_TYPE_INT = 4 +
AVRO_TYPE_LONG = 5 +
AVRO_TYPE_STRING = 6 +
AVRO_TYPE_ENUM = 7
