////
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at
  http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
////
:description: Transforms are the cornerstone of HOP, they are used to process and transforming your data, they are the building blocks that are used to create a pipeline.

[[transform-plugins]]
= Transform Plugins

Transforms are the cornerstone of HOP, they are used to process and transform your data, they are the building blocks that are used to create a pipeline.
The pages nested under this topic contain information on how to use the transforms and what you have to pay attention too when using them.

* xref:pipeline/transforms/abort.adoc[Abort]
* xref:pipeline/transforms/addchecksum.adoc[Add a checksum]
* xref:pipeline/transforms/addconstant.adoc[Add Constants]
* xref:pipeline/transforms/addsequence.adoc[Add Sequence]
* xref:pipeline/transforms/addfieldschangesequence.adoc[Add value fields changing sequence]
* xref:pipeline/transforms/addxml.adoc[Add XML]
* xref:pipeline/transforms/analyticquery.adoc[Analytic Query]
* xref:pipeline/transforms/apache-tika.adoc[Apache Tika]
* xref:pipeline/transforms/append.adoc[Append Streams]
* xref:pipeline/transforms/avro-decode.adoc[Avro Decode]
* xref:pipeline/transforms/avro-encode.adoc[Avro Encode]
* xref:pipeline/transforms/avro-file-input.adoc[Avro File Input]
* xref:pipeline/transforms/avro-file-output.adoc[Avro File Output]
* xref:pipeline/transforms/aws-sns-notify.adoc[AWS SNS Notify]
* xref:pipeline/transforms/aws-sqs-reader.adoc[AWS SQS Reader]
* xref:pipeline/transforms/azure-event-hubs-listener.adoc[Azure Event Hubs Listener]
* xref:pipeline/transforms/azure-event-hubs-writer.adoc[Azure Event Hubs Writer]
* xref:pipeline/transforms/beambigtableinput.adoc[Beam Bigtable Input]
* xref:pipeline/transforms/beambigtableoutput.adoc[Beam Bigtable Output]
* xref:pipeline/transforms/beambigqueryinput.adoc[Beam BigQuery Input]
* xref:pipeline/transforms/beambigqueryoutput.adoc[Beam BigQuery Output]
* xref:pipeline/transforms/beamgcppublisher.adoc[Beam GCP Pub/Sub : Publish]
* xref:pipeline/transforms/beamgcpsubscriber.adoc[Beam GCP Pub/Sub : Subscribe]
* xref:pipeline/transforms/beamfileinput.adoc[Beam File Input]
* xref:pipeline/transforms/beamfileoutput.adoc[Beam File Output]
* xref:pipeline/transforms/beamkafkaconsume.adoc[Beam Kafka Consume]
* xref:pipeline/transforms/beamkafkaproduce.adoc[Beam Kafka Produce]
* xref:pipeline/transforms/beamkinesisconsume.adoc[Beam Kinesis Consume]
* xref:pipeline/transforms/beamkinesisproduce.adoc[Beam Kinesis Produce]
* xref:pipeline/transforms/beamtimestamp.adoc[Beam Timestamp]
* xref:pipeline/transforms/beamwindow.adoc[Beam Window]
* xref:pipeline/transforms/blockingtransform.adoc[Blocking transform]
* xref:pipeline/transforms/blockuntiltransformsfinish.adoc[Blocking until transforms finish]
* xref:pipeline/transforms/calculator.adoc[Calculator]
* xref:pipeline/transforms/cassandra-input.adoc[Cassandra Input]
* xref:pipeline/transforms/cassandra-output.adoc[Cassandra Output]
* xref:pipeline/transforms/sstable-output.adoc[Cassandra SSTable Output]
* xref:pipeline/transforms/calldbproc.adoc[Call DB procedure]
* xref:pipeline/transforms/changefileencoding.adoc[Change file encoding]
* xref:pipeline/transforms/checkfilelocked.adoc[Check if file is locked]
* xref:pipeline/transforms/checkwebserviceavailable.adoc[Check if webservice is available]
* xref:pipeline/transforms/clonerow.adoc[Clone row]
* xref:pipeline/transforms/closure.adoc[Closure]
* xref:pipeline/transforms/coalesce.adoc[Coalesce]
* xref:pipeline/transforms/columnexists.adoc[Column exists]
* xref:pipeline/transforms/combinationlookup.adoc[Combination lookup/update]
* xref:pipeline/transforms/concatfields.adoc[Concat Fields]
* xref:pipeline/transforms/copyrowstoresult.adoc[Copy rows to result]
* xref:pipeline/transforms/creditcardvalidator.adoc[Credit card validator]
* xref:pipeline/transforms/csvinput.adoc[CSV File Input]
* xref:pipeline/transforms/databasejoin.adoc[Database Join]
* xref:pipeline/transforms/databaselookup.adoc[Database Lookup]
* xref:pipeline/transforms/datagrid.adoc[Data Grid]
* xref:pipeline/transforms/delay.adoc[Delay row]
* xref:pipeline/transforms/delete.adoc[Delete]
* xref:pipeline/transforms/serialize-de-from-file.adoc[De-Serialize From File]
* xref:pipeline/transforms/detectemptystream.adoc[Detect Empty Stream]
* xref:pipeline/transforms/detectlanguage.adoc[Detect Language]
* xref:pipeline/transforms/dimensionlookup.adoc[Dimension lookup/update]
* xref:pipeline/transforms/dorisbulkloader.adoc[Doris Bulk Loader]
* xref:pipeline/transforms/rulesaccumulator.adoc[Drools Rule Accumulator]
* xref:pipeline/transforms/rulesexecutor.adoc[Drools Rule Executor]
* xref:pipeline/transforms/dummy.adoc[Dummy (do nothing)]
* xref:pipeline/transforms/dynamicsqlrow.adoc[Dynamic SQL row]
* xref:pipeline/transforms/edi2xml.adoc[Edi to XML]
* xref:pipeline/transforms/emailinput.adoc[Email Messages Input]
* xref:pipeline/transforms/enhancedjsonoutput.adoc[Enhanced JSON Output]
* xref:pipeline/transforms/excelinput.adoc[Excel input]
* xref:pipeline/transforms/excelwriter.adoc[Excel writer]
* xref:pipeline/transforms/execprocess.adoc[Execute a process]
* xref:pipeline/transforms/execsqlrow.adoc[Execute row SQL script]
* xref:pipeline/transforms/execsql.adoc[Execute SQL script]
* xref:pipeline/transforms/exectests.adoc[Execute Unit Tests]
* xref:pipeline/transforms/execinfo.adoc[Execution Information]
* xref:pipeline/transforms/fake.adoc[Fake data]
* xref:pipeline/transforms/fileexists.adoc[File exists]
* xref:pipeline/transforms/filemetadata.adoc[File Metadata]
* xref:pipeline/transforms/filesfromresult.adoc[Files from result]
* xref:pipeline/transforms/filestoresult.adoc[Files to result]
* xref:pipeline/transforms/filterrows.adoc[Filter Rows]
* xref:pipeline/transforms/formula.adoc[Formula]
* xref:pipeline/transforms/fuzzymatch.adoc[Fuzzy match]
* xref:pipeline/transforms/generaterandomvalue.adoc[Generate Random Value]
* xref:pipeline/transforms/getdatafromxml.adoc[Get Data From XML]
* xref:pipeline/transforms/getfilenames.adoc[Get filenames]
* xref:pipeline/transforms/getfilesrowcount.adoc[Get files rowcount]
* xref:pipeline/transforms/getrecordsfromstream.adoc[Get records from stream]
* xref:pipeline/transforms/getrowsfromresult.adoc[Get Rows from Result]
* xref:pipeline/transforms/getsubfolders.adoc[Get SubFolder names]
* xref:pipeline/transforms/getsystemdata.adoc[Get System Info]
* xref:pipeline/transforms/gettablenames.adoc[Get table names]
* xref:pipeline/transforms/getvariable.adoc[Get variables]
* xref:pipeline/transforms/google-analytics.adoc[Google Analytics Input]
* xref:pipeline/transforms/google-sheets-input.adoc[Google Sheets Input]
* xref:pipeline/transforms/google-sheets-output.adoc[Google Sheets Output]
* xref:pipeline/transforms/groupby.adoc[Group By]
* xref:pipeline/transforms/html2text.adoc[HTML 2 Text]
* xref:pipeline/transforms/http.adoc[HTTP client]
* xref:pipeline/transforms/httppost.adoc[HTTP Post]
* xref:pipeline/transforms/identifylastrow.adoc[Identify last row in a stream]
* xref:pipeline/transforms/ifnull.adoc[If Null]
* xref:pipeline/transforms/injector.adoc[Injector]
* xref:pipeline/transforms/insertupdate.adoc[Insert / Update]
* xref:pipeline/transforms/javafilter.adoc[Java Filter]
* xref:pipeline/transforms/javascript.adoc[JavaScript]
* xref:pipeline/transforms/jdbcmetadata.adoc[JDBC Metadata]
* xref:pipeline/transforms/joinrows.adoc[Join Rows]
* xref:pipeline/transforms/jsoninput.adoc[JSON Input]
* xref:pipeline/transforms/jsonoutput.adoc[JSON Output]
* xref:pipeline/transforms/kafkaconsumer.adoc[Kafka Consumer]
* xref:pipeline/transforms/kafkaproducer.adoc[Kafka Producer]
* xref:pipeline/transforms/languagemodelchat.adoc[Language Model Chat]
* xref:pipeline/transforms/ldapinput.adoc[LDAP Input]
* xref:pipeline/transforms/ldapoutput.adoc[LDAP Output]
* xref:pipeline/transforms/loadfileinput.adoc[Load file content in memory]
* xref:pipeline/transforms/mail.adoc[Mail]
* xref:pipeline/transforms/mapping-input.adoc[Mapping Input]
* xref:pipeline/transforms/mapping-output.adoc[Mapping Output]
* xref:pipeline/transforms/memgroupby.adoc[Memory Group By]
* xref:pipeline/transforms/mergejoin.adoc[Merge Join]
* xref:pipeline/transforms/mergerows.adoc[Merge rows (diff)]
* xref:pipeline/transforms/metainject.adoc[Metadata Injection]
* xref:pipeline/transforms/metadata-input.adoc[Metadata Input]
* xref:pipeline/transforms/metastructure.adoc[Metadata structure of stream]
* xref:pipeline/transforms/accessoutput.adoc[Microsoft Access Output]
* xref:pipeline/transforms/monetdbbulkloader.adoc[MonetDB Bulk Loader]
* xref:pipeline/transforms/mongodbdelete.adoc[MongoDB Delete]
* xref:pipeline/transforms/mongodbinput.adoc[MongoDB Input]
* xref:pipeline/transforms/mongodboutput.adoc[MongoDB Output]
* xref:pipeline/transforms/multimerge.adoc[Multiway Merge Join]
* xref:pipeline/transforms/mysqlbulkloader.adoc[MySql Bulk Loader]
* xref:pipeline/transforms/neo4j-cypher.adoc[Neo4j Cypher]
* xref:pipeline/transforms/neo4j-gencsv.adoc[Neo4j Generate CSVs]
* xref:pipeline/transforms/neo4j-getloginfo.adoc[Neo4j Get Logging Info]
* xref:pipeline/transforms/neo4j-graphoutput.adoc[Neo4j Graph Output]
* xref:pipeline/transforms/neo4j-import.adoc[Neo4j Import]
* xref:pipeline/transforms/neo4j-output.adoc[Neo4j Output]
* xref:pipeline/transforms/neo4j-split-graph.adoc[Neo4j Split Graph]
* xref:pipeline/transforms/nullif.adoc[Null If]
* xref:pipeline/transforms/numberrange.adoc[Number range]
* xref:pipeline/transforms/orabulkloader.adoc[Oracle Bulk Loader]
* xref:pipeline/transforms/parquet-file-input.adoc[Parquet File Input]
* xref:pipeline/transforms/parquet-file-output.adoc[Parquet File Output]
* xref:pipeline/transforms/pipeline-data-probe.adoc[Pipeline Data Probe]
* xref:pipeline/transforms/pipeline-logging.adoc[Pipeline Logging]
* xref:pipeline/transforms/pgpdecryptstream.adoc[PGP decrypt stream]
* xref:pipeline/transforms/pgpencryptstream.adoc[PGP encrypt stream]
* xref:pipeline/transforms/pipeline-executor.adoc[Pipeline Executor]
* xref:pipeline/transforms/postgresbulkloader.adoc[PostgreSQL Bulk Loader]
* xref:pipeline/transforms/processfiles.adoc[Process files]
* xref:pipeline/transforms/propertyinput.adoc[Properties file Input]
* xref:pipeline/transforms/propertyoutput.adoc[Properties file Output]
* xref:pipeline/transforms/redshift-bulkloader.adoc[Redshift Bulk Loader]
* xref:pipeline/transforms/regexeval.adoc[Regex Evaluation]
* xref:pipeline/transforms/replacestring.adoc[Replace in String]
* xref:pipeline/transforms/reservoirsampling.adoc[Reservoir Sampling]
* xref:pipeline/transforms/rest.adoc[REST Client]
* xref:pipeline/transforms/rowdenormaliser.adoc[Row Denormaliser]
* xref:pipeline/transforms/rowflattener.adoc[Row Flattener]
* xref:pipeline/transforms/rowgenerator.adoc[Row Generator]
* xref:pipeline/transforms/rownormaliser.adoc[Row Normaliser]
* xref:pipeline/transforms/runssh.adoc[Run SSH commands]
* xref:pipeline/transforms/salesforcedelete.adoc[Salesforce Delete]
* xref:pipeline/transforms/salesforceinput.adoc[Salesforce Input]
* xref:pipeline/transforms/salesforceinsert.adoc[Salesforce Insert]
* xref:pipeline/transforms/salesforceupdate.adoc[Salesforce Update]
* xref:pipeline/transforms/salesforceupsert.adoc[Salesforce Upsert]
* xref:pipeline/transforms/samplerows.adoc[Sample Rows]
* xref:pipeline/transforms/sasinput.adoc[SAS Input]
* xref:pipeline/transforms/script.adoc[Script]
* xref:pipeline/transforms/selectvalues.adoc[Select Values]
* xref:pipeline/transforms/serialize-to-file.adoc[Serialize To File]
* xref:pipeline/transforms/serverstatus.adoc[Server Status]
* xref:pipeline/transforms/setvalueconstant.adoc[Set field value to a constant]
* xref:pipeline/transforms/setvaluefield.adoc[Set field Value to a field]
* xref:pipeline/transforms/setvariable.adoc[Set Variables]
* xref:pipeline/transforms/simple-mapping.adoc[Simple Mapping]
* xref:pipeline/transforms/snowflakebulkloader.adoc[Snowflake Bulk Loader]
* xref:pipeline/transforms/sort.adoc[Sort Rows]
* xref:pipeline/transforms/sortedmerge.adoc[Sorted Merge]
* xref:pipeline/transforms/splitfields.adoc[Split Fields]
* xref:pipeline/transforms/splitfieldtorows.adoc[Split fields to rows]
* xref:pipeline/transforms/splunkinput.adoc[Splunk Input]
* xref:pipeline/transforms/sqlfileoutput.adoc[SQL File Output]
* xref:pipeline/transforms/standardizephonenumber.adoc[Standardize Phone Number]
* xref:pipeline/transforms/stanfordnlp.adoc[Stanford NLP]
* xref:pipeline/transforms/schemamapping.adoc[Static Schema Mapping]
* xref:pipeline/transforms/streamlookup.adoc[Stream Lookup]
* xref:pipeline/transforms/streamschemamerge.adoc[Stream Schema Merge]
* xref:pipeline/transforms/stringcut.adoc[String cut]
* xref:pipeline/transforms/stringoperations.adoc[String operations]
* xref:pipeline/transforms/switchcase.adoc[Switch / Case]
* xref:pipeline/transforms/synchronizeaftermerge.adoc[Synchronize after merge]
* xref:pipeline/transforms/tablecompare.adoc[Table Compare]
* xref:pipeline/transforms/tableexists.adoc[Table Exists]
* xref:pipeline/transforms/tableinput.adoc[Table Input]
* xref:pipeline/transforms/tableoutput.adoc[Table Output]
* xref:pipeline/transforms/terafast.adoc[Teradata Bulk Loader]
* xref:pipeline/transforms/textfileinput.adoc[Text File Input]
* xref:pipeline/transforms/textfileoutput.adoc[Text File Output]
* xref:pipeline/transforms/tokenreplacement.adoc[Token Replacement]
* xref:pipeline/transforms/uniquerows.adoc[Unique Rows]
* xref:pipeline/transforms/uniquerowsbyhashset.adoc[Unique Rows (HashSet)]
* xref:pipeline/transforms/update.adoc[Update]
* xref:pipeline/transforms/userdefinedjavaclass.adoc[User Defined Java Class]
* xref:pipeline/transforms/userdefinedjavaexpression.adoc[User Defined Java Expression]
* xref:pipeline/transforms/validator.adoc[Data Validator]
* xref:pipeline/transforms/valuemapper.adoc[Value Mapper]
* xref:pipeline/transforms/verticabulkloader.adoc[Vertica Bulk Loader]
* xref:pipeline/transforms/webservices.adoc[Web services lookup]
* xref:pipeline/transforms/workflow-executor.adoc[Workflow Executor]
* xref:pipeline/transforms/workflow-logging.adoc[Workflow Logging]
* xref:pipeline/transforms/writetolog.adoc[Write to log]
* xref:pipeline/transforms/xmlinputstream.adoc[XML Input Stream (StAX)]
* xref:pipeline/transforms/xmljoin.adoc[XML Join]
* xref:pipeline/transforms/xmloutput.adoc[XML Output]
* xref:pipeline/transforms/xsdvalidator.adoc[XSD Validator]
* xref:pipeline/transforms/xslt.adoc[XSL Transformation]
* xref:pipeline/transforms/yamlinput.adoc[Yaml Input]
* xref:pipeline/transforms/zipfile.adoc[Zip file]