////
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at
  http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
////
[[HopServerRestApi]]
:imagesdir: ../assets/images
:description: Hop Server has a rich set of web services that can be used to query and manage the server's status, or to manage pipeline and workflow execution details.

= Hop Server Web Services Overview

Hop Server has a rich set of web services that can be used to query and manage the server's status, or to manage pipeline and workflow execution details.

TIP: when specified, the id for a workflow or pipeline represents one execution of a workflow or pipeline on the server.

== addExport

name::
addExport

description::
Upload a resources export file.
Add a zipped pipeline or workflow to the body payload as a binary file.

endPoint::
GET `hop/addExport`

parameters::
* type: `pipeline` or `workflow`
* load: -

example request::
`+http://localhost:8081/hop/addExport/?type=workflow+` with zipped workflow as payload

result::
A zip file with the export is created on the server's file system.

[source,xml]
----
<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message>file:///tmp/export_70eb8ef1-9721-4cf5-afa3-940cd0f771d9.zip</message>
    <id/>
</webresult>
----

== addPipeline

name::
addPipeline

description::
Add a pipeline for execution

endPoint::
GET `hop/addPipeline`
* Content-Type: text/xml;charset=UTF-8

parameters::
* xml Request body should contain xml containing pipeline_configuration (pipeline and pipeline_execution_configuration wrapped in pipeline_configuration tag).

example request::
`+http://localhost:8081/hop/addPipeline/xml=Y+` with XML payload

result::
- 

== addWorkflow

name::
addWorkflow

description::
Add a workflow for execution

endPoint::
GET `hop/addWorkflow`
* Content-Type: text/xml;charset=UTF-8

parameters::
* xml Request body should contain xml containing workflow_configuration (pipeline and workflow_execution_configuration wrapped in pipeline_configuration tag).

example request::
`+http://localhost:8081/hop/addWorkflow/xml=Y+` with XML payload

result::
-

== getPipelineImage

name::
getPipelineImage

description::
Generate a SVG image of a pipeline

endPoint::
GET `hop/pipelineImage`

parameters::
* name: name of the pipeline to generate the image for
* id: id of the pipeline to generate the image for

example request::
GET `+http://localhost:8081/hop/pipelineImage/?name=remote-pipeline&id=c1451bfb-b867-4c76-b123-c29d2b05da17+`

result::
an SVG image of the pipeline graph

== getPipelineStatus

name::
getPipelineStatus

description::
Get the status of a pipeline

endPoint::
GET `hop/pipelineStatus`

parameters::
* name: name of the pipeline to get the status for
* id: id of the pipeline to get the status for
* xml (optional): return the information as xml (default HTML) use &xml=Y
* json (optional): return the information as json (default HTML) use &json=Y

example request::
GET `+http://localhost:8081/hop/pipelineStatus/?name=<NAME>>&id=<ID>+`

result::
an HTML response with the execution status, transform details and canvas preview for this pipeline

== Status

name::
status

description::
Get the status of the server

parameters::
none

example request::
GET `+http://localhost:8081/hop/status/+`

result::
an HTML page with an overview of the pipelines and workflows on the server, their execution details and the server's configuration details.

== getWorkflowImage

name::
getWorkflowImage

description::
Generate an SVG image of a workflow

endPoint::
GET `hop/workflowImage`

parameters::
* name: name of the workflow to generate the image for
* id: id of the workflow to generate the image for

example request::
GET `+http://localhost:8081/hop/workflowImage/?name=<NAME>>&id=<ID>+`

result::
an SVG image of the workflow graph

[#_getworkflowstatus]
== getWorkflowStatus

name::
getWorkflowStatus

description::
Get the status of a workflow

endPoint::
GET `hop/workflowStatus`

parameters::
* name: name of the workflow to get the status for
* id: id of the workflow to get the status for
* xml (optional): return the information as xml (default HTML) use &xml=Y
* json (optional): return the information as json (default HTML) use &json=Y

example request::
GET `+http://localhost:8081/hop/workflowStatus/?name=<NAME>&id=<ID>+`

result::
an HTML response with the execution status, action details and canvas preview for this workflow

== pausePipeline

name::
pausePipeline

description::
Pause or continue a pipeline

endPoint::
GET `/hop/pausePipeline`

parameters::
* name: name of the pipeline to pause or restart
* id: id of the pipeline to pause or restart

example request::
GET `+http://localhost:8081/hop/pausePipeline/?name=<NAME>&id=<ID>+`

result::
HTML page with the request status, e.g.

[source,html]
----
<HTML>

<HEAD>
	<TITLE>Pause pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/pipelineStatus?name=<NAME>&id=<ID>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline [tmp] : pause requested.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=<ID>">Back to the pipeline status page</a>
	<p>
		<p>
</BODY>

</HTML>
----

== Prepare Execution

name::
prepareExec

description::
Prepare the execution of a pipeline

endPoint::
GET `/hop/prepareExec`

parameters::
* xml: use xml, default Y
* name: the name of the pipeline to prepare execution for
* id: the id of the pipeline to prepare execution for

example request::
GET `+http://localhost:8081/hop/prepareExec/?xml=Y&name=<NAME>&id=<ID>+`

result::

Example result:

[source,html]
----
<?xml version="1.0" encoding="UTF-8"?>
<webresult>
    <result>OK</result>
    <message/>
    <id/>
</webresult>
----


//=== Register Package
//
//name::
//registerPackage
//
//description::
//Upload a resources export file
//
//endPoint::
//GET `/hop/registerPackage`
//
//parameters::
//* load
//* type
//
//example request::
//-
//
//result::
//-


== Register Pipeline

name::
registerPipeline

description::
Register a pipeline for execution

endPoint::
GET `hop/registerPipeline`
* Content-Type: text/xml;charset=UTF-8

parameters::
* xml Request body should contain xml containing pipeline_configuration (pipeline and pipeline_execution_configuration wrapped in pipeline_configuration tag).

example request::
`+http://localhost:8081/hop/registerPipeline/xml=Y+`

with XML payload (example):
[source,xml]
----
<pipeline_configuration>
<pipeline>
  <info>
    <name>generate_rows</name>
    <name_sync_with_filename>Y</name_sync_with_filename>
    <description/>
    <extended_description/>
    <pipeline_version/>
    <pipeline_type>Normal</pipeline_type>
    <parameters>
    </parameters>
    <capture_transform_performance>N</capture_transform_performance>
    <transform_performance_capturing_delay>1000</transform_performance_capturing_delay>
    <transform_performance_capturing_size_limit>100</transform_performance_capturing_size_limit>
    <created_user>-</created_user>
    <created_date>2022/02/03 13:47:49.645</created_date>
    <modified_user>-</modified_user>
    <modified_date>2022/02/03 13:47:49.645</modified_date>
    <key_for_session_key>H4sIAAAAAAAA/wMAAAAAAAAAAAA=</key_for_session_key>
    <is_key_private>N</is_key_private>
  </info>
  <notepads>
  </notepads>
  <order>
    <hop>
      <from>Generate rows</from>
      <to>Delay row</to>
      <enabled>Y</enabled>
    </hop>
    <hop>
      <from>Delay row</from>
      <to>result</to>
      <enabled>Y</enabled>
    </hop>
  </order>
  <transform>
    <name>Delay row</name>
    <type>Delay</type>
    <description/>
    <distribute>Y</distribute>
    <custom_distribution/>
    <copies>1</copies>
    <partitioning>
      <method>none</method>
      <schema_name/>
    </partitioning>
    <scaletime>seconds</scaletime>
    <timeout>1</timeout>
    <attributes/>
    <GUI>
      <xloc>416</xloc>
      <yloc>96</yloc>
    </GUI>
  </transform>
  <transform>
    <name>Generate rows</name>
    <type>RowGenerator</type>
    <description/>
    <distribute>Y</distribute>
    <custom_distribution/>
    <copies>1</copies>
    <partitioning>
      <method>none</method>
      <schema_name/>
    </partitioning>
    <fields>
      <field>
        <length>-1</length>
        <name>value</name>
        <precision>-1</precision>
        <set_empty_string>N</set_empty_string>
        <type>String</type>
        <nullif>test</nullif>
      </field>
    </fields>
    <interval_in_ms>5000</interval_in_ms>
    <last_time_field>FiveSecondsAgo</last_time_field>
    <never_ending>N</never_ending>
    <limit>10000</limit>
    <row_time_field>now</row_time_field>
    <attributes/>
    <GUI>
      <xloc>160</xloc>
      <yloc>96</yloc>
    </GUI>
  </transform>
  <transform>
    <name>result</name>
    <type>Dummy</type>
    <description/>
    <distribute>Y</distribute>
    <custom_distribution/>
    <copies>1</copies>
    <partitioning>
      <method>none</method>
      <schema_name/>
    </partitioning>
    <attributes/>
    <GUI>
      <xloc>720</xloc>
      <yloc>96</yloc>
    </GUI>
  </transform>
  <transform_error_handling>
  </transform_error_handling>
  <attributes/>
</pipeline>
  <pipeline_execution_configuration>
    <pass_export>N</pass_export>
    <parameters>
    </parameters>
    <variables>
    <variable><name>HOP_AUDIT_FOLDER</name><value>/Users/hans/config/audit</value></variable>
    <variable><name>HOP_AUTO_CREATE_CONFIG</name><value>Y</value></variable>
    <variable><name>HOP_CONFIG_FOLDER</name><value>/Users/hans/config</value></variable>
    <variable><name>HOP_DATASETS_FOLDER</name><value>/Users/hans/test/datasets</value></variable>
    <variable><name>HOP_ENVIRONMENT_NAME</name><value>test1</value></variable>
    <variable><name>HOP_METADATA_FOLDER</name><value>/Users/hans/test/metadata</value></variable>
    <variable><name>HOP_PIPELINE_PAN_JVM_EXIT_CODE</name><value/></variable>
    <variable><name>HOP_PROJECTS</name><value>/Users/hans/tmp/</value></variable>
    <variable><name>HOP_PROJECT_NAME</name><value>test</value></variable>
    <variable><name>HOP_UNIT_TESTS_FOLDER</name><value>/Users/hans/test</value></variable>
    <variable><name>NEO4J_CONNECTION</name><value>neo4j</value></variable>
    <variable><name>PROJECT_HOME</name><value>/Users/hans/test</value></variable>
    <variable><name>TEST</name><value>TEST</value></variable>
    <variable><name>ftp.nonProxyHosts</name><value>local|*.local|169.254/16|*.169.254/16</value></variable>
    <variable><name>http.nonProxyHosts</name><value>local|*.local|169.254/16|*.169.254/16</value></variable>
    <variable><name>jdk.debug</name><value>release</value></variable>
    <variable><name>native.encoding</name><value>UTF-8</value></variable>
    <variable><name>p1</name><value>a</value></variable>
    <variable><name>socksNonProxyHosts</name><value>local|*.local|169.254/16|*.169.254/16</value></variable>
    </variables>
    <log_level>Basic</log_level>
    <log_file>N</log_file>
    <log_filename/>
    <log_file_append>N</log_file_append>
    <create_parent_folder>N</create_parent_folder>
    <clear_log>Y</clear_log>
    <show_subcomponents>Y</show_subcomponents>
    <run_configuration>local</run_configuration>
</pipeline_execution_configuration>
<metastore_json>H4sIAAAAAAAA/+1XbVPbOBD+Kxl/upshzQsUAt/SxBzchSRNTDudpuNRZNlRkSVXkgk5hv9+K0t+
CSl3cDP37coH8Gq1evTs7rPqo6eIvCfSu/j66G3JephlU5QS78Lzjjyl2EjwmCbeBc8ZKww3IoLV
GDFFjrxMiofdXEht/TdCaW539/pn77rw0wNzhpTaChmB2edY7jJNolZ/Tc4HKMaDU4TO4j45wese
Qv3T9zgyJjyAjVzwuTnhCuIqe4QArJJGxH+gSlOewHpGpKZENUE50w1Smkh7H3uBzGId9AYGmMPK
ydZzd7mqbwCmHLhxX5jlJpb39A08aUYY5aQNW9aw+BVsOae6rYnS9lNG61QVnLq/Hr3ldPb5cjL8
wzcfEdJojRSpuc5YnlB+bUiqPY88yiPyEKA1IypDuLrHFkmyEYDPbjbhDp0QxkSpYJeBpbuXnZeS
4lUMVYgcwiUX25ihO2LKoqgY3qAVaS3pOtfE3vR2Pp8tgmUYXN/4y2B4Mw/Hw2AYBl/mcHnvC4T4
eDsL/HA4mYSX1/5kvATzFMzVzg+z2cQfTg/2Xc4WIz+8HvvT4Bp2LuCMWTiZffYXo+HSd1HmC3/p
Lz75ofs9Dj/PFuPQeZgwy4+TcDSbTv1RYO/607gA5nncS3+xgHjL0ZV/Mwynwxuz5j0deSniOWK3
kh1Ujvf0VFVaQjiRFPdhR6My5rNl8Btg/Tj5SWlkkLVEArF7JdLY8mKNvLUomMCIme83tOzZOSa9
bnR6fox6+Lwun/cnx/3nJTS3N7GY/6+ht9TQBnHVrKMsKYQIMnQXM7FtM5FY3cGQN8QjidpYcE6w
poLbFU7Eyfd2IlG2aaeg4cyaQfLbJhfUVAjIlZZwVCxk6pI2ztN01/olEi0u9AYE91dAFlNGHJJV
5xa2q1XHQFx1jAKuOqbyljbou03Gih2ERS7kPWK5kRFmFXypkc5r9XZxFUozRkKAV6KDywmuCde2
dj2UZYxiZG646nxXcM8jD1BBvUPVapmTPamWOTeUwCzLJXKsPII/lCdZ5Hy0v/TofSAoHVMJDM5d
CL/wNWuapNlEYOdbsHGx6pifeyRXnViwqKDkvgu0fE9P77Ik6nbPTjmO7rJ4IJPjaNuFfwkAD1Yd
0w1awnlAxpXIgjID6kMex0Qu6Z9OsnmerokMRRyaxMMR1hwj/TuSdc0MQWVMD0KsF0NfwjjbXAOd
ErJht1aZnxdNO2JQSkQ1R4EKBCQrcWgesgNHU6F7JH9CkhYK5GrQ5rYYk6BPRGFJM8diIYkv52NB
UqEL9smD0ZgQJFHkEpvQJtkwcktDqJFMiA5tHizYetH+3lu0WhRmgrGQ7nECVRNi9wKy+li8crKw
fDS5t0MzArQW2v0jG49VK/cOqThyPQJFXvS5c5WWgzcyNylgwx8xIdEa4btQ2Yp6b2rQgLet5qy9
pk1DqwElYZJTWJqgIm9SbBXQW/u7HTEJ0+bjUG3ENiwPraxaZAKSYKaEsyRIb4C8lID+41oIinpE
hYIZ/IX1lQVWZmqfJyBHtXaQ/lapCqpVeLJda0v1pgUwWkqDfCIZ2ZUWNE3l3bIc24wUarqnsY/V
S3pvkq4LuqR7JJ+dnJ2491pjxhfRvMYAcNdBuRYp3BOX1yxnZa6god1gLk53nEmRG0mtn8KvHuPr
qH8cRYNujAupfXZAfb5XwHwI6uQsIG+7gJbvypqTCb2H545Sow3Bd8YBwD33ecE8xD9yquiBB3S6
0kPGRuZ9Hxv1J+oA26iBICa6QubI2fevKH7O4VzAdNlZp7Vgukzg6eDsBSBVBCgCBaeflMrEn2e5
AXEuBKsVvjHyS+8yWBO2qcBU8ERE64M5nyFAVH8iteO4vT/lm3OxejlUb4m3TspKXw4V4KUWbrZp
/2/6tMT0uj4tvas+fZ0s/jvY/x3qb/Z91Y5ITDltJAAzoXJZvgWa/3U9GB/Fc2tcBXByoggUB9JC
ljWkYK7zO7toJAlk3Xw8/QVlC+SCFRAAAA==
</metastore_json>
</pipeline_configuration>
----

The metastore_json is base64 encoded GZip content


result::
-

== Register Workflow

name::
registerWorkflow

description::
Register a workflow on the server

endPoint::
GET `/hop/registerWorkflow`
* Content-Type: text/xml;charset=UTF-8

parameters::
* xml:

example request::
-

result::
-

== Remove Pipeline

name::
removePipeline

description::
Remove a pipeline from the server

endPoint::
GET `/hop/removePipeline`

parameters::
- name: the name of the pipeline to remove
- id: the id of the pipeline to remove

example request::
GET `+http://localhost:8081/hop/removePipeline/?name=<NAME>&id=<ID>+`

result::

Example result:

[source,html]
----
<HTML>

<HEAD>
	<TITLE>The pipeline was removed</TITLE>
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H3>The pipeline with name [<NAME>] and hopServer object id <ID> was removed from Hop
		Server.</H3>
	<a href="/hop/status">Back to the status page</a><br>
	<p>
</BODY>
----

== Remove Workflow

name::
removeWorkflow

description::
Remove a workflow from the server

endPoint::
GET `/hop/removeWorkflow`

parameters::
* name: the name of the workflow to remove
* id: the id of the workflow to remove

example request::
GET `+http://localhost:8081/hop/removeWorkflow/?name=<NAME>&id=<ID>+`

result::

Example result:

[source,html]
----
<HTML>

<HEAD>
	<TITLE>The workflow was removed</TITLE>
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H3>The workflow with name [<NAME>] and hopServer object id <ID> was removed from Hop
		Server.</H3>
	<a href="/hop/status">Back to the status page</a><br>
	<p>
</BODY>
----

== Sniff Transform

name::
sniffTransform

description::
Sniff test a pipeline transform

endPoint::
GET `/hop/sniffTransform`

parameters::
* xml: use XML (default: Y)
* pipeline: name of the pipeline to sniff
* id: id of the pipeline to sniff
* transform: name of the transform to sniff
* type: (input/output) sniff input or output
* lines: number of lines to wait for
* copynr: when using multiple copies you can specify the copynr

following parameters are required::
* xml
* id or pipeline
* transform


example request::
GET `+http://localhost:8081/hop/sniffTransform/?xml=Y&pipeline=<NAME>&id=<ID>&transform=<TRANSFORMNAME>&type=output&lines=100+`

result::

Example result (empty):

[source,xml]
<row-buffer>
<row-meta/>
</row-buffer>

== Start Pipeline Execution

name::
startExec

description::
Start the execution of a pipeline

endPoint::
GET `/hop/startExec`

parameters::
* name: the name of the pipeline to start

example request::
GET `+http://localhost:8081/hop/startExec?name=<NAME>+`

result::

Example result:

[source,html]
----
<HTML>

<HEAD>
	<TITLE>Start of pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/pipelineStatus?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline [<NAME>] was started.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=8bea27db-de97-4bd0-a210-d9bba3aacac2">Back to the status page</a>
	<p>
		<p>
</BODY>

</HTML>
----

== Start Pipeline

name::
startPipeline

description::
Prepare and start the execution of a pipeline

endPoint::
GET `/hop/startPipeline`

parameters::
* name: the name of the pipeline to start

example request::
GET `+http://localhost:8081/hop/startPipeline?name=<NAME>+`

result::

Example result:

[source,html]
----
<HTML>

<HEAD>
	<TITLE>Start of pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/pipelineStatus?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline [<NAME>] was started.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=8bea27db-de97-4bd0-a210-d9bba3aacac2">Back to the status page</a>
	<p>
		<p>
</BODY>

</HTML>
----

== Start Workflow

name::
startWorkflow

description::
Prepare and start the execution of a workflow

endPoint::
GET `/hop/startPipeline`

parameters::
* name: the name of the workflow to start

example request::
GET `+http://localhost:8081/hop/startWorkflow?name=<NAME>+`

result::

Example result:

[source,html]
----
<HTML>

<HEAD>
	<TITLE>Start of workflow</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/startWorkflow?name=<NAME>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Workflow [<NAME>] was started.</H1>
	<a href="/hop/workflowStatus?name=<NAME>&id=8bea27db-de97-4bd0-a210-d9bba3aacac2">Back to the status page</a>
	<p>
		<p>
</BODY>

</HTML>
----

== Stop Pipeline

name::
stopPipeline

description::
Stop a pipeline

endPoint::
GET `/hop/stopPipeline`

parameters::
* name: the name of the pipeline to stop
* id: the id of the pipeline to stop

example request::
GET `+http://localhost:8081/hop/stopPipeline?name=<NAME>&id=<ID>+`

result::

Example Result:

[source,html]
----
<HTML>

<HEAD>
	<TITLE>Stop pipeline</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/pipelineStatus?name=<NAME>>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Pipeline [<NAME>] stop requested.</H1>
	<a href="/hop/pipelineStatus?name=<NAME>&id=<ID>">Back to the pipeline status page</a>
	<p>
		<p>
</BODY>

</HTML>
----

== Stop Workflow

name::
stopWorkflow

description::
Stop a workflow

endPoint::
GET `/hop/stopWorkflow`

parameters::
* name: the name of the workflow to stop
* id: the id of the workflow to stop

example request::
GET `+http://localhost:8081/hop/stopWorkflow?name=<NAME>&id=<ID>+`


result::

Example Result:

[source,html]
----
<HTML>

<HEAD>
	<TITLE>Stop workflow</TITLE>
	<META http-equiv="Refresh" content="2;url=/hop/workflowStatus?name=<NAME>>">
	<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</HEAD>

<BODY>
	<H1>Workflow [<NAME>] stop requested.</H1>
	<a href="/hop/workflowStatus?name=<NAME>&id=<ID>">Back to the pipeline status page</a>
	<p>
		<p>
</BODY>

</HTML>
----

