{"revision":"c10276e953456835cdbb1f3a60d58ae1980be353","document":{"id":21114,"path":"integrate-applications/integration-hub/create-data-stream-action.md","title":"Create a Data Stream action","bundle":null,"doc_type":null,"release":"australia","body":"\n# Create a Data Stream action\n\nCreate a reusable action to process a stream of response data within a flow.\n\n## Before you begin\n\n-   Set up an application in Guided Application Creator to store Workflow Studio content.\n-   Disable any conflicting business rules or workflows before creating an action.\n-   Role required: action\\_designer or admin\n\n## About this task\n\nCreating a custom application to contain your Workflow Studio content enables you to deploy it using the application repository or the ServiceNow Store.\n\n**Note:** Clearing an option in a configuration page removes the step from the Data Stream outline and deletes all data associated with the step.\n\n## Procedure\n\n1.  Navigate to **All** &gt; **Flow Designer** &gt; **Designer**.\n\n2.  Click the **Actions** tab and select **Data Stream**.\n\n3.  Fill in the Action Properties and click **Submit**.\n\n    |Field|Description|\n    |-----|-----------|\n    |Action name|Enter a unique name for your data stream action.|\n    |Description|Description of the data stream action.|\n    |Application|Application scope of the data stream action.|\n    |Domain|Domain scope of the data stream action. For more information about domain separation, see [Domain separation explained](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/platform-security/bp-what-is-domain-separation.md).|\n    |**Additional properties**|\n    |Accessible From|Accessible from all application scoped or only within the specified application scope.|\n    |Category|Defined category within the application scope for an action. Beginning with the Xanadu release, you can create a custom category to organize your data stream actions.|\n    |Protection|Select whether the data stream action is read-only. You can only select a value when you create the data stream action in an application scope you own. The default value is None.|\n    |Action annotation|Help text that appears under the data stream action title in Workflow Studio to help data stream action authors understand what the data stream action does when used in a flow.|\n\n    An empty Data Stream action opens.\n\n4.  Define action inputs to make data available to the action steps.\n\n    1.  Select **+ Create Input** and complete the fields.\n\n        Inputs are represented as data pills in the right-hand pane.\n\n    For more information about action inputs, see [Workflow Studio - Building custom actions](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/build-workflows/actions.md).\n\n5.  Click **Action Preprocessing** in the Data Stream outline and configure the desired options.\n\n    1.  Select **Retrieve connection info** to add the Get Connection Info step as the first step in the action preprocessing.\n\n        The Get Connection Info step enables you to retrieve connection and credential details to use in your action. For more information, see .\n\n    2.  Select **Enable preprocessing script** to run a preprocessing script before the action sends the initial API request. For example, validate action inputs or set default values. Preprocessing executes once per action, before the first API request.\n\n        Selecting this option adds a script step to the Data Stream action. For more information, see [Script step](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/build-workflows/javascript-step-action-designer.md).\n\n        **Important:** Data Stream actions require a constant connection to the response stream. They do not support preprocessing scripts that pause the action to call a MID Server or pause the action to wait for a condition or duration. A preprocessing script that pauses the action for any reason returns an error message.\n\n6.  Click **Request** in the Data Stream outline and configure the desired options.\n\n    1.  In the **How will you get data** field, select **REST Step**, **SOAP Step**, or **JDBC Step** to add the associated step to the Data Stream action.\n\n        For more information, see [REST step](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/rest-request-action-designer.md), [SOAP step](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/soap-step-action-designer.md), and [JDBC step](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/jdbc-step.md) .\n\n        This section can run on either the MID Server or the instance. The environment is determined by the **Use MID server** field in the associated Connection \\[sys\\_connection\\] record.\n\n        **Note:**\n\n        -   If you are using the JDBC step, you must test it. Upon successful execution, you can use the result to create the step output and data stream action output.\n        -   If you are using the REST step or SOAP step, you must manually create the data stream action output.\n    2.  Select **Enable pagination** to request results in batches. Once one page of data is processed, the Data Stream action runs the request section again to return the next set of results. This option adds a Pagination Setup step to the Data Stream outline.\n\n        **Note:** For a JDBC step, pagination is not applicable. Each page can retrieve up to 1 GB data and one request can retrieve up to 8 GB data.\n\n    3.  Select **Run a script before each request** to run a script before every request for the next page when calling a paginated API.\n\n        For example, write a script to transform variable data types from the initial response before sending a request for the next page.\n\n        Selecting this option adds a script step to the Data Stream action. For more information, see [Script step](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/build-workflows/javascript-step-action-designer.md).\n\n        **Note:** For a JDBC step, this is not applicable.\n\n        This section can run on either the MID Server or the instance. The environment is determined by the script step's **Required Runtime** field.\n\n        **Important:** Avoid shifting the execution environment between the instance and the MID Server multiple times. For example, you might configure the Request Script step to run on the MID Server, but the REST step to run on the instance. In this case, the system shifts environments between the instance and MID Server for every page of data, which may degrade performance.\n\n7.  If pagination is enabled, configure the Pagination Setup step.\n\n    Configure the Pagination Setup step manually, or select a pre-built template to apply common configurations. For example, apply the **Limit / Offset** template to specify the number of items you want returned per page \\(limit\\), and the starting number for the first item \\(offset\\). After applying a template, update the values to ensure that the configuration complies with the API's requirements.\n\n    1.  Create pagination variables.\n\n        For example, if the third-party API takes a `limit` parameter in the request, create the `limit` variable and set the initial value to limit the number of results per page. The initial value is only used in the first request. `getNextPage` is a reserved, read-only variable. As long as the `getNextPage` variable is true and the previous page contains data, the action continues to send requests for the next page.\n\n    2.  In the **Next Value From** field, define how pagination variables receive a value for subsequent requests.\n\n        Choose from:\n\n        -   **Script**: Write a Pagination Variables Script to define how variables are populated. Pagination variables only support the string data type. To perform math operations, you must convert the value to an integer, perform any required operations, then convert it back to a string.\n        -   **Response Body**: Use a value in the response from the previous request to populate the variable. If the response is JSON format, set **Extract value using** to **JSONPath Expression** and provide the path to the value in the **Expression** field. If the response is XML format, set **Extract value using** to **XPath Expression** and provide the path to the value.\n    In this example, the `getNextPage` variable is true until the `nextOffset` variable reaches the total count value returned in the API response header. As long as the `getNextPage` variable is true and the previous page contains data, the action continues to send requests for the next page. This example includes a common limit/offset pagination configuration. The third-party APIs that your Data Stream action interacts with may use a page token or other method.\n\n    \\[Omitted image \"pagination-setup.png\"\\] Alt text: Pagination setup with a pagination variables script.\n\n    **Important:** Avoid infinite loops in pagination requests by creating a condition that sets the `getNextPage` variable to false. Cancel any long-running flows. Always test Data Stream actions before using them in production.\n\n8.  Click **Parsing** in the Data Stream outline and configure the desired options.\n\n    **Note:** For a JDBC step, this is not applicable.\n\n    1.  In the **How will you identify each record** field, select **JSON/XML Splitter** to add a Splitter step to the Data Stream outline.\n\n    2.  In the **How will you parse each item into an object** field, select **Script Parser** to add a Script Parser step to the Data Stream outline.\n\n9.  Configure the Splitter step.\n\n    This step identifies the parent node in the response stream to map to a complex object. For example, identify a user element in an XML payload to create a complex object for each user in the response stream.\n\n    **Note:** For a JDBC step, this is not applicable.\n\n    1.  In the **Source Format** field, select the format returned by the Request section.\n\n        -   **JSON**: Identifies objects from a stream of JSON data. Use a JSONPath expression to identify a JSON array containing repeated data.\n        -   **XML**: Identify objects from a stream of XML data. Use an XPath expression to identify an XML element containing repeated data.\n    2.  In the **Item Path** field, define the absolute path to the item in the data stream you want to map to an object.\n\n        -   **JSON**\n\n            Identify the absolute path to an array of objects as a JSONPath expression. For example, use `$.result` to separate each item in a JSON `result` array into a `result` object.\n\n            **Note:** If you select an array at the root node of a JSON data stream, the system displays the absolute path as `$.*` in execution details and error messages.\n\n        -   **XML**\n\n            Identify the absolute path to a repeating data object as a XPath expression. For example, use `/result` to separate each instance of a `result` XML element into a `result` object.\n\n            **Note:** The Splitter step ignores XML namespaces.​\n\n        For example, suppose the REST Step returns a stream of JSON data. The item path `$.response.result.companies` returns each item in the `companies` array.\n\n        ```\n        {​\n          \"response\": {​\n            \"result\": {​\n              \"companies\": [​\n                {​\n                  \"name\": \"company1\"​\n                },​\n                {​\n                  \"name\": \"company2\"​\n                },​\n                {​\n                  \"name\": \"company3\"​\n                }​\n              ],​\n              \"metadata\": {​\n                \"token\": 1558666526​\n              }​\n            }​\n          }​\n        } \n        ```\n\n        For example, suppose the REST Step returns a stream of XML data. The item path `/response/result/companies​/company` returns each `company` element from the XML data.\n\n        ```\n        <response>​\n          <result>​\n            <companies>​\n              <company>​\n                <!-- company 1 info -->​\n              </company>​\n              <company>​\n                <!-- company 2 info -->​\n              </company>​\n              <company>​\n                <!-- company 3 info -->​\n              </company>​\n              <company>​\n                <!-- company 4 info -->​\n              </company>​\n              <batch>10645C53D4BED73YQ</batch> ​\n            </companies>​\n            <metadata>​\n              <timestamp>1558666526</timestamp>​\n            </metadata>​\n          </result>​\n        </response>\n        ```\n\n10. In the Script Parser step, use JavaScript and ServiceNow APIs to map items in the response stream to a complex object output represented by the `targetObject` global object.\n\n    For example, map incident record elements identified in the splitter step to a complex object containing incident fields. If the data stream includes siblings to the item identified in the splitter step that you do not want mapped to a complex object, include conditions to exclude those items.\n\n    **Note:** For a JDBC step, this is not applicable.\n\n    You can access outputs from previous steps in your data stream action using the fd\\_data object, excluding:\n\n    -   REST or SOAP step Response Body, Stream, or Error Message outputs.\n    -   Splitter step outputs.\n    An example script that parses a JSON response.\n\n    ```\n    (function parse(inputs, outputs) {\n        var record = JSON.parse(inputs.sourceItem);\n        outputs.targetObject.id=record.number;\n        outputs.targetObject.name=record.short_description;\n    })(inputs, outputs)\n    ```\n\n    An example script that parses an XML response.\n\n    ```\n    (function parse(inputs, outputs) {\n        var xmlDoc = new XMLDocument(inputs.sourceItem, false);\n        outputs.targetObject.id = xmlDoc.getNodeText(\"/result/number\");\n        outputs.targetObject.name = xmlDoc.getNodeText(\"/result/short_description\");\n    })(inputs, outputs)\n    ```\n\n11. If you selected the JDBC step, click **Transform**.\n\n    After successfully testing the JDBC step, click **Use Result** to create the step output and data stream action output. You can use this result or customize it using the transform script.\n\n    1.  Select the **Enable Transform Script** check box to transform and customize the default data stream output.\n\n12. Create a complex object output.\n\n    **Note:** For a JDBC step, this is not applicable.\n\n    1.  Select **Outputs** in the Action Outline and click **+** to create an output.\n\n    2.  Update the **Label** field to represent the object.\n\n        For example, if the action parses a stream of company records, add the **Company** label.\n\n    3.  Update the **Type** field to **Object**.\n\n        This is the complex object output represented by the `targetObject` global object in the Script Parser step.\n\n    4.  Add child fields to the complex object using the **+** icon.\n\n    5.  Make the name of each child field more user-friendly so that you can meaningfully refer to it in script.\n\n        The value in the **Name** field is the internal name used in the Script Parser step. For example, to refer to the City output in the Script Parser step, you would use `outputs.targetObject.city`.\n\n        \\[Omitted image \"ds-output-label.png\"\\] Alt text: Default values for child items in the complex object output.\n\n        **Warning:** After saving the Data Stream action, you cannot change values in the **Name** field.\n\n13. Click **Save**.\n\n    Workflow Studio saves a draft of the action.\n\n\n## What to do next\n\n[Test a Data Stream action](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/test-data-stream-action.md).\n\n**Parent Topic:**[Data Stream actions and pagination](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/data-stream-actions.md)\n\n"}}