{"revision":"c10276e953456835cdbb1f3a60d58ae1980be353","document":{"id":21082,"path":"integrate-applications/integration-hub/configure-message-topic-reps-sc.md","title":"Create message and Kafka topic replications in Stream Connect","bundle":null,"doc_type":null,"release":"australia","body":"\n# Create message and Kafka topic replications in Stream Connect\n\nSet up message and topic replications to replicate data between your Apache Kafka environment and ServiceNow.\n\n## Before you begin\n\n- Role required: message\\_replication\\_admin\n- This feature requires a subscription. For more information, see [Using Stream Connect for Apache Kafka](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/stream-connect-apache-kafka.md).\n- Verify the Hermes Messaging Service and Instance PKI Certificate Generator are working.\n - [Check the status of and connection to the Hermes Kafka cluster](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/servicenow-platform/run-hermes-messaging-service-diagnostics.md).\n - Check the [Key Management Framework Health](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/platform-security/kmf_diagnostics.md) to verify that all three items in the Instance PKI section are operational.\n- Configure and start a MID Server or MID Server cluster. For more information, see [Configuring MID Servers](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/servicenow-platform/c_MIDServerConfiguration.md) and [Configure a MID Server cluster](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/servicenow-platform/t_ConfigureAMIDServerCluster.md).\n\n The MID Server must be able to access Hermes endpoints through the firewall. Work with your network administrator to make sure that the following port ranges are open.\n\n - Producer: 4000–4050\n - Consumer1: 4100–4150\n - Consumer2: 4200–4250\n **Note:** Proxy settings on the MID Server don't apply to Hermes connectivity. The MID Server proxy configuration applies only to HTTP-based communication. Hermes uses Kafka-native TCP connections, so traffic to the Hermes cluster bypasses proxy settings regardless of how the MID Server is configured.\n\n If you're using a MID Server cluster, it must have a cluster type of **Load Balance**. The **Failover** cluster type is not supported.\n\n- Stream Connect Message Replication uses a Connection & Credential alias to connect to your local Kafka. [Create a Connection & Credential alias](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/platform-security/connection-alias.md) with a **Connection type** of **Kafka**. A Connection & Credential alias requires a [Kafka Connection](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/configure-kafka-connection.md) and a [Kafka Credential](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/create-kafka-ssl-credentials.md).\n- Stream Connect Message Replication requires the ServiceNow Stream Connect Installer \\[com.glide.hub.stream\\_connect.installer\\] plugin.\n\n## About this task\n\nThis page shows you how to create a Message Replication record and an associated Kafka Topic Replication record.\n\nA Message Replication record represents a single Kafka cluster. For example, if you have two Kafka clusters, you would create two different message replication records, one for each cluster. A Message Replication record is the parent record for all the topics being replicated to or from that cluster. Message Replication records are stored in the Message Replications \\[sys\\_sc\\_message\\_replication\\] table.\n\nA Kafka Topic Replication record specifies the replication from a single source topic to a single destination topic. You can't replicate a single source topic to multiple destinations. You can only replicate to each destination once. Kafka Topic Replication records are stored in the Kafka Topic Replications \\[sys\\_kafka\\_topic\\_replication\\] table.\n\n## Procedure\n\n1. Navigate to **All** > **IntegrationHub** > **Stream Connect** > **Message Replications** and select **New**.\n\n2. On the Message Replication form, fill in the fields.\n\n |Field|Description|\n |-----|-----------|\n |Name|Name of the message replication.|\n |Messaging service connection|Connection & Credential alias specifying which messaging service to connect to.|\n |Application|Application scope for the message replication. This field is automatically set.|\n |Active|Option to activate the message replication.|\n\n3. Select **Save**.\n\n After you save the form, the Message Replication Statuses and Kafka Topic Replications tabs appear at the bottom of the page.\n\n4. On the Kafka Topic Replications tab, select **New**.\n\n5. On the Kafka Topic Replication form, fill in the fields.\n\n\n
\n\nField\n\n\n\nDescription\n\n
\n\nName\n\n\n\nName of the topic replication. The name is generated automatically based on the values in other fields.\n\n To generate the **Name**, enter the values for the **Direction**, **ServiceNow topic**, and **Topic name** fields, then select **Save**.\n\n When the **Direction** is **To ServiceNow**, the **Name** has the following format: `topic name [replication name] -> ServiceNow topic name [SN]`.\n\n When the **Direction** is **From ServiceNow**, the **Name** has the following format: `ServiceNow topic name [SN] -> topic name [replication name]`.\n\n
\n\nDirection\n\n\n\nDirection of the topic replication.- **To ServiceNow** replicates a topic from your local Kafka environment to ServiceNow.\n- **From ServiceNow** replicates a topic from ServiceNow to your local Kafka environment.\n\n\n
\n\nServiceNow topic\n\n\n\nReference to the Kafka Topics \\[sys\\_kafka\\_topic\\] table.\n\n
\n\nReplication\n\n\n\nMessage replication that manages this topic replication.\n\n
\n\nTopic name\n\n\n\nName of the topic in your local Kafka environment.\n\n
\n\nActive\n\n\n\nOption to activate the topic replication.\n\n
\n\nTrack in current update set\n\n\n\nWhen selected, the topic replication is added to the current update set, so it's simpler to promote it from sub-production to production instances. You may need to configure the form to add this field.\n\n
6. Select **Submit**.\n\n7. Follow steps 1–6 to create message and topic replications for all your Kafka clusters and topics.\n\n\n## Result\n\nAfter you've created all your message and topic replication records, messages should be replicating. You can check the Message Replication Statuses \\[sys\\_sc\\_message\\_replication\\_status\\] tab on the Message Replication form to view the status, MID Server, and error messages for each message replication.\n\nYou can also check the Channel Replication Statuses \\[sys\\_sc\\_channel\\_replication\\_status\\] tab on the Kafka Topic Replication record to view status details for each topic replication, including the **MID Server**, **Error message**, and **Error Since** values.\n\nEach Kafka Topic Replication record also has a related list named Message Replication Statistics. Once replications are running, this list displays metrics records, which are created every 60 seconds for each topic replication. Each metrics record has information about the topic replication, including the **Message count**, which shows the number of messages replicated in each collection interval.\n\n## What to do next\n\nIf you're replicating messages from your local Kafka into ServiceNow, you can configure a [Script](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/configure-script-consumer.md), [ETL](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/configure-etl-consumer.md), or [Transform Map](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/configure-transform-map-consumer.md) consumer to process the messages or set up a Kafka Message trigger to start a flow.\n\nIf you're replicating messages from ServiceNow to your local Kafka, you can use a in Workflow Studio or use the ProducerV2 API to publish messages to the Hermes topic and your local Kafka.\n\n**Parent Topic:**[Stream Connect Message Replication](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/integrate-applications/integration-hub/stream-connect-message-replication.md)\n\n"}}