{"revision":"c10276e953456835cdbb1f3a60d58ae1980be353","document":{"id":755,"path":"api-reference/server-api-reference/EncoderAPI.md","title":"Encoder - Global","bundle":null,"doc_type":null,"release":"australia","body":"\n# Encoder- Global\n\nThe Encoder API provides a scriptable object used in Predictive Intelligence stores. This object converts input data into vectors of numbers, based on encoder-specific goals and configurations. Encoders can be used independently to run encodings or can be configured as part of solutions to encode text columns.\n\nThis API requires the Predictive Intelligence plugin \\(com.glide.platform\\_ml\\) and is provided within the `sn_ml` namespace.\n\nEncoders are text processing objects that are either pre-trained or trained based on the language datasets you provide. You can train encoders that determine how the system interprets and processes text fields. For ML solutions that include text, you can train an encoder to specify how to process text and use the trained encoder in a solution.\n\nEncoders have configuration and versions, and can be trained independently with their own retraining frequency. API-defined encoders are different from UI-defined encoders, because the retraining of UI-defined encoders is controlled by the solutions using them.\n\nThe encoder setup-to-training flow is as follows:\n\n1. Create one or more datasets using the [DatasetDefinition](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/DatasetDefinitionAPI.md) API.\n2. Use the [constructor](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderAPI.md) to create an encoder object.\n3. Add the encoder object to the encoder store using the [EncoderStore - add\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderStoreAPI.md) method.\n4. Train the encoder using the [submitTrainingJob\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderAPI.md) method. This creates a version of the object that you can manage using the [EncoderVersion](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderVersionAPI.md) API.\n\nOnce you have trained an encoder, you can use it in a solution object:\n\n- [ClassificationSolution](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/ClassificationSolutionAPI.md)\n- [ClusteringSolution](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/ClusteringSolutionAPI.md) \\(required unless using the Levenshtein distance algorithm\\)\n- [RegressionSolution](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/RegressionSolutionAPI.md)\n- [SimilaritySolution](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/SimilaritySolutionAPI.md) \\(required\\)\n\n**Note:** This API runs with full privileges before the Vancouver Patch 7 Hotfix 2b and Washington DC Patch 7 releases. With later releases, grant access using ACLs. For more information see [Query ACLs](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/platform-security/query-acl-rule.md).\n\nFor usage guidelines, refer to [Using ML APIs](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/intelligent-experiences/using-ml-apis.md).\n\n**Parent Topic:**[Server API reference](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/api-server.md)\n\n## Encoder - Encoder\\(Object config\\)\n\nCreates an encoder.\n\nTo get an encoder for one or more datasets, use this constructor to create a new encoder object with a unique name.\n\n\n
\n\nName\n\n\n\nType\n\n\n\nDescription\n\n
\n\nconfig\n\n\n\nObject\n\n\n\nJavaScript object containing configuration properties of theencoder.```\n{\n \"algorithmConfig\": {Object},\n \"datasets\": [Array],\n \"domainName\": \"String\",\n \"label\": \"String\",\n \"minRowCount\": \"String\",\n \"processingLanguage\": \"String\",\n \"stopwords\": [Array],\n \"trainingFrequency\": \"String\"\n}\n```\n\n
\n\nconfig.algorithmConfig\n\n\n\nObject\n\n\n\nOptional. JavaScript object containing algorithm configuration properties. ```\n'algorithmConfig' : {\n \"algorithm\": \"String\"\n}\n```\n\n
\n\nconfig.algorithmConfig.algorithm\n\n\n\nString\n\n\n\nName of the algorithm for training this encoder. Possible values:\n\n- **paravec**: Paragraph vector word embedding.\n- **tf-idf**: Term Frequency–Inverse Document Frequency \\(TF-IDF\\)-based text.\n\n
\n\nconfig.datasets\n\n\n\nArray\n\n\n\nList of [DatasetDefinition](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/DatasetDefinitionAPI.md) object names.\n\n
\n\nconfig.domainName\n\n\n\nString\n\n\n\nOptional. Domain name associated with this dataset. Default: Current domain, for example, `\"global\"`.\n\n
\n\nconfig.label\n\n\n\nString\n\n\n\nIdentifies the prediction task.\n\n
\n\nconfig.minRowCount\n\n\n\nString\n\n\n\nOptional. Minimum number of records required in the dataset for training.Default: 10000\n\n
\n\nconfig.processingLanguage\n\n\n\nString\n\n\n\nOptional. Processing language in two-letter ISO 639-1 language code format. Default: \"en\"\n\n
\n\nconfig.stopwords\n\n\n\nArray\n\n\n\nOptional. Preset list of strings that the system automatically generates based on the **language** property setting. For details, see [Create a custom stopwords list](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/intelligent-experiences/create-custom-stopwords-list.md). Default: English Stopwords\n\n
\n\nconfig.trainingFrequency\n\n\n\nString\n\n\n\nThe frequency to retrain the model. Possible values:\n\n- every\\_30\\_days\n- every\\_60\\_days\n- every\\_90\\_days\n- every\\_120\\_days\n- every\\_180\\_days\n- run\\_once\n\nDefault: run\\_once\n\n
The following example shows how to create an encoder job and add it to the encoder store.\n\n```\nvar myPrbData = new sn_ml.DatasetDefinition({\n 'tableName' : 'problem',\n 'fieldNames' : ['short_description'],\n 'encodedQuery' : 'activeANYTHING'\n});\n\nvar myIncidentData = new sn_ml.DatasetDefinition({\n 'tableName' : 'incident',\n 'fieldNames' : ['short_description', 'description'],\n 'encodedQuery' : 'activeANYTHING'\n});\n\nvar myEncoder = new sn_ml.Encoder({\n 'label': \"encoder\",\n 'datasets' : [myPrbData, myIncidentData],\n 'algorithmConfig' : {\n 'algorithm' : 'tf-idf'\n }\n});\nvar myEncoderName = sn_ml.EncoderStore.add(myEncoder);\n```\n\n## Encoder - cancelTrainingJob\\(\\)\n\nCancels a job for a encoder object that has been submitted for training.\n\n|Name|Type|Description|\n|----|----|-----------|\n|None| | |\n\n|Type|Description|\n|----|-----------|\n|None| |\n\nThe following example shows how to cancel an existing training job.\n\n```\nvar myEncoder = sn_ml.EncoderStore.get('ml_sn_global_global_encoder');\n\nmyEncoder.cancelTrainingJob();\n```\n\n## Encoder - getActiveVersion\\(\\)\n\nGets the active EncoderVersion object.\n\n|Name|Type|Description|\n|----|----|-----------|\n|None| | |\n\n|Type|Description|\n|----|-----------|\n|Object|Active [EncoderVersion](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderVersionAPI.md) object.|\n\nThe following example shows how to get an active Encoder version from the store and return its training status.\n\n```\nvar mlEncoder = sn_ml.EncoderStore.get('ml_x_snc_global_global_encoder');\n\ngs.print(JSON.stringify(JSON.parse(mlEncoder.getActiveVersion().getStatus()), null, 2));\n```\n\nOutput:\n\n```\n{\n \"state\": \"encoder_complete\",\n \"percentComplete\": \"100\",\n \"hasJobEnded\": \"true\"\n}\n```\n\n## Encoder - getAllVersions\\(\\)\n\nGets all versions of an encoder.\n\n|Name|Type|Description|\n|----|----|-----------|\n|None| | |\n\n|Type|Description|\n|----|-----------|\n|Array|Existing versions of an encoder object. See also [EncoderVersion](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderVersionAPI.md) API.|\n\nThe following example shows how to get all Encoder version objects and call the getVersionNumber\\(\\) and getStatus\\(\\) encoder version methods on them.\n\n```\nvar mlEncoder = sn_ml.EncoderStore.get('ml_x_snc_global_global_encoder');\n\nvar mlEncoderVersions = mlEncoder.getAllVersions();\n\nfor (i = 0; i < mlEncoderVersions.length; i++) {\ngs.print(\"Version \" + mlEncoderVersions[i].getVersionNumber() + \" Status: \" + mlEncoderVersions[i].getStatus() +\"\\n\");\n};\n```\n\nOutput:\n\n```\nVersion 3 Status: {\"state\":\"encoder_complete\",\"percentComplete\":\"100\",\"hasJobEnded\":\"true\"}\n\nVersion 2 Status: {\"state\":\"encoder_complete\",\"percentComplete\":\"100\",\"hasJobEnded\":\"true\"}\n\nVersion 1 Status: {\"state\":\"encoder_cancelled\",\"percentComplete\":\"0\",\"hasJobEnded\":\"true\"}\n```\n\n## Encoder - getLatestVersion\\(\\)\n\nGets the latest version of an encoder.\n\n|Name|Type|Description|\n|----|----|-----------|\n|None| | |\n\n|Type|Description|\n|----|-----------|\n|Object|[EncoderVersion](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderVersionAPI.md) object corresponding to the latest version of an [Encoder\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderAPI.md).|\n\nThe following example shows how to get the latest version of an encoder and return its training status.\n\n```\nvar mlEncoder = sn_ml.EncoderStore.get('ml_x_snc_global_global_encoder');\n\ngs.print(JSON.stringify(JSON.parse(mlEncoder.getLatestVersion().getStatus()), null, 2));\n```\n\nOutput:\n\n```\n{\n \"state\": \"encoder_complete\",\n \"percentComplete\": \"100\",\n \"hasJobEnded\": \"true\"\n}\n```\n\n## Encoder - getName\\(\\)\n\nGets the name of the object to use for interaction with the store.\n\n|Name|Type|Description|\n|----|----|-----------|\n|None| | |\n\n|Type|Description|\n|----|-----------|\n|String|Name of the encoder object.|\n\nThe following example shows how to update Encoder dataset information and print the name of the object.\n\n```\n// Update encoder\nvar myIncidentData = new sn_ml.DatasetDefinition({\n 'tableName' : 'incident',\n 'fieldNames' : ['category', 'short_description', 'priority'],\n 'encodedQuery' : 'activeANYTHING'\n});\n\nvar eligibleFields = JSON.parse(myIncidentData.getEligibleFields(encoder));\n\nvar myEncoder = new sn_ml.Encoder({\n 'label': \"my encoder\",\n 'datasets' : [myIncidentData],\n 'inputFieldNames': eligibleFields['eligibleInputFieldNames'],\n 'predictedFieldName': 'category'\n});\n\n// update encoder\nsn_ml.EncoderStore.update('ml_x_snc_global_global_my_definition_4', myEncoder);\n\n// print encoder name\ngs.print('Encoder Name: '+myEncoder.getName());\n```\n\nOutput:\n\n```\nEncoder Name: ml_x_snc_global_global_my_definition_4\n```\n\n## Encoder - getProperties\\(\\)\n\nGets solution object properties.\n\n|Name|Type|Description|\n|----|----|-----------|\n|None| | |\n\n\n
\n\nType\n\n\n\nDescription\n\n
\n\nObject\n\n\n\nContents of the Dataset and [Encoder\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderAPI.md) object details in the [EncoderStore](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderStoreAPI.md).```\n{\n \"algorithmConfig\" : {Object},\n \"datasetsProperties\": [Array],\n \"domainName\": \"String\",\n \"label\": \"String\",\n \"name\": \"String\",\n \"processingLanguage\": \"String\",\n \"scope\": \"String\",\n \"stopwords\": [Array],\n \"trainingFrequency\": \"String\"\n}\n```\n\n
\n\n<Object>.algorithmConfig.algorithm\n\n\n\nName of the algorithm for training this encoder. Possible values:\n\n- **paravec**: Paragraph vector word embedding.\n- **tf-idf**: Term Frequency–Inverse Document Frequency \\(TF-IDF\\)-based text.\n\n Data type: String.\n\n
\n\n<Object>.algorithmConfig\n\n\n\nOptional. JavaScript object containing algorithm configuration properties. ```\n'algorithmConfig' : {\n \"algorithm\": \"String\"\n}\n```\n\n Data type: Object.\n\n
\n\n<Object>.datasetsProperties\n\n\n\nList of [DatasetDefinition\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/DatasetDefinitionAPI.md) properties associated with the encoder.\n\n ```\n{\n \"encodedQuery\": \"String\",\n \"fieldDetails\": [Array],\n \"fieldNames\": [Array],\n \"tableName\": \"String\"\n}\n```\n\n Data type: Array.\n\n
\n\n<Object>.datasetsProperties.tableName\n\n\n\nName of the table for the dataset. For example, `\"tableName\" : \"Incident\"`. Data type: String.\n\n
\n\n<Object>.datasetsProperties.fieldNames\n\n\n\nList of field names from the specified table as strings. For example, `\"fieldNames\" : [\"short_description\", \"priority\"]`. Data type: Array.\n\n
\n\n<Object>.datasetsProperties.fieldNames.fieldDetails\n\n\n\nList of JavaScript objects that specify field properties. ```\n[\n {\n \"name\": \"String\",\n \"type\": \"String\"\n }\n]\n```\n\n Data type: Array.\n\n
\n\n<Object>.datasetsProperties.fieldNames.fieldDetails.<object>.name\n\n\n\nName of the field defining the type of information to restrict this dataset to. Data type: String.\n\n
\n\n<Object>.datasetsProperties.fieldDetails.<object>.type\n\n\n\nMachine-learning field type. Data type: String.\n\n
\n\n<Object>.datasetsProperties.fieldDetails.encodedQuery\n\n\n\nEncoded query string in the standard platform format. See [Encoded query strings](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/platform-user-interface/c_EncodedQueryStrings.md).Data type: String.\n\n
\n\n<Object>.domainName\n\n\n\nDomain name associated with this dataset. See [Domain separation and Predictive Intelligence](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/intelligent-experiences/domain-separation-predictive-intelligence.md).Type: String\n\n
\n\n<Object>.label\n\n\n\nIdentifies the prediction task. ```\n{\n \"label\": \"my first prediction\"\n}\n```\n\n Data type: String.\n\n
\n\n<Object>.name\n\n\n\nSystem-assigned name. Data type: String.\n\n
\n\n<Object>.predictedFieldName\n\n\n\nIdentifies a field to be trained for predictability. Data type: String.\n\n
\n\n<Object>.processingLanguage\n\n\n\nProcessing language in two-letter ISO 639-1 language code format. Data type: String.\n\n
\n\n<Object>.scope\n\n\n\nObject scope. Currently the only valid value is `global`.Data type: String\n\n
\n\n<Object>.stopwords\n\n\n\nOptional. Preset list of strings that the system automatically generates based on the **language** property setting. For details, see [Create a custom stopwords list](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/intelligent-experiences/create-custom-stopwords-list.md). Data type: Array.\n\n
\n\n<Object>.trainingFrequency\n\n\n\nThe frequency to retrain the model. Possible values:\n\n- every\\_30\\_days\n- every\\_60\\_days\n- every\\_90\\_days\n- every\\_120\\_days\n- every\\_180\\_days\n- run\\_once\n\nDefault: run\\_once\n\n Data type: String.\n\n
The following example gets properties of an encoder object in the store.\n\n```\nvar myEncoder = sn_ml.EncoderStore.get('ml_sn_global_global_encoder');\n\ngs.print(JSON.stringify(JSON.parse(myEncoder.getProperties()), null, 2));\n```\n\nOutput:\n\n```\n*** Script: {\n \"datasetsProperties\": [\n {\n \"tableName\": \"incident\",\n \"fieldNames\": [\n \"assignment_group\",\n \"short_description\",\n \"description\"\n ],\n \"encodedQuery\": \"activeANYTHING\"\n }\n ],\n \"domainName\": \"global\",\n \"label\": \"my encoder definition\",\n \"name\": \"ml_x_snc_global_global_my_encoder_definition\",\n \"processingLanguage\": \"en\",\n \"scope\": \"global\",\n \"stopwords\": [\n \"Default English Stopwords\"\n ],\n \"trainingFrequency\": \"run_once\"\n}\n```\n\n## Encoder - getVersion\\(String version\\)\n\nGets an encoder by provided version number.\n\n|Name|Type|Description|\n|----|----|-----------|\n|version|String|Existing version number of an encoder.|\n\n|Type|Description|\n|----|-----------|\n|Object|Specified version of the [Encoder\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderAPI.md) object on which you can call [EncoderVersion](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderVersionAPI.md) API methods.|\n\nThe following example shows how to get the training status of an encoder by version number.\n\n```\nvar mlEncoder = sn_ml.EncoderStore.get('ml_x_snc_global_global_encoder');\n\ngs.print(JSON.stringify(JSON.parse(mlEncoder.getVersion('1').getStatus()), null, 2));\n```\n\nOutput:\n\n```\n{\n \"state\": \"encoder_complete\",\n \"percentComplete\": \"100\",\n \"hasJobEnded\": \"true\"\n}\n```\n\n## Encoder - setActiveVersion\\(String version\\)\n\nActivates a specified version of an encoder in the store.\n\n\n
\n\nName\n\n\n\nType\n\n\n\nDescription\n\n
\n\nversion\n\n\n\nString\n\n\n\nName of the [Encoder\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderAPI.md) object version to activate.Activating this version deactivates any other version.\n\n
|Type|Description|\n|----|-----------|\n|None| |\n\nThe following example shows how to activate an encoder version in the store.\n\n```\nsn_ml.Encoder.setActiveVersion(\"ml_incident_categorization\");\n```\n\n## Encoder - submitTrainingJob\\(\\)\n\nSubmits a training job.\n\n**Note:** Before running this method, you must first add an encoder to the store using the [EncoderStore - add\\(\\)](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderStoreAPI.md) method.\n\n|Name|Type|Description|\n|----|----|-----------|\n|None| | |\n\n|Type|Description|\n|----|-----------|\n|Object|[EncoderVersion](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderVersionAPI.md) object corresponding to the [Encoder](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/api-reference/server-api-reference/EncoderAPI.md) being trained.|\n\nThe following example shows how to create a dataset, apply it to an encoder, add it to a store, and submit the training job.\n\n```\n// Create a dataset \nvar myData = new sn_ml.DatasetDefinition({\n\n 'tableName' : 'incident',\n 'fieldNames' : ['assignment_group', 'short_description', 'description'],\n 'encodedQuery' : 'activeANYTHING'\n\n});\n\n// Create an encoder \nvar myEncoder = new sn_ml.Encoder({\n\n 'label': \"my encoder definition\",\n 'datasets' : [myData],\n 'predictedFieldName' : 'assignment_group',\n 'inputFieldNames':['short_description']\n\n});\n\n// Add the encoder to the store to later be able to retrieve it.\nvar my_unique_name = sn_ml.EncoderStore.add(myEncoder);\n\n// Train the encoder - this is a long running job \nvar myEncoderVersion = myEncoder.submitTrainingJob();\n```\n\n"}}