How to Enable DOM Manipulation in ServiceNow Service Portal Catalog Client Scripts

Note: This article has been updated, on 9/29/17! Scroll down to the "Update" section to see the new info! 

ServiceNow has effectively prevented its customers from utilizing any form of DOM manipulation in the service portal. For those who were brave enough to invest heavily in the Service Portal early on, this has caused major issues.

Nearly every ServiceNow customer with an even moderately utilized Service Catalog, has some Catalog Client Scripts which make use of things like g_form.getElement(). Here are just a few things we've run into, that we're not able to do in Service Portal without MAJOR custom hacks:

  • Use the document object to retrieve URI parameters in order to parse them for a client script
  • Adds event listeners to input fields on load in order to monitor for events other than "change" which requires entering the field, modifying some data, then tabbing out of the field again. 
  • Parse URI parameters to do any kind of auto-population or other processing based on them, by using document.URL.parseQuery() (the solution recommended by ServiceNow).
  • Toggle form containers
  • Get information from fields in other widgets,
  • Redirect pages
  • Show/hide/change a variable label, variable set, or container dynamically

Often during conversation with prospective ServiceNow customers, users, and developers, I am asked what it is that I like about the platform. I tell them the same thing I remember writing in one of the first articles on this website: I love ServiceNow, in large part, because whenever a client describes some pie-in-the-sky moonshot functionality, and asks if it's possible to do in ServiceNow, I almost never have to say "No, that's literally impossible". 

Don't get me wrong; I'll often say things like "That's not a good idea", or more commonly: "I get what you're driving at, but what if we did it a little more like this....", to nudge them in the direction of compliance with best-practice. The point is though, that it's very seldom impossible to do something. 
Is that dangerous? Sure, it can be. But should the customer and I be able to decide for ourselves if we want to take the risk? I firmly believe so.

That is why the Service Portal appears to be something of a "new direction" for ServiceNow. It brings a boatload of new, beautiful, amazing front-end features, properly implements Angular... but at the same time, it renders a great many of your catalog client scripts useless. Not only that, but no alternative has been provided for the functionality that was removed. 

Okay, that's not completely true. There are ways to replicate some DOM-manipulation-esque functionality in the Service Portal, but for the most part, it must be done on the widget level. That means that if you want to enable some basic functionality previously provided by getControl(), you're going to have to write a separate custom widget for the one catalog item you want it to run on, and it's going to require a lot of custom code. For each catalog item. 

Okay, so how do we un-break it? 

If you're a developer who's gone through an upgrade in the recent past, and considered moving your Service Catalog into the Service Portal, you already know all about these complaints. You've had them all yourself (or at least, your clients have); so let's get on to the part where we solve them! 

Note: Credit for the initial idea goes to resident SNG Service Portal guru, Kim!

The trick to all of this, is that on any given service portal record (in the sp_portal table), you will find a reference field called theme, which references the sp_theme table. Themes have a relationship to the JS Include (sp_js_include) table. These JS Includes can load scripts into the page, either via URL, or from UI Scripts. These scripts load and run as the document loads, before ServiceNow does whatever it does to obfuscate the DOM from being accessible to client scripts.

This is the key. By simply creating a reference to the obfuscated 'window' and 'document' objects in one of these scripts, we will then retain a reference pointer to the document. We can use this reference to reconstruct the methods of g_form that were axed (such as getElement / getControl), and we can replace former calls to the original document/window objects, with  calls to our reference objects. 

To get started, first create a new UI Script. This will be the script which runs as if it were a JS library, and has access to the DOM. I named mine setupDOM. Make sure not to set it as Global. This way, it'll only load on the Service Portal (and only on portals we want it to). 

setupDOM();

function setupDOM() {
    myDocument = document;
    myWindow = window;
    getDOMHandler();
}

function getDOMHandler() {
    var i, varz, vSid, vName, vLabel, catalogItemSid;

    window.sp_form = {};
    var itemVariables = {};

    var hostLocation = window.location.host + '';
    var sidBegin = window.location.search.indexOf('sys_id=') + 7;
    var sidEnd = window.location.search.indexOf('&', sidBegin);

    if (sidEnd >= 0) {
        catalogItemSid = window.location.search.slice(sidBegin, sidEnd);
    } else {
        catalogItemSid = window.location.search.slice(sidBegin);
    }

    var requestBody = "";
    var client = new XMLHttpRequest();
    //Updated to also get variables from variable sets on the
    client.open("get", "https://" + hostLocation + 
        "/api/now/table/item_option_new?sysparm_query=sys_idINjavascript%3Anew%20CatItemVariables().getSysIdsForQuery('" + 
        catalogItemSid + "', 'sys_id')" + 
        "%5Eactive%3Dtrue&sysparm_fields=sys_id%2Cname%2Cquestion_text&sysparm_limit=100");
    client.setRequestHeader('Accept', 'application/json');
    client.setRequestHeader('Content-Type', 'application/json');
    client.setRequestHeader('X-UserToken', g_ck);
    client.onreadystatechange = function() {
        if (this.readyState == this.DONE) {
            if (this.status == 200 || this.status == 201) {
                varz = JSON.parse(this.response).result;

                for (i = 0; i < varz.length; i++) {
                    vSid = varz[i].sys_id;
                    vName = varz[i].name;
                    vLabel = varz[i].question_text;

                    itemVariables[vName] = {
                        name:         vName,
                        sid:          vSid,
                        label:        vLabel,
                        inputElement: document.getElementById('sp_formfield_IO:' + vSid),
                        divElement:   document.getElementById(vSid)
                    };
                }
                sp_form = {
                    variables:  itemVariables,
                    getElement: function(varName) {
                        return this.variables[varName].inputElement;
                    },
                    getControl: function(varName) {
                        return this.variables[varName].divElement;
                    }
                };
            } else {
                console.error('Some kind of REST error happened. Error: ' + this.status);
            }
        }
    };
    client.send(requestBody);
}

Basically, the first function in this script is creating a reference to the document and the window objects, but places them in the global scope so they can be accessed in a client script by the names myDocument and myWindow. This step alone, means that you can do basic DOM manipulation already, from client scripts; either by using myDocument directly like so: 

myDocument.getElementById('sp_formfield_IO:' + vSid);

Or of course, you can just set the original object names back to your reference to the actual objects which ServiceNow have prevented you from typically accessing through catalog client scripts in the Service Portal by doing something like this: 

document = myDocument;
window = myWindow;
document.getElementById('sp_formfield_IO:' + vSid);

This works because when the "JS libraries" (like jQuery, for example) load, they must have access to the DOM (which, by the way, stands for Document Object Model). By creating a reference to the document and window objects at this phase (before ServiceNow goes about blocking access to them from your client scripts), we have this little side-door that we can make use of. 

So that's what those first few lines do, but what about that big function below it? 
Now that you have DOM access, this function re-enables (or rather, re-implements) the getElement/getControl methods. It creates an object in the global/window scope, called sp_form. sp_form has two methods: getElement(), and getControl(). It also has an object property called variables, containing each variable's name, label, sys_id, element, and parent element. If you like, you can assign these methods to g_form so you can use g_form.getElement() again, by adding the following line into a Catalog Client Script: 

Object.assign(g_form(sp_form));

You would have to do this from a Catalog Client Script, since our "script library" will not have access to g_form when it loads. 

Pro Tip: If you write a Catalog Client Script to do this, set it to run on load, and only on "Mobile / Service Portal". This way, if your catalog items are viewed in the CMS or directly in sc_cat_item.do, your DOM manipulation scripts won't run (because you can already access the DOM outside of the Service Portal)!

Now that our magical new UI Script is defined, we just have to tell the specific Service Portal we're using, to load it as a JS library.

First, navigate to the Service Portal list, by going to Service Portal > Portals from the application navigator, or heading straight to the sp_portal table. From there, select your Service Portal from the list. For our example, I'm going to use the sp Service Portal. 

Inside the portal record, find the Theme field, and click on the reference icon to be taken to the theme your portal is using. 

Once on the Theme record, scroll down to the related lists, and select JS Includes. If the related list isn't there, add it; then, click the blue New button. 

Give your JS Include a name like Enable DOM in SP, make sure that the Source is set to UI Script, and select your UI Script in the UI Script field. 

Finally, submit your JS Include record.

You may have noticed that the query we passed into client.open() contained a reference to a script include, so we'll need to create that next. To do that, navigate to System Definition > Script Includes, and click New. Name it CatItemVariables and set it to Client Callable; then make sure it's available from all scopes. 

Here's the script you can copy and paste in into the Script field: 

var CatItemVariables = Class.create();
CatItemVariables.prototype = Object.extendsObject(AbstractAjaxProcessor, {

    getSysIdsForQuery: function(itemSid, fieldName) {
        if (!fieldName) {
            fieldName = 'sys_id';
        }
        var a = [];
        var item = GlideappCatalogItem.get(itemSid);
        var variables = item.getVariables();
        while (variables.next()) {
            a.push(variables[fieldName].toString());
        }
        return a.join(',');
    },

    type: 'CatItemVariables'
});

Once you've got those two records created and the UI Script associated with the Service Portal, that's all there is to it! You can now access document, window, sp_form.variables, sp_form.getElement(), and sp_form.getControl() inside of any catalog client script which executes inside the service portal!

Pro Tip: Want your Catalog Client Script to be able to access things like getElement() whether you're in the Service Portal or not? Just use the following code! 
if (typeof sp_form !== 'undefined') {   //if sp_form exists
    Object.assign(g_form, sp_form);     //map its prop/methods onto g_form
}

Update

Hello, future readers! I just wanted to add that in a recent patch, ServiceNow have actually exposed a lot more access. Unfortunately, they haven't really documented it anywhere, but you now have a few options for almost complete DOM access. The only thing that's missing (which the scripts above restore) is access to the g_form APIs, and being able to get variable sys_ids (which will be necessary because variable document elements are no longer given IDs that match their variable names. Instead, they're listed with name properties that match "IO:" plus their sys_ids. 

First, the method: It's as simple as using the this object. Here are some usage examples: 

//Gets the full URL from the browser
this.location.href; 
//Gets the query parameters, starting with the "?"
this.location.search; 
//Access the actual document object!
this.document; 
//Gets the HTML Element containing the variable with the specified sys_id.
this.document.getElementsByName('IO:' + variableSysID)[0]; //returns an array, get element 0
//Gets the value of the variable. Works for text fields and such, doesn't seem to work for checkboxes.
this.document.getElementsByName('IO:' + variableSysID)[0].value;

So this is all pretty great. The only problem is: How do you get the variable sys_ids? You can't access them by name in the service portal, and there isn't a built-in good way to retrieve the sys_ids of every variable on a given catalog item from the client.

In order to do that, we could utilize the REST API call, and our "CatItemVariables" Client-Callable Script Include that we created above, just like we do in the UI Script. However, since we have access to the GlideAjax API in Catalog Client Scripts, it's probably best to use that.
First though, we'll need to add a new method that we can call using GlideAjax; the existing method is written specifically for returning a query. 

First, I've written a new Script Include (of the non-client-callable variety) to handle getting a list of sys_ids, and constructing the final object for us:

var CatItemVariableUtil = Class.create();
CatItemVariableUtil.prototype = {
    initialize: function() {
    
    },
    
    getSysIDs: function(itemSid) {
        var a = [];
        var item = new GlideappCatalogItem.get(itemSid);
        var variables = item.getVariables();
        while (variables.next()) {
            a.push(variables.sys_id);
        }
        return a.join(',');
    },
    
    getQuestionText: function(itemSid) {
        var a = [];
        var item = new GlideappCatalogItem.get(itemSid);
        var variables = item.getVariables();
        while (variables.next()) {
            a.push(variables.question_text);
        }
        return a.join(',');
    },
    
    getLabels: function(itemSid) {
        var a = [];
        var item = new GlideappCatalogItem.get(itemSid);
        var variables = item.getVariables();
        while (variables.next()) {
            a.push(variables.label);
        }
        return a.join(',');
    },
    
    getVarData: function(itemSid) {
        var itemVariables = {};
        var item = new GlideappCatalogItem.get(itemSid);
        var variables = item.getVariables();
        while (variables.next()) {
            var vName = variables.getValue('name');
            var vQuestion = variables.getValue('question_text');
            var vSid = variables.getValue('sys_id');
            var vType = variables.type.getDisplayValue();
            var vDefaultVal = variables.getValue('default_value');
            var vOrder = variables.getValue('order');
            
            if (itemVariables.hasOwnProperty(vName)) {
                gs.error('Duplicate variable names somehow located on catalog item ' + itemSid, 'getInfo method of CatItemVariables SI');
            } else {
                itemVariables[vName] = {
                    name:          vName,
                    question_text: vQuestion,
                    sys_id:        vSid,
                    type:          vType,
                    default_value: vDefaultVal,
                    order:         vOrder
                };
            }
        }
        return itemVariables;
    },
    
    type: 'CatItemVariableUtil'
};

Now, we can add a function to our client-callable script include, which now looks like this: 

var ClientCatItemVariables = Class.create();
ClientCatItemVariables.prototype = Object.extendsObject(AbstractAjaxProcessor, {
    
    getSysIdsForQuery: function(itemSid, fieldName) {
        if (!fieldName) {
            fieldName = 'sys_id';
        }
        var a = [];
        var item = GlideappCatalogItem.get(itemSid);
        var variables = item.getVariables();
        while (variables.next()) {
            a.push(variables[fieldName].toString());
        }
        return a.join(',');
    },
    
    getVarData: function() {
        var itemSid = this.getParameter('sysparm_item_sid');
        var ciVarUtil = new CatItemVariableUtil();
        var variableData = ciVarUtil.getVarData(itemSid);
        return JSON.stringify(variableData);
    },
    
    type: 'ClientCatItemVariables'
});

Finally, we can call this client-callable script include from a catalog client script like so: 

function onChange() {
    var ciVarUtil = new GlideAjax('ClientCatItemVariables');
    ciVarUtil.addParam('sysparm_name', 'getVarData');
    ciVarUtil.addParam('sysparm_item_sid', g_form.getUniqueValue());
    ciVarUtil.getXMLAnswer(function(itemVars) {
        itemVars = JSON.parse(itemVars);
        var requirementsElement = this.document.getElementsByName('IO:' + itemVars['Additional_software_requirements'])[0];
    });
}

Boy, that last line is a bit complicated, isn't it? And what's worse - that will only work on the Service Portal! We could create a second version of this script just in case we ever need to render this catalog item in the CMS/standard view, but that's not ideal.

Instead, let's see if we can be a little bit extra clever about how we handle this process of retrieving an element, so it'll handle both service portal and CMS. The code below works just like the code above, except that it works in both the Service Portal, and CMS, thanks to the getElement() function. 

function onChange() {
    var ciVarUtil = new GlideAjax('ClientCatItemVariables');
    ciVarUtil.addParam('sysparm_name', 'getVarData');
    ciVarUtil.addParam('sysparm_item_sid', g_form.getUniqueValue());
    ciVarUtil.getXMLAnswer(function(itemVars) {
        itemVars = JSON.parse(itemVars);
        var requirementsElement = getElement(itemVars['Additional_software_requirements']);
    });
}

function getElement(itemVar) {
    var doc = this.document ? this.document : document;
    return doc.getElementsByName('IO:' + itemVar.sys_id)[0];
}