How to Use ControlUp’s REST API Actions to Send Incidents to PagerDuty

One of the very first blog posts I wrote during my tenure here at ControlUp, “way back” in 2020, is still a popular blog post: two or three people visit it and read it every day.

And I think one of the reasons why it remains popular is because it’s a useful article to learn how to extend the capabilities of our platform into third-party solutions.

But a lot of things have changed since June 2020. Besides adding various new capabilities and ways to improve the digital workplace experience, we added new ways to integrate with third parties, including:

• Out-of-the-box integrations for expanding our DEX capabilities to Windows 365 Cloud PC workloads and ChromeOS devices;
• Automated ticket creation for ServiceNow (in our physical endpoints offering and for synthetic testing);
• The ability to monitor the experience of unified communications platforms Zoom and Microsoft Teams
• And, more recently, integrations for Okta and Entra ID in support of our User Risk Analytics feature in Secure DX.

But in this blog post, just like in the post from 2020, I’m going to talk about a more “freeform” of third-party integration: using RESTful APIs and webhooks to provide unlimited ways to interact with other platforms. And once again, I am going to use a practical example to walk you through things.

The last time I used ServiceNow as an example. This time I will use a different, but also popular, tool in the industry: PagerDuty. Now keep in mind that if you want to build a different integration, the approach will be very similar. However, the details on how to set up the third party will vary, as do some of the configuration details. (For more information on setting up a third-party, check the vendor’s documentation or ask in the ControlUp Community. I’ll note varying configurations details where needed.) This post will give you enough direction to get started to get other, similar integrations done as well.

Figure 2 – PagerDuty Services interfaceControlUp has various ways to use webhooks, including sending notifications triggered by events that impact the digital experience on their endpoint device, the apps they run, as well as failed tests from our synthetic testing tool. But in our scenario, we will be using a ControlUp for VDI & DaaS trigger with a “Send a RESTFul API request” follow-up action (introduced with our v8.6.5 release) to send an alert through PagerDuty to an on-call engineer (I will be playing the role of engineer).

To set up this integration, we are going to complete the following four steps:

1. Set up a new “service” in PagerDuty that will receive the calls coming from the ControlUp Platform
2. Create a trigger in ControlUp
3. Configure the follow-up action to send the information to PagerDuty
4. Test the integration

If you are more of a visual learner, watch the video where I go through the set-up process. If you prefer to read the steps, continue reading below.

Step 1 – Setting up PagerDuty

I am not going to get into all the details on how to set this up, because PagerDuty has extensive documentation on this, but basically there are 2 parts to set up: the service and the integration.

In PagerDuty, a “service” represents a specific application, component, or team that you want to manage incidents for, while an “integration” is a connection between your service and another external tool (in this case ControlUp), allowing alerts or events from that tool to be sent to your PagerDuty service and trigger incidents if necessary.

Your service settings will normally be tailored for your organization, with assigned on-call engineers, how they get notified (e.g. call, text, through the app etc.), schedules, priorities and urgency settings. For the integration, we will use PagerDuty’s Event API v2.

The new service wizard in the PagerDuty UI will ask you for:

• A name and optionally a description for the service. In my example I am using “ControlUp for VDI & DaaS (Tech Marketing)” as the name
• To assign an “escalation policy”
• To select an integration for which, as mentioned above, we will use the Event API v2 integration

Once the service is created, you will see a tab for Integrations, under which you will find the Events API v2 integration. When you expand the item, you can change the name if you want, and on the right side you will find some important information that we will need in Step 3.

Step 2 – Creating a ControlUp Trigger

For the remaining steps, we will mostly work inside the ControlUp Management Console, our Windows app for configuring a part of the ControlUp VDI & DaaS settings.

After launching the Console, select “Triggers” (1) from the top menu bar and select an existing trigger you want to configure the PagerDuty follow-up action, or, as we will in our example, click on “Add Trigger” (2).

Set up the trigger to meet your requirements as described in the ControlUp documentation.

In our example we are going to use the built-in “Process Ended” trigger (3).

During the trigger creation, I will set a filter for the process name, in my case excel.exe. (This is just for demo purposes and probably not the best real-life example, unless you like getting lots of notifications.)

Select a scope and schedule that is appropriate for your environment, in my case I am leaving it to the defaults.

Step 3 – Configure the trigger follow-up action

When you get to the step for Follow-up actions, click on Add and select “Send a RESTful API request”.

We will come back to the URI field in a minute, but we will first create a new template. To do so, from the Template drop-down menu select “<Add new template>”.

In the “Add Template” dialog box, give the Template a name, for example “PagerDuty notification”.

Leave the “HTTP Headers” section blank (this might be required for other integrations).

And in the Body section we are going to use the information from the Events API v2 trigger from PagerDuty which I alluded to already in Step 1 above.

You can of course write the payload from scratch, but I find it easiest to copy and paste the cURL example provided by PagerDuty and remove the parts we don’t need.

Please note: You only need the payload data starting with { and ending with } from the provided cURL example.

But if we leave the payload unchanged, every alert will look the same due to the hardcoded “Test alert” in the sample code. So, let’s make some improvements there as well.

To ensure that the contents of alerts send to PagerDuty have enough actionable information for an on-call engineer to determine the best way of action (or notact), you can include additional information about the event that happened in the payload. For that we can use ControlUp “template variables”.

For our example we will change the “summary” field to use the string value “Process $(ProcessName) has ended in session $(SessionID) for user $(UserName) at $(TimeStamp)”.

Modify the severity to match the best level, e.g. “warning”. And if you want you can change the source to “$(ReportedBy)” (which is also a ControlUp variable). The eventual full Body code should look something like this:

{
  "payload": {
      "summary": "Process $(ProcessName) has ended in session $(SessionID) for user $(UserName) at $(TimeStamp)",
      "severity": "warning",
      "source": "Reported by $(ReportedBy)"
  },
  "routing_key": "<insert your own key from PagerDuty>",
  "event_action": "trigger"
}

Once you are back at the Follow-up Action configuration screen, make sure that the just created Template is selected.

The final part we need to configure is the URI field, which we can again get from the PagerDuty Integrations tab we were previously.

Copy that URL into the URI field in the ControlUp Follow-up Action dialog box to look like this:

Step 4 – Testing the integration

While still on the Follow-up Action dialog box we can do a test to make sure that we formatted everything correctly. Do so by clicking on the Test button. If we did things right, a 202, Accepted result should be presented.

At the same time the incident should be visible in the Activity tab for our PagerDuty service.

Since this was a test, and no actual process was ended, the variables will all be blank and not be presented in the UI, hence the odd “Process has ended in session for user at” Title of the incident.

To really test this, we will need to complete our trigger and force an end process event to happen.

Go back to the Trigger creation dialog in the ControlUp Management Console  and click OK. On the final step give it a friendly name so it’s easier to find back later. I am using “Send REST API call to PagerDuty when excel.exe process ends”.

Save the Trigger by clicking Finish and click on OK to close the Trigger Settings window.

The final “real” test is straightforward: on a monitored machine (matching the scope for the trigger we just configured) launch Excel and then close Excel. This should match the condition that the excel.exe process ended and trigger the alert.

And that is all there is to it: 4 steps (6 steps less than my previous blog post from 2020).

If you are an existing customer and want to learn more about how you can integrate with the tools you use, contact your customer experience account team or check out the ControlUp Community for assistance.

Not yet using ControlUp and want to learn more? Schedule a demo meeting with us or get started yourself today with our fully functional 21-day trial.

Joel Stocker

Joel Stocker is a technologist's technologist. An industry veteran with over 25 years of experience in End-User Computing (EUC), he's held numerous technical field sales and product roles at Citrix. Never one to settle, Joel always seeks to unlock the super powers of ControlUp technology and expand its usefulness for our customers. All of this, paired with a genuine love of technology and the ways it can change business (not to mention incisive wit and witticism) helps him help our customers and partners improve their EUC infrastructure and deliver stellar end-user experiences.