📘

Note: This is a Premium feature

How to: Quality Assurance (Random Sampling) within Unit21

What is QA?

Quality Assurance is a process wherein a sample of alerts, cases, or filings are reviewed after the work has been completed, as a way to measure and improve upon the quality of a given transaction monitoring program.

Samples are generally selected on the basis of two factors:

1. The tenure and history of particular analysts
2. The capacity of the quality team

Longer tenured analysts with a good history of high quality work are sampled at a lower rate than analysts with a shorter tenure, or with poor performance in prior quality reviews.
The capacity of the quality team is an additional factor in sampling alerts, cases, and filings. Specifically, managers of quality teams want to ensure that their team is able to review the articles selected.
The results of QA reviews are used for performance management and coaching.

How do Fraud, Risk, and AML teams do this today?

A QA manager or team lead will generally be responsible for generating a sample on a certain cadence. We have seen people have weekly, monthly, or quarterly cadences, depending on the company and the purpose of the QA cycle.

The process usually looks something like this:

1. QA team lead will export alerts, cases, or filings (aka articles) from Unit21 into an Excel spreadsheet.
2. QA team lead, with the help of a data analyst, will use the randomize function in Excel on the rows in the spreadsheet, and then go down the list manually selecting articles based on assignee, ensuring that the sample includes the desired distribution (e.g. sampling Bob at 30% because he is new, but Alice at 5% because she has had good performance and has been on the team for a while).
3. QA team lead will send the spreadsheet to their team of QA analysts, who will click the links in the spreadsheet to access the articles within Unit21. Alternatively, the QA analysts will use the spreadsheet to copy and paste the article ID into Unit21 search, to look up the article.
4. QA analyst will fill out additional columns in the spreadsheet as they review each article, marking whether or

How can I do this in Unit21?

Setting up your QA workflow

1. Make sure you have the “Access quality cycles” and “Manage quality cycles” permissions set up.

   

2. Create a QA Team, with all your QA analysts, including the QA team lead or manager.

   

3. Create a QA queue for the article type that you are looking to sample. For example, if you are sampling alerts, create an alert queue called “Alert QA Queue”, or something to that effect. Mark it as “TM”, with no rules, and add the QA Team to this queue.

   

4. Mark this queue as “restricted.” This way, agents without access to the queue can still see that the articles within it exist (e.g. in Prior Activity on an alert, the Risk tab on the Entity page, in Search, etc.), but will not be able to click into it or alter its contents while it is in QA.

   

5. Create your QA checklist using Investigation Checklist for the article type you want to QA (e.g. if you are QA’ing alerts, create an Alert Investigation Checklist, if you are QA’ing cases, create a Case Investigation Checklist). We suggest creating two groups of questions - one for material errors, and the other for procedural errors. Bonus: If you want to use our AI Agent feature to automate some of these questions, make sure to make your checklist questions as detailed as possible.

   

6. Publish your checklist to the Alert QA Queue you created earlier.

   

7. Create a bulk workflow button to move samples into the QA queue. The button should be visible only to the “QA Analysts” team you created above, and marked as a “Quality” button. Additionally, it should change the status of the alert to “Mark In Review”, so that the contents of the alert (or case) are not editable, but the checklist remains editable, and comments on the narrative will remain available. Also, it should not require a completed checklist, should be enabled for bulk resolution, and be shown in all queues except the QA Queue. Finally, it should move selected alerts to the “Alert QA Queue”, and add a tag called “qa:alerts.” (See screenshots)

   

8. Create a second workflow button for “QA Complete.” This button should only show in the Alert QA queue, marked as a “Quality” button (so that the original disposition does not change), and only be visible to members of the QA Analysts team. It should be set up to “Close” the alert. Finally, it should require the investigation checklist to be complete, and add a tag called “qa:complete”. (see screenshot).

   

9. Now let’s create the actual quality cycle.

Creating a quality cycle

1. On the left navigation bar, select “Workflows”, and click on the “Quality” tab near the top right (see screenshot).

   

2. Create a meaningful description for your quality cycle. For example, you could call it “High Risk Alert QA: October 2023” if you are sampling high risk alerts from the month of October.

   

3. Select the article type you are looking to sample - this could be alerts, cases, or filings.

   

4. Once you select the article type, you will see the filter dropdown. These are the same filters as are available on the admin alerts page. Select the filters that are applicable to your sample. For example, you may want to sample from a set of alerts that were dispositioned in the month of October in the P0 queue. There is no limit to how many filters you can apply at this stage. Once you have selected the filters, they will appear near the filter selector as shown in the second screenshot below.

   

Pro tip: Use the filters to exclude anything with the tags “qa:complete” and “qa:sampled” in order to ensure you don’t sample anything twice!

5. Select how you want to sample the selection - based on last dispositioning agent, last assigned agent, or based on program. If you select “Last Dispositioning Agent”, it will generate your sample based on the agent who did the last action on the alert, case, or filing. If you select “Last Assigned Agent”, it will generate the sample based on the agent who was assigned at the time of the last action, even if it was actioned by someone else.

   

6. If you select “Last Dispositioning Agent” or “Last Assigned Agent”, you will be able to choose a different percentage for each agent such that the selection conforms to the filters you selected (e.g. 20% of Alice, 50% of Bob, 90% of Charlie’s alerts from October from the P0 queue).

   

7. On the other hand, if you select “Program”, you will get a single slider to select an overall percentage (e.g. 27% of all alerts done in October in the P0 queue).

   

8. Once you have finalized your sample, you can either save it as a draft, or can review the sample by clicking the blue button on the bottom right. The text on this button will dynamically calculate the total number of alerts that have been selected so you can make sure never to put your team overcapacity by oversampling.

   

9. Review your random sample to make sure everything looks right, and if so, click the checkbox on the top left of the table to select all the articles in the sample. Then, select the “Action” button on the top to view all your “bulk action” workflow buttons.

   

10. Click the “Move to QA Queue” button you created previously to move the sample into your QA queue, mark them as “In Review”, and append the QA checklist so that QA analysts can fill them out! Since this is a “Quality” button (as opposed to an “Investigation” button), the disposition, subdisposition, disposition notes, and dispositioned by will not be overwritten.

    

11. Now your sample is ready for review! QA analysts can now go into the Alert QA queue and fill out the QA checklist right in Unit21. Once they are done with QA, they can click the “QA Complete” button, which should only show up for them (ie. other teams cannot accidentally click it).

QA Reporting

1. Once QA analysts have completed their QA checklists, use the navigation panel on the left to open “Workflows”, and then click on “Investigation Checklists.”

   

2. Click on the three dots to the right of the “Alert QA Checklist” that you created above, and select “Export.”

   

3. Select your preferred file format, and select the checkbox to get an email notification once the export is complete.

   

4. Once you receive the email, simply download the results of your QA cycle! Pro tip: you can open the Reporting tab and click the magnifying glass on the right hand side to ask questions from your QA report using AI.