Ever wonder what makes your anti-money laundering (AML) system a beautiful model in the eyes of your regulator? It starts with periodic independent model validations from a qualified consulting firm such as RSM. At RSM, we know the Regulator’s expectations of model risk management. Our industry-leading team of AML professionals has a deep understanding of model risk management and is well-versed in the key elements of regulatory bulletin OCC 2011-12 (SR Letter 11-7), as well as applicable elements of the FFIEC BSA/AML Examination Manual. We provide independent validation testing of models and systems, including suspicious activity monitoring, customer risk scoring and sanctions screening. We have experience with many of the AML and the Office of Foreign Assets Control (OFAC) systems, including BAM, Patriot Officer, Verafin, Actimize, SAS-AML, Fiserv FCRM, Yellow Hammer and Bridger.
The importance of model validations was again underscored by a recent FinCEN enforcement action. In the enforcement action, FinCEN determined that the financial institution “failed to have its automated transaction monitoring system validated by a suitable independent individual (or entity).” The regulator required the financial institution to implement “statistically valid processes to validate and optimize monitoring system settings and thresholds, and to measure the effectiveness of the automated system and individual scenarios, where appropriate.” In another case, a consent order was recently issued against a financial institution that included “a requirement for an independent third-party validation of the models used for the BSA/AML monitoring systems in order to ensure that all accounts and transactions are captured, and the systems are adequate to detect potentially suspicious activity.” The message to financial institutions is clear: implement a model risk management program that includes qualified periodic independent model validations.
An effective model risk management program starts with AML model methodology and governance documentation that provides critical foundational elements for effective model risk management. An independent model validation should evaluate this documentation to understand management’s implementation and usage of the AML system, ensuring that the conceptual design is adequate in mitigating AML risks. System access and change management procedures are other important elements of AML systems. A validator should evaluate staff roles and the level of authority granted to determine if the right people have the right levels of access. Banks should have change management protocols and quality assurance controls involved with important system changes and updates, including alert settings and thresholds. Another important component of model governance is the ongoing system monitoring and reporting. A model validation should review monitoring and reporting to determine if the important aspects of the AML software operation are being reported to senior management.
Data is critical for models to effectively monitor activity. An independent validation should ensure the completeness and accuracy of data importation into the AML system by performing a detailed analysis of core system transaction activity and data extracts against activity imported into the AML system. The validation should evaluate the quality of the core data and assess data schema, transaction coding, data extraction and any transformations that occur. Additionally, the validation should evaluate the accuracy of customer demographic information by selecting a sample of customers for testing, as most AML systems have targeted customer monitoring alerts.
Customer risk scoring is a required component of AML programs for covered institutions, and it is critical for risk-based monitoring. A financial institution should adequately document the rationale supporting the customer risk scoring methodology, ensuring the appropriateness of the risk scoring factors and measurement calculations. An independent model validation should evaluate the types of factors used and the weightings that different factors receive. In addition, it should validate that the customer risk scoring system is accurately calculating the risk scores and ratings.
A qualified validation will evaluate your monitoring system to determine if it is effectively monitoring the risk. The financial institution should use its BSA/AML risk assessment as a foundation for understanding the types of activity that should be detected by the AML system. The validation should match the system alert design with the risks and evaluate the adequacy of the documented rationale used for supporting the rules in production and current alert settings. An independent validation may replicate alert logic and rules by creating control models, and conduct simulated or parallel testing to verify the completeness and accuracy of the alerts generated for a specified period of time.
An independent validation may use both qualitative and quantitative methods during the evaluation of the suspicious activity alert effectiveness. It is important that the output from the AML system is valuable and targeted at detecting potential suspicious activity. A validation will evaluate alert volumes relative to transaction volumes and review alert-to-case and case-to-SAR volumes to gain an initial understanding of the system output. A qualified validation will identify opportunities to modify alert parameters through statistical analysis and other methods that result in fewer false positives to enhance the system’s efficiency, while retaining the ability to adequately identify suspicious activity. Depending on the scope of the engagement, the validation will also help institutions with performing above- and below-the-line testing.
The OFAC sanctions screening systems are considered models as well, and should be validated accordingly using a similar risk-based approach as used for AML models described above. The OFAC model validation should evaluate the conceptual soundness and rationale documentation of the sanctions screening model, including search logic and filtering techniques. It should assess the completeness, accuracy and timeliness of the sanctions lists used for screening, and evaluate the data integrity of the feeds into the model. The validation should evaluate the filtering operations to validate they are working as intended and that they align with the OFAC risk assessment. Based on the scope of work, the validation may also assess the adequacy and effectiveness of alert clearing and recordkeeping processes.
RSM can assist your financial institution with AML or OFAC system development and implementation, as well as model validation assessments and testing. We have the right leadership and experienced consultants that continuously monitor the regulatory landscape to assist our clients in improving their AML programs, systems and risk management activities, as well as avoid regulatory scrutiny. We provide our professionals with relevant training, work programs and insight, so they are able to help clients stay ahead of evolving issues.
