Best practices for tuning transaction monitoring thresholds

We recently migrated to a new TMS and are in the process of setting our alert thresholds. The vendor defaults generate a massive number of false positives (~95%), which is drowning our investigations team.

What approaches have you used to tune thresholds effectively? We're considering:

• Back-testing against 12 months of historical SARs
• Peer-group segmentation (retail vs. commercial vs. correspondent)
• Statistical models to set dynamic thresholds per customer risk tier

Any experience with above-the-line / below-the-line testing methodologies? How long did your tuning project take, and did your regulator have any concerns with the approach?

One thing I'd add: make sure you involve your BSA/AML officer in the sign-off process. We made the mistake of treating threshold tuning as a purely technical exercise, and our auditor flagged the lack of senior management approval.

Also, consider maintaining a tuning log that records every change with before/after impact metrics. It makes exam preparation much easier.

Has anyone used machine learning to supplement rule-based thresholds? We're exploring an ML overlay that scores alerts based on historical disposition data. The idea is to prioritize the investigation queue rather than eliminate alerts entirely. Curious if regulators are comfortable with this approach.

We actually implemented an ML-based prioritization model about 8 months ago. FinCEN and our state regulator were both fine with it as long as we didn't use it to suppress alerts entirely. The key was framing it as a triage tool, not a filtering tool. We still review all alerts — ML just determines the order.

Our SARs-to-alerts conversion rate actually improved by 15% because analysts were looking at higher-risk alerts first.