Detecting Hidden Privilege with Machine Learning: Anomaly Detection in BeyondTrust’s True Privilege Graph

Of the 236 paths to an Azure Directory Role, the large majority are relatively direct paths, consisting of one or two hops. In these paths, the user was directly assigned the role, has assignment through group membership, is eligible for the role through Entra PIM, or has a path to the role through Microsoft Entra Connect Sync. However, there are four paths here that seem particularly interesting as they only have a single occurrence.

Alternatively, for the same underlying data, we can model the number of steps between the source and destination nodes. Figure 3 displays the distribution of path lengths. Though the pair of three-hop paths might be concerning, we will focus on the single four-hop path.

In the four-hop case, we have a path that is indirect compared to the usual one- to two-hop paths. It involves (1) an Active Directory user (2) synced to Entra ID (3) that has ownership permissions over an Entra ID application (4) with a path to a privileged directory role. This is more obvious when viewed as a graph, as shown in Figure 4.

To summarize, the model finds this path unusual for two reasons. One is its low observed probability (0.004), occurring in only 1/236 paths to an Azure Directory Role. The second reason is its path length, which is 2.64 times greater than the mean path length.

Contrary to our expectations, our analysis found that very long paths are not necessarily anomalous. For instance, several nine-hop paths from users to secrets were not flagged as anomalies. Because the nine-hop paths were much longer than the typical two to three-hop paths, our initial instinct was to override the model. However, the data showed that conditional on the destination node, nine-hop paths were actually common. This can be seen in Figure 5 where we observe a number of paths having seven, eight, and nine hops.

To make this clear, nine-hop paths are very rare when all destination nodes are modeled together, but common when we compare only within the same destination node. This finding was validated after consulting with our security researchers who verified that nine-hop paths are expected, given the limited number of ways paths to secrets can occur. Additionally, as the privilege graph is developed further, the number of long paths will continue to grow—resulting in an even greater need for models capable of distinguishing long paths that are truly rare, from those that are common.

Path length can be modeled in a number of ways. A cumulative distribution or right-tail p-value approach could be used to identify extreme values. Table 1 shows a simple example where paths of three to four hops can be considered anomalous due to low probability of seeing those lengths or greater, as indicated by right-tail p-values of 0.005 and 0.001.

Path length could also be compared to a particular quantile of the associated distribution. For instance, all paths where Observed Path Length / Quantile (Path Length, 0.9) > = 2 could be deemed anomalous. This effectively filters anomalies to only those paths that are at least twice as large as the 0.9 quantile (the path length value that is greater than 90% of path lengths). For the secrets case with distribution given in Figure 5, the nine-hop paths are scored as 9/8 = 1.125, failing to meet the minimum threshold of two to be considered anomalous. Additionally, we note that in the case of distributions where only a small number of distinct path lengths have occurred, larger quantiles may not be well-defined. Under this scenario it may make more sense to use a smaller quantile, sample average, or the p-value approach mentioned earlier.

A regression approach could also be utilized, allowing for the inclusion of covariates such as account type or source node domain type. Anomalies under this approach would be those paths with large deviations between observed and predicted path lengths. An encoding of destination node into a series of indicator variables or a hierarchical model would be useful methods for avoiding the estimation of many separate models, an issue with the stratified modeling approach mentioned earlier.

Given the discrete nature of path length, the natural choices for probability distribution consist of the Poisson (with a variable dispersion parameter) and negative binomial. However, in our analysis, there were numerous instances where path length was nearly binary or limited to a small number of distinct values, making the binomial distribution more appropriate.

Initial modeling efforts focused on modeling unexpected access to a destination node given a source node’s entire privilege configuration. Several of the accounts flagged under this approach as having unusual privilege configurations corresponded with IT administrator and software engineering roles. This makes sense, but is not particularly interesting from a security standpoint. It suggested that a model including covariates representing account-specific features might be effective in separating users in IT-related roles from those in non-IT roles.

The updated model with account-specific covariates found a number of external accounts, such as Entra ID Guest accounts, with access to Azure secrets and Azure RBAC role definitions. This is visualized in Figure 7 for the Azure RBAC case, where the model has identified a single external Entra ID guest account out of over 1,000 accounts with access to Azure RBAC role definitions.

We encountered several challenges during this analysis, particularly due to problems stemming from limited data. Difficulties with model convergence were common due to small strata causing separation-related issues. Regularized models, utilizing penalized log-likelihoods, were used to resolve issues with model convergence. Limited data also constrained the possible complexity of models, making interactions and inclusion of nonlinear terms difficult. In the most extreme cases, certain destination nodes were so rare that modeling with any covariates was problematic.

Simple workarounds in this setting might be to group together multiple categories with insufficient data, or to label all occurrences as anomalies. More sophisticated approaches that were not explored here, though worth considering, include Firth’s method for rare events and Bayesian models using informative priors to account for limited data.

Required minimum thresholds on model performance were set, as certain response variables could not be well-modeled with the data available; models that fell below this threshold were excluded. We also explored count-based models that structured the response as the number of destination nodes, rather than a binary variable. However, in certain cases, this was problematic due to many response variables having a limited number of distinct values.