A Guide to Using Longitudinal Data Analysis for Improved Identity Threat Detection

Why Security Teams Are Using Longitudinal Data Analysis to Detect Anomalous Privilege Activity

With organizations managing a sprawling number of identities—both human and non-human—across on-prem, cloud, and SaaS environments, today’s attack surface has expanded significantly. Each of these identities presents an opportunity for a threat actor to exploit and gain access to sensitive system data and resources. As a result, identity security has become critical in securing today’s digital environment. One useful approach for detecting threats in the identity space is Longitudinal Data Analysis (LDA), the analysis of changes in user behavior across time. By analyzing temporal patterns of access and privilege changes, security teams can uncover risks that are not evident using strictly point-in-time analysis.

Depending on the field, Longitudinal Data Analysis may be called Panel Data Analysis, Event History Analysis, or Repeated Measures Analysis. For the purpose of this blog, we will refer to this approach as Longitudinal Data Analysis.

LDA represents a critical approach in identifying suspicious changes in the configuration and/or behavior of an account, IP address, session, or other entity over time. Rather than examining only the current state of an entity, such as a user’s level of privilege, LDA employs a variety of methods to combine past state with current state to identify anomalous activity or configurations. This form of analysis can identify unusual behavior that would otherwise go unidentified if only the current state was analyzed. Escalation of privilege, or privilege jumps, for example, may indicate account compromise, making detection essential in securing an organization’s IT infrastructure. LDA and its associated methods can be leveraged by security teams to monitor and identify unusual privilege changes, helping organizations mitigate the risks posed by account vulnerabilities.

This blog marks the first in a series that explores applications of LDA in identity security. In this initial blog, we will focus on how LDA can be leveraged to detect anomalous privilege changes, a common security risk in large organizations.

Understanding Longitudinal Data Analysis (LDA)

At its core, LDA is a technique for analyzing change over time. It is a variation on time series analysis that, rather than focusing on a single entity over time, is used for the analysis and modeling of many entities across time. A defining feature of LDA is the collection of repeated measurements per entity, allowing for the study of change over time. This approach is well-suited to the identity security setting, where many identities and their behaviors/configurations need to be observed over time to detect when suspicious changes in behavior occur. For instance, these repeated measurements could take the form of a daily record of privilege level, number of failed authentications, and number of IP addresses per account. LDA utilizes various approaches to data modeling and structuring to allow for the temporal analysis of changes.

Examples of these temporal changes include:

Lag and Cumulative Lag Variables: These variables represent previous states or events. They are often used to identify when current behavior deviates significantly from past behavior.

(Start / Stop) Time: This represents the structuring of data into time intervals during which an event or state is active, such as when an account is in a low-privileged or high-privileged state.

Interval Data: This represents the structuring of data into (possibly equally spaced) time intervals where the state of each entity is recorded at various points in time.

Generalized Estimating Equations / Mixed Models: Statistical methods are used to account for dependencies within repeated measures or clustered data, such as the modeling of the expected privilege of each account within an organization. These methods are particularly useful as they can assume specific correlation structures (e.g. scenarios where an account’s current state is correlated with previous states). Later blogs in this series will describe applications of these models in greater detail.

Hidden Markov Models: An unsupervised statistical method for identifying an entity’s latent states at a given time—such as normal/anomalous—given its observed behavior.

Why LDA is Important for Identity Security

Identity security is inherently temporal. In a general sense, identifying unusual behavior requires both knowledge of the current state of an entity as well as its previous states. This involves modeling the state of an entity at the current time period conditionally to its state at previous time periods. For instance, identifying whether the current configuration or behavior of an account is anomalous often requires examining past configurations or behaviors. LDA provides a framework for achieving this.

A separate benefit to LDA is the ability to include subject-specific information within the model. For example, certain accounts might be expected to have higher levels of privilege than others; LDA approaches provide various methods of incorporating this subject-specific information into the model. The importance of this is illustrated in the graph below. Accounts A and B both have sudden privilege increases, or privilege jumps, at time 10. If we were to only compare privilege at times 9 and 10, both changes would be concerning; however, when Account B’s entire history is included, this increase in privilege is much less surprising. Inclusion of this subject-specific information is thus critical in identifying anomalous changes over time.

The comparison of privilege jumps in accounts A and B demonstrates how subject-specific information can help with identifying potential threats based on anomalous changes over time.

In modeling this behavior, we can use the function: ⨍(𝒴𝑡 |𝒴𝑡-1, 𝒴𝑡-2). This function is visualized in the graph below (adapted from Bishop, 2006). In this second-order model, the state (𝒴) at the current time period (𝑡) is modeled conditionally on the state in the previous two time periods: 𝒴𝑡-1 and 𝒴𝑡-2. This enables the model to utilize past behavior when predicting expected values at the current time period.

This second-order model shows the state at the current time period is modeled conditionally on the state in the previous two time periods.

Now, let’s apply this to identity security. A user may have standard access, but later gain excessive or unwarranted permissions, which could be exploited if left unchecked. Longitudinal data analysis allows security teams to see this evolution in access patterns, track trends in permissions, and flag any deviations that might otherwise go unnoticed. This approach is especially useful for identifying unusual changes in state, such as changes in account dormancy, privilege escalation, and lateral movement within a system.

Here are a few more examples of how LDA can be used in identity security:

Anomaly Detection in Login Behavior: Identifying unusual login patterns, such as logins from unfamiliar locations, devices, or times that deviate from a user’s historical behavior.

Suspicious Access Patterns: Detecting unusual access requests to sensitive resources, especially if they deviate from established user or group patterns, which could indicate account compromise.

Privilege Escalation Monitoring: Detecting incremental privilege accumulation or sudden privilege spikes.

Monitoring Unusual Behavior in Dormant Accounts: Identifying dormant or infrequently used accounts that may pose security risks if suddenly reactivated.

Detecting Lateral Movement: Observing patterns of network and login activity for a user to identify anomalous changes in behavior, such as a sudden increase in the systems being accessed.

Session Hijacking: Detecting when session activity for an account changes unexpectedly, such as changes in IP address, device, location, or user agent.

How to Implement Longitudinal Data Analysis in Identity Security

Applying LDA requires a systematic approach to data collection, analysis, and anomaly detection:

1. Data Collection

Collecting data over time is the foundation of LDA. In the context of identity security, this includes logging account activity, privilege levels, access requests, and other interactions with the identity management system.

To enable LDA, it is critical that both the current state and previous states are recorded. For example, if the goal is to identify unusual privilege changes, it is insufficient to record only an account’s current state. At a minimum, we would require the user’s current privilege level and previous privilege level, although more effective analysis can be performed with a more complete collection of state changes.

2. Data Structuring

As we’ve established above, Longitudinal Data Analysis is time-based—that means the data needs to be properly structured to preserve timestamps, or temporal markers, to allow it to ensure events are comparable and identify trends, anomalies, or behavioral shifts can be made visible.

There are several approaches to structuring data in LDA:

Repeated Measurements - Most commonly, data is structured such that each entity is observed at various points in time. Measurements at each time period are then recorded in separate rows. Representing each change of state in a separate row is the most comprehensive approach, but it can be computationally expensive.

Interval or Snapshot-based - An alternative approach that utilizes interval or snapshot-based data, where observations are taken at specific time periods. This approach is less costly, although it may lack the necessary precision for certain analyses.

Wide Format - The use of additional columns in the form of lag and cumulative lag variables is another method that allows combining information on the current state and previous state without necessitating multiple records per entity.

This table demonstrates how data is structured in the Repeated Measurements approach.

Time	Account ID	Privilege	privilege_lag1
t-7	7ab0d09	low	NA
t-6	7ab0d09	low	low
t-5	7ab0d09	low	low
t-4	7ab0d09	high	low
t-3	7ab0d09	high	high
t-2	7ab0d09	high	high
t-1	7ab0d09	high	high
t	7ab0d09	high	high

3. Feature Engineering

In LDA for identity security, we derive features that capture critical information about user behavior, access patterns, and privilege changes over time. Feature engineering transforms raw logs into meaningful signals that help detect anomalies, predict risk, and uncover hidden threats. This step is essential to enable models to learn temporal dependencies and behavioral baselines.

Some key types of features include:

Inactivity Periods: The length of time since an account was last active.

Account Context: Information regarding the intent of the account, such as whether it is a service account or an account intended for daily use by a person.

Privilege Level Changes: Any increases or decreases in an account's permissions.

Patterns of Access Requests: Frequency and type of access attempts over time, including contextual information regarding the location, timing, and method of access requests.

Risk Flags: Indicators such as the number of failed login attempts or unusual times of access.

Resources Accessed: Types and sensitivity levels of resources accessed by an account, highlighting any changes in access patterns that may signal risk.

4. Anomaly Detection

Through LDA, unusual patterns of account activity, changes in privilege, and dormant periods can be visualized and analyzed. By setting thresholds for unusual activity—such as unexpected privilege escalation (privilege jumps), erratic login behavior, or inactivity—security teams can effectively flag accounts or actions that warrant further investigation.

This detection process can leverage different analytical methods, including:

Rules-Based Detection: A traditional approach that uses predefined logic or thresholds to flag anomalies. For example, a rule might trigger an alert if a user logs in outside of business hours or if their privilege level changes twice within an hour. While this method is simple and transparent, it may miss subtle or novel threats that fall outside the defined parameters.

Supervised Anomaly Detection: This technique uses labeled historical data to train models that can distinguish between normal and abnormal behavior. It requires examples of both benign and malicious activity. Once trained, the model can identify similar patterns in new data. This approach is powerful when quality labels are available but may struggle with detecting previously unseen attack techniques.

Unsupervised Anomaly Detection: Ideal for detecting unknown or emerging threats, this method analyzes data without prior labels. It identifies deviations from learned behavioral baselines or statistical norms. Techniques like clustering, density estimation, or autoencoders can uncover subtle anomalies that might not be apparent through rules or prior examples.

By combining these approaches within a longitudinal analysis framework, identity security teams can move beyond static thresholds and gain a more adaptive, context-aware view of risk that is capable of detecting both known and novel identity-based threats.

Using LDA to Identify Suspicious Privilege Jumps

In this use case, we’ll apply LDA to identify suspicious privilege changes in service accounts. Service accounts pose a unique security risk because they often have elevated privileges that do not change over time. Often unmonitored, these accounts may have access to sensitive resources, increasing the organization’s attack surface. If compromised, these accounts can be exploited for privilege escalation attacks, and unauthorized actions can be performed due to those elevated access levels.

Here are the technical steps security teams can follow to identify suspicious elevation of privilege activity using LDA:

Step 1: Identify Service Accounts

Service accounts can be identified via rule-based or model-based approaches. Under the model-based approach, this could take the form of a model estimating the conditional probability of an account being a service account given its configuration and/or behavior.

Step 2: Detect Privilege Changes

Next, we look for increases in privilege levels for these service accounts. This requires a quantitative or ordinal value representing the privilege level of each account as well as a history of each account’s privilege. A simple yet effective detection might identify any privilege increase on a service account as concerning. Non-service accounts, having potentially more variability in their privilege changes due to just-in-time (JIT) access or other organizational policies, may require more complex model-based approaches that incorporate additional lags, account-specific effects, and/or estimate a particular correlation structure.

A model to identify service accounts with unusual privilege changes can be represented in the manner shown in the graph below. Additional features that might be useful in modeling expected privilege level, such as the number of days since last activity, are represented by the vector 𝑿.

An example of a model that can be used to identify service accounts with unusual privilege changes.

The following table describes a hypothetical scenario in which a privilege increase is observed on a service account that has been inactive for a prolonged period. Additional information, such as the specific privileges, groups, or roles added to this account, can be included to provide additional context for the analyst. In this case, a simple comparison of the current privilege level with the previous privilege level can identify the anomalous change in privilege.

The asterisked selection on this table shows a privilege jump on a service account that has been inactive for a prolonged period.

time	accountId	isServiceAccount	privilege	privilege_lag1	daysSinceLastActivity
t-4	7ab0d09	True	low	low	91
t-3	7ab0d09	True	low	low	92
t-2	7ab0d09	True	high	low	93
t-1	7ab0d09	True	high	high	1
t	7ab0d09	True	high	high	2

Step 3: Alerting and Logging

For any service account showing a significant increase in privilege, an alert is generated for the security team to investigate further. The frequency of alerts can be modified by tuning the thresholds used to identify service accounts and relevant privilege changes.

Additional Recommendations

LDA not only helps in identifying unusual privilege changes, but also empowers security teams to improve overall security practices:

Automate Alerts for Anomalies - By setting up automated alerts, security teams can be notified in real-time when service accounts show privilege jumps or other suspicious activities.

Regularly Review Privileged Accounts - Periodic reviews of privileged accounts can prevent security risks. Accounts that are inactive for long periods should either be deactivated or have their privileges reduced. The same concepts used here in identifying privilege change could also be used to identify dormant accounts that have suddenly become active.

Establish Thresholds for Anomalies - Defining threshold levels for privilege jumps and dormancy periods, adjusting as needed based on organizational policy, enables fine-tuning and the reduction of false positives/negatives.

Conclusion: Why Longitudinal Analysis Belongs in Your Identity Security Strategy

Longitudinal Data Analysis is an invaluable, albeit often underutilized tool, for identity security. Unlike point-in-time-based assessments, LDA enables security teams to track and analyze user behavioral trends, detect subtle anomalies, and spot privilege shifts as they occur over time. The detection of privilege jumps in service accounts is just one example of LDA's potential.

When woven into a broader security strategy, LDA equips organizations with the context and continuity needed to truly understand how identities evolve and behave across their environments. In doing so, it strengthens defenses against modern threats—many of which thrive in the blind spots between traditional detection methods.

At BeyondTrust, we integrate LDA-driven insights into the BeyondTrust Pathfinder Platform to help you see and secure the Paths to Privilege™ in your environment. By gaining visibility into behavior over time—not just at a single moment—you can uncover hidden risks, reduce attack surfaces, and respond to threats with greater speed and confidence.

LDA-based techniques are used within a variety of detections, including anomalous service account behavior, unusual account creation, and potential session hijacking—with additional detections currently in development. In one instance, our system detected a service account deviating from its normal behavior and accessing unusual applications with a rare user agent. The plot below shows anomaly scores for this account over time, clearly highlighting suspicious activity in May 2024. Our model successfully detected this anomalous activity, and we were able to notify the customer within 24 hours. It was later confirmed that the activity was part of a scheduled penetration test.

The above plot shows successful detection of suspicious privilege escalation activity in May 2024.

Ready to take the next step? Explore how BeyondTrust Identity Security Insights® can help you harness the power of LDA to uncover hidden privilege risks and harden your identity landscape by taking our free Identity Security Assessment.

About the Author

Darren Maynard

Sr Data Scientist

Darren Maynard is a Senior Data Scientist at BeyondTrust with over a decade of experience in data science and machine learning. His work has focused on solving complex problems involving large datasets and rare event detection in both industry and government. He holds a master’s degree in statistics and previously worked within the defense sector.

Phantom Labs™

BeyondTrust

BeyondTrust Phantom Labs™ believes the best way to fully understand cybersecurity threats is to work closely with our customers and partners, conducting real world research into the attacks that matter most to them. By dissecting emerging attack methods and exploitation techniques of threat actors, as well as conducting novel research, the team’s mission is to help organizations defend against identity threats.

Watch Product Demos

Buyer's Guide for Complete PAM

Gartner® Magic Quadrant™ for PAM

See what customer success looks like

Find a Partner

Leader in Privilege-Centric Identity Security