Database Size and Resource Consumption
Data Retention Considerations
The Audit Event database and Microsoft SQL Server Reporting Services database that supports Privilege Management Reporting can be hosted and scaled independently.
It's important to identify the length of time that Privilege Management audit event data must be retained in the Privilege Management database as it drives resource utilization projections, and initial allocation.
Privilege Management Reporting is designed to report on activity in recent time, not as a long term archival data storage solution.
- BeyondTrust provides a database purge utility that may be used to purge data manually, or automatically on a configured period to ensure database growth is capped.
- Unlimited database growth reduces query execution performance, and increases resource utilization for queries.
Before purging large sets of data, please ensure your SQL Transaction logs can grow to accommodate. It may be necessary to delete data in stages when setting this up for the first time.
For more information to facilitate your decision making regarding retention time in the Privilege Management database, please see the following:
- For a description of the views of data exposed in Privilege Management Reporting, see the Reporting Dashboards Guide.
- For a description of the events audited by Privilege Management, see Auditing and Reporting - Events in the Administration Guide.
- For a description of the Workstyle parameters, see Workstyle Parameters in the Administration Guide. You may consider these as the fields that are collected in the audit events, eventually stored in the Privilege Management Audit Events database.
The Audit Event database must be sized to accommodate substantial data volume, matching the number of clients generating audit data, and the desired retention period.
Database storage requirements may be estimated roughly using the following calculation:
Number of hosts × Number of events per host per day × 5Kb per event × Number of retention days
10,000 × 15 × 5 × 30 = 22,500,000Kb, or 21.5Gb
A typical event volume is 10-20 events per host per day and varies based on Privilege Management Reporting auditing configuration, user job function (role/Workstyle), and user activity patterns.
Database resource utilization (CPU, Memory) is highly variable depending on the hardware platform.
Discovery: Between 40 – 60 events per machine per day
(4.6K per event (based on real world data))
Average total: 67.06 GB
Production: Between 2 – 10 events per machine per day
(4.6K per event (based on real world data))
Average total: 5.66GB
If the number of events per machine per day is raised to 15, then the average total increases to 16.99 GB.
- Volume of inbound audit event records: As noted above, the number of events per hour may be estimated following simple calculations. The audit event records are bulk inserted (no integrity checks, transactions) in batches of 100 by the Event Parser, and post-processed by a scheduled job that normalizes the audit event records into the Audit Event database schema.
- Queries triggered from Microsoft SQL Reporting Services Reports: As the database grows in size, the resource impact of the reporting platform queries becomes important.
- The volume of data maintained in the audit event database affects the duration and resource cost of these queries.
To maintain good performance, we recommend PurgeData be used to limit the timespan of audit event data retained in the database. See Purge Reporting Data for more information.
- Finer-grained audit data management and clean-up is possible using the Privilege Management Reporting Database Administration Dashboard. The Database Administration Dashboard allows the purging of audits related to specific applications and suppression of incoming audit items related to those applications.
Before purging large sets of data, please ensure your SQL transaction logs can grow to accommodate. It may be necessary to delete data in stages when setting this up for the first time.
For more information, please see the Database Administration description in the Reporting Dashboard Guide.
The Microsoft SQL Server Reporting Services database remains relatively small, so capacity planning is not important.
A dedicated server should be sized according to Microsoft SQL Server Minimum Specifications and assessed periodically.