File Integrity Monitoring

Endpoint Privilege Management for Unix and Linux includes File Integrity Monitoring (FIM) to enhance system security and audit. FIM policies can be configured to schedule regular checks of the integrity of operating systems, software applications, and customer data. This verifies file permissions, ownership, and even cryptographic checksums, producing details reports for security alerts, vulnerability assessments, and audit.

Overview

FIM policies are configured and maintained in a centralized repository. FIM clients are assigned to specific policy, and automatically retrieve and use these policies to compare the local file system against a system baseline. Any policy violations or inappropriate changes to the file system are detailed in a report which is compiled and sent back to the central repository for future reference. Events are also generated to alert administrators of the security transgression.

Policy Configuration

FIM Policy configuration is stored as a JSON script. Each policy is named and details which file systems and files to verify, what aspects of each file to check, and assigns a perceived risk level to any violation. Each individual client host is assigned to a single named policy (or uses the default policy if unassigned). Each policy is named and is subject to change control if it is enabled.

Each policy is split into sections dealing with predefinitions of what aspects of individual files and file types to check, a list of included paths and files and what predefinitions to use for them, and a list of paths or files to exclude from the file integrity check.

Predefinitions Section

This section of the policy defines what checks need to be made on specific file types.

"sysconf": {
"exec": { "ino": true, "mode": true, "uid": true, "gid": true, "pmask": "022", "size": true, "mtime": true,"ctime": true, "hash": true, "risk": 10 },
"script": { "ino": true, "mode": true, "uid": true, "gid": true, "pmask": "022", "size": true, "mtime": true, "ctime": true, "hash": true, "risk": 10 },
"dev": { "uid": true, "gid": true, "mode": true, "risk": 10 },
"other": { "ino": true, "mode": true, "uid": true, "gid": true, "pmask": "002", "size": true, "mtime": true, "ctime": true, "risk": 6 }
}

The above predefinition details file system attributes to check for executable files, scripts, devices, and then all other files.

File types are defined as:

dir: Directory
dev: Character or block device
link: Symbolic link
sock: File socket
script: Script (#! interpreter)
exec: Executable file
file: Any other file type

A file type of all overrides all other entries within that predef.

A file type of other matches anything that isn’t matched by more specific types.

The complete list of file system attribute checks are:

dev	Boolean	Check the file device
ino	Boolean	Check the file inode
pdev	Boolean	Check the parent device
pino	Boolean	Check the parent inode
mode	Boolean	Check the file mode (07777, includes owner/group/other and setuid/setgid)
uid	Boolean	Check the uid
gid	Boolean	Check the gid
size	Boolean	Check the file size
hash	Boolean	Check the SHA256 hash of the file
hash_smaller	Integer	Check the SHA256 hash if the file is smaller than <num> kilobytes
size_min	Boolean	Check the file is the same or larger (for log files)
mtime	Boolean	Check the modification time
mtime_later	Boolean	Check the modification is equal or later (for log files)
ctime	Boolean	Check the change time
ctime_later	Boolean	Check the change time is equal or later (for log files)
perm	07777	Check specified permissions are set
pmask	07777	Check specified permissions are NOT set
risk	Integer	Arbitrary number, Magnitude denotes risk
own	Integer/String	UID or Username of ownership of the file
gown	Integer/String	GID or Group ownership of the file
file	Boolean	Check the file’s file name
ftype	Boolean	Check the file’s type

When symbolic links are scanned and checked, and the link is not broken, the link target’s canonicalized name is reported and the link target’s device and inode. These are reported as fields linktarget, linkdev, and linkino. This data is stored and checked within the hash field.

For more information on perm and pmask, see man umask.

Include Section

The include section lists all those files to include in the file system scan. Each entry defines a file specification which may be wildcarded, the name of the predef check that is performed, whether to recurse down the folder (if one is specified), whether to follow links into different directories, and whether to cross devices.

When wildcards are used on directory names, any dot files in that directory are not matched. To match files whose name begins with a dot, the path and directory name must not include wildcards.

Files are scanned in order of entries in the include section, according to the first pattern that matches the file. Any subsequent file matching patterns is ignored.

If we generally want /etc/* scanned with the sysconf predef, yet we specifically want /etc/passwd scanned with the log predef, the rule for /etc/passwd must appear before the entry for /etc.

"include": [
{ "path": "/etc/mtab",   "chk": "log",     "recurse": false },
{ "path": "/etc/motd",   "chk": "log",     "recurse": false },
{ "path": "/etc/passwd", "chk": "log",     "recurse": false },
{ "path": "/etc/shadow", "chk": "log",     "recurse": false },
{ "path": "/etc/*",      "chk": "sysconf", "recurse": true, "follow": false, "xdev": true },
{ "path": "/proc",       "chk": "log",     "recurse": false },
{ "path": "/mnt",        "chk": "log",     "recurse": false },
{ "path": "/boot/*",     "chk": "sysconf", "recurse": true, "follow": false, "xdev": true },
{ "path": "/bin/*",      "chk": "bin",     "recurse": true, "follow": false, "xdev": true },
{ "path": "/var/log/*",  "chk": "log",     "recurse": true, "follow": false, "xdev": true },
{ "path": "/var/adm/*",  "chk": "log",     "recurse": true, "follow": false, "xdev": true }
]

Exclude Section

The exclude section defines a list of wildcard strings that excludes files that have previously been specifically included in the previous section.

‘exclude’: [
      ‘/root/.*sh_history’,
      ‘/home/*’,
      ‘/etc/pb.db’
]

Policy Configuration and Maintenance

All configuration and maintenance of FIM policy is carried out using the pbdbutil utility. The utility has been enhanced to provide methods to retrieve, store, and list policies, as well as retrieve and summarize the reports generated from client hosts. The utility is also used on client hosts to initiate the file system checks, either from the command line, or scheduled using cron.

Syntax

Usage

pbdbutil --fim [<options>] [ <file> <file> ...]

Options for FIM client:

-r	Run FIM check
-U	Run FIM check and update database

Options for client remote management:

--client='{ params }'

Specify client REST parameters (NB use of = for optional parameter)

Options for FIM server database management:

-n	Create new FIM database
-l	List all FIM configurations in database
-l	Add an extra '-l' to list host assignments
-s <[-\|+]attribute>	Sort the list of records by attribute name (asc/desc)
-i <file>	Import FIM configuration file
-e <name> <file>	Export specified FIM configuration
-g <name>	Get FIM configuration by name
-d <name>	Delete FIM configuration
-d { "cfg" : { "name" : "<wildcard>" }}	Delete FIM configuration matching wildcard
-u { "name" : "<name>", "cfg": { json param... }}	Update FIM configuration
-A <name> <host(s)>	Assign host to configuration
-X <host(s)>	Assign host to configuration
-g { "rpt" : { "uuid" : "<uuid>" }}	Get FIM report
-g { "rpt" : { params ... }}	Retrieve report summarized from multiple reports
--format '[ "header", "header2", ... ]'	Define retrieved fields when using CSV report
-d { "rpt" : { "uuid" : "<uuid>" }}	Delete FIM report
-d { "rpt" : { params ... }	Delete FIM report(s)
-L [{	Retrieve, List or Delete FIM reports, with the attributes: ["name" : "<wildcard>",] ["uuid" : "<uuid>",] ["host" : "<wildcard>",] ["older" : <epoch>, ["newer" : <epoch>,] ["older" : { "years" :"months" : n, "days" : "hours" : n ] ["newer" : { "years" : n, "months" : n, "days" : n, "hours" : n ] ["updates" : <bool>,] ["risk" : <lvl>,] ["risk_higher" : <lvl>,] ["risk_lower" : <lvl>,] ["regexp" : true]}
-s <[-\|+]attribute>	Sort the list of records by attribute name (asc/desc)

Initial Setup and Configuration of the Service

Specify the FIM database by adding the keyword fileintegritydb to the pb.settings on all hosts.
If you are using Registry Name Service, define your primary server in the RNS database using:
```
pbdbutil –-svc –u ‘{ "svcgname":"dflt_fim_service", "cn" : "host", "role", "primary"}’
```
If you are not using Registry Name Service the submitmasters keyword is used to specify the FIM Server.
Initially, on the centralized server, the FIM database needs to be created using:
```
pbdbutil –-fim  -n
```
A configuration must be defined and stored using:
```
pbdbutil --fim –u ‘{ "cfg": { …. }}’
```
Once this has been performed, each client is assigned a configuration using:
```
pbdbutil --fim –A <cfgname>  <hostname>
```
On the client, run the following command to setup the initial baseline database and check the results are expected.
```
pbdbutil --fim –U
```
Once this has been completed, add a cron job to run the check on a regular basis:
```
0 0 * * * /usr/sbin/pbdbutil --fim –r > /dev/null 2>&1
```

Each run uploads the results to the centralized report repository. If configured by adding the setting fileintegrityevents on the primary service, each run also sends an event to the eventdb database.

File Integrity Monitor Reports

Reports consist of a JSON array of results, containing before (for example, existing entries in the baseline database) and after or new items which represent the current file system file status. Depending upon the checks run, various attributes are reported in each before or after item.

All character data (for example, file names) must consist of valid UTF-8 characters. Character bytes that are not valid UTF-8 characters are reported as \[<hex value of the byte>], and further, the slash is escaped and appears as two slashes.

For example, \\[98] represents the byte value 98 hex, or 10011000 binary.

# pbdbutil -P --fim -r
{
   "rpt": [
      {
         "path": "/etc",
         "risk": 6,            
         "change": "updated",            
         "before": {"size": 249,"mtime": "2016-09-23 09:13:19","ctime": "2016-09-23 09:13:19","name": "resolv.conf",
            "dev": 2051,"ino": 3415679},            
         "after": {"size": 0,"mtime": "2016-09-26 08:20:09","ctime": "2016-09-26 08:20:09","name": "foo",                "dev": 2051,"ino": 3415679 }        
      },
      {            
         "path": "/var/log",            
         "risk": 4,            
         "change": "new",            
         "after": {"pdev": 2051,"pino": 3933733, "mode": "600", "uid": 0, "gid": 0, "size": 1493, "mtime": "2016-09-26 08:20:01",                                    
            "ctime": "2016-09-26 08:20:01", "ftype": "file", "name": "cron", "hash": null, "dev": 2051, "ino": 3933604 }
      }
   ],
   "new": 1,
   "deleted": 0,            
   "updated": 1,            
   "policy": 0,            
   "total": 2,
   "max_risk": 6,            
   "name": "default",            
   "hostname": "pbuild",
   "uuid": "e62df7ea-980c-4252-a924-79a71a6ccc0f",        
   "whoami": "ctaylor"
}

Details at the end of the report include the number of new files, deleted files, updated files and those reported because they do not meet policy. Also detailed at the end of the report is the maximum risk (for example, the highest risk item that was tagged as updated), the name of the configuration, the hostname, a unique uuid of the report, who ran it, and the number of updates that were logged. The attributes name, inode, device, and path are always logged for every item so offending files can be uniquely identified on the file system.

As well as the standard report, the customer can choose to report in CSV using the -C option on the command line.

rpt{*}.path,max_risk,name,host,uuid,rundate,total,new,updated,policy,deleted,rpt{*}.risk,rpt{*}.change,rpt{*}.before.name,rpt{*}.before.ftype,rpt{*}.before.dev,rpt{*}.before.ino,rpt{*}.before.pdev,rpt{*}.before.pino,rpt{*}.before.size,rpt{*}.before.ctime,rpt{*}.before.mtime,rpt{*}.before.mode,rpt{*}.before.gid,rpt{*}.before.uid,rpt{*}.before.hash,rpt{*}.after.name,rpt{*}.after.ftype,rpt{*}.after.dev,rpt{*}.after.ino,rpt{*}.after.pdev,rpt{*}.after.pino,rpt{*}.after.size,rpt{*}.after.ctime,rpt{*}.after.mtime,rpt{*}.after.mode,rpt{*}.after.gid,rpt{*}.after.uid,rpt{*}.after.hash

/etc/pki,6,default,-,eff6b3c4-5a2e-4d10-8352-3c7e1a1d4523,-,10,7,3,0,0,6,updated,nssdb,-,2051,3408221,-,-,-,2016-09-22 15:02:45,2016-09-22 15:02:45,-,-,-,-,nssdb,-,2051,3408221,-,-,-,2016-09-26 08:09:35,2016-09-26 08:09:35,-,-,-,-

/etc,6,default,-,eff6b3c4-5a2e-4d10-8352-3c7e1a1d4523,-,10,7,3,0,0,6,new,-,-,-,-,-,-,-,-,-,-,-,-,-,resolv.conf,file,2051,3428436,2051,3407873,249,2016-09-26 08:19:15,2016-09-26 08:19:15,644,0,0,-

The summary information is added to every row, thereby flattening the CSV report. Additionally, a list of headers can be provided using the --format option.

pbdbutil –C –fim –r --format=’["rpt{*}.path","max_risk","name","host","uuid","rpt{*}.change"]’