Server Security Best Practices for Unix & Linux Systems

Server security, and the protection of Unix and Linux environments, has never been more urgent. While the adoption of PaaS, IaaS, and SaaS models has been underway for years, demand for all things cloud has become supercharged in this era of accelerated digital transformation to accommodate a vast increase in remote work and distributed ways of conducting business. This speed of evolution and increase in complexity contributes to server vulnerabilities and gaps in protections that attackers—whether external threat actor or insider—can exploit to inflict substantial damage. In this blog we will explore what Unix/Linux server security is, including:

• Common use cases for Unix and Linux
• Challenges to securing Linux and Unix servers
• The pros and cons of open source solutions and automation tools for Unix/Linux server security
• What Linux Security Modules are and how they can be used to improve security
• Enterprise-class solutions for Unix/Linux security
• 10 best practices for hardening and securing Unix & Linux environments
• Top solutions for securing Unix/Linux servers

Why Linux & Unix Security is also Cloud Security

Linux is the most widely used operating system that runs in the Cloud, with workloads that have huge variance. Linux is used to run not only web, database and mail server applications, but also blockchains, containers, TVs, IoT, SCADA and CI/CD pipelines, to name a few. The increasing volume of internet-facing cloud applications makes it more critical than ever to ensure these highly valued systems are protected against a growing threat landscape.

The nature of distributed systems means that they are prone to insecure configuration, creating gaps in security defenses. For example, a simple misconfiguration in a web application could allow threat actors to remotely execute commands on the underlying server. A determined threat actor will use this access to persistently probe the system, seek elevation of privilege, then ultimately take control of the entire system. Given time, advanced persistent threats (APTs) like this can morph from an isolated annoyance to a broad-scale breach that can shake the global digital ecosystem, such as we have seen with the SolarWinds attack.

There is not only a problem with scale for the Unix or Linux configuration; as the number of systems and platforms grow, infrastructure operators must also manage a rapidly increasing number of privileged identities (human and machine). This exponential increase in identities puts organizations in a difficult situation. Just a single improperly stored SSH key could serve as an attack vector through which an organization could be compromised. Simply put, the odds are stacking against you - the attacker aim is a greater than 0% chance of success, whereas the security team needs to aim for 100% coverage all the time.

Now think back to where Linux and Unix systems run--they are the cornerstones of critical business processes across almost every vertical. Some of these systems control continuous processes, where the recovery time from a shutdown can extend out to weeks. If these systems are unavailable due to a cyberattack or misconfiguration, it will cause significant reputational and financial impact to the organization.

What are the Challenges to Unix and Linux Security?

There are many challenges that come with a growing, leading-edge technology. Cybersecurity teams must proactively manage accounts on Linux and Unix systems, especially highly privileged accounts like root. This is where the challenge starts. Each system has a unique local user repository. Although this is fine for very small environments, as a company grows and builds more systems, the number of privileged accounts starts to multiply.

To ensure you are well-positioned to manage these accounts, it is important think about the answers to the following questions:

• How are System Administrators storing all root passwords?
• Are the passwords written down or stored anywhere in plain text?
• How can IT Security ensure these passwords are not weak?
• How can IT Security ensure default passwords are changed?
• How can System Administrators ensure the integrity of our system files?
• How do System Administrators ensure application security? How do System Administrators protect SSH Keys?

Managing Privileged Identities & Access across Servers: The 5 Ws

When it comes to managing these privileged identities in any environment, there are 5Ws to keep in mind to achieve appropriate security measures and processes for your organization:

1. Who

Who is the identity of the Linux or Unix user?

Who should have access?

2. What

What is the target asset?

What is the connectivity method?

3. Why

Why does the Linux or Unix user need access to this system?

Why should this be actioned?

4. When

When should this change be done?

When is this change acceptable to the business?

5. Where

Where will you perform this change from?

Where will the full audit trail of this change live?

Open Source Solutions for Managing Server Privileges

Many open source solutions are pre-installed on Unix/Linux platforms to help address the questions and challenges discussed in the sections above. For example, sudo, is a free application in most distributions that allows a standard user to run a command as a superuser (root) or another specified user. It can be compared to the right click ‘runas’ option in Windows. This allows for a straightforward principle of least privilege (PoLP) to be applied, ensuring users can elevate privileges on demand. However, this simple solution proves inadequate for the diverse demands of computing today, let alone infrastructures which are dynamically expanding at scale. It is worth noting that a major sudo vulnerability was recently discovered by the Qualys research team. This vulnerability has existed since 2011, but was only recently brough to public attention. You can read more about it in this blog.

As the number of systems increases, management with sudo becomes more challenging. Sudo requires you to manage every host separately, and quickly gets complex, time-consuming, and ultimately, untenable, to administer. This lack of centralization inevitably leads to gaps, and gaps lead to risk. (There are free automation tools that try to address the scaling problem, which we will explore later.)

In addition to lack of scalability, sudo does not fully address the privilege problem, or actually manage the root identity. Consequently, team members share root and other accounts, leading to workarounds, or even worse, breaches,

Once the root account is shared, you run into an audit/compliance problem – you have completely lost the ability to prove an indelible audit trail for any privileged activity. Sudo activity logs are not tamper-proof, making it impossible to prove compliance or effectively investigate a forensic issue.

To avoid getting to this point of sudo impasse, or to escape it, the best and most successful path is to migrate to using a true least privilege server management solution. This will empower your organization to run all users without any standing privileges, or without sharing accounts. Operators and administrators can continue running the applications they have always used, but elevation is only performed for the individual commands based on business policy. From the user’s point of view, this is as simple as swapping out a command-to-action elevation. All events are then fully recorded for accountability, and a granular policy allows you to control what actions are available. All privileged activity information is then captured in a tamper-proof audit trail, ready for use by auditors or incident response teams.

As you can see in my example below, there is a minimal impact to the existing workflow:

Understanding How Linux Security Modules (LSMs) Function

LSMs or Linux Security Modules, are shipped with the Linux kernel. LSMs are additional security frameworks within the Linux kernel that implement Mandatory Access Controls (MAC) rather than the traditional Discretionary Access Controls (DAC) for the entire system. The LSM inserts hooks at every point in the kernel where a user system call is requesting access to an internal, important kernel object. This framework then enables different implementations of a Mandatory Access Control-based security model.

There are several LSM modules included in the Linux kernel, each having a slightly different approach and management style, but they will all equally enhance security on your system. Looking at SELinux specifically, this solution provides granular security policies that goes further than the traditional default existing permissions of Read, Write, Execute, and assigning permissions to files or directories.

What makes LSMs particularly powerful is that all system calls are checked against a Policy Database and are denied by default. This is achieved by applying context to files and network ports (for example) and applying labels to these objects. Policies then reference these mappings, to provide a single policy.

Luckily, Administrators don’t need to write rules and labels from scratch. Many templated rules exist today to simplify deployment and testing on your system.

Why Use Linux Security Modules?

Linux Security Modules are proven to stop attackers in their tracks.

For example, if you have a vulnerable PHP site running on an Apache Linux server, if the LSM is correctly deployed, any outbreaks in the PHP vulnerability further into the machine will be automatically blocked.

When the attacker has command and control through the vulnerability, they will attempt to elevate or move laterally. Important files like sudoers, passwd, known_hosts, resolve.conf, etc. will all reference different labels to the PHP security policy the attacker is in. The PHP policy will not permit any read, write, or execute rights to any other object.

Such attacks like this are interesting—and dangerous—because the server could be patched, fully updated, and managed securely, but an attacker could still gain access through another vulnerability. LSMs provide a critical security layer here—but, you are only as strong as your weakest link.

• What about managing those root accounts to stop privilege elevation?
• Does IT Security know the ‘good’ baseline of system activity and session data?
• Does IT Security have an audit trail of the 5Ws?

This is where LSMs reach their limit, and additional layers of security are necessary.

BeyondTrust’s Privileged Management for Unix and Linux product (part of our Endpoint Privilege Management solution that also includes our Privilege Management for Windows & Mac product) can create a known baseline of activity on the endpoint and a central audit trail. All administrators of the system can sign on as a standard user, and only elevate the tools and applications required to complete the job. In addition, BeyondTrust Password Safe will automatically scan, onboard, and manage privileged accounts (human, application, machine, etc.). Not only the built-in root account, but also the identities and SSH Keys of other known or discovered privileged users. By taking these accounts under management, Password Safe ensures all credentials are released just-in-time, and changed after their use.

Automation tools – Infrastructure as Code

Automation tools, specifically Infrastructure as Code (IaC), have really simplified the configuration management of large environments. What used to be a painfully long and tedious undertaking (such as changing a configuration file on 100 servers, or even copying a file to them), can now be achieved in one tool across many platforms and operating systems.

Teams can apply this technology to simplify account management on these systems. New users, groups, and home directories can be created and defined from a central parent node. However, these built-in solutions, like SSSD, struggle in complex environments and lack advance features like Group Policy, multi-factor authentication (MFA), and central reporting. These tools also introduce additional risks. For instance, the highly privileged account that powers the automation tools is often static, with vast amounts of standing privilege. These types of accounts are arguably the most desirable target for a determined threat actor in any cyberattack, as it allows total control of the organization’s infrastructure.

This also does not replace the need for central identity management. Without centralized identity management, every System Administrator will need to maintain at least two accounts, each with unique passwords and policies. Using a tool like Active Directory (AD) Bridge, which is also part of BeyondTrust’s Endpoint Privilege Management solution, System Administrators can consolidate all user identities into Active Directory. By centralizing all user identification and authorization into AD, the business can achieve a single source of truth, a single point of management of all systems, and not just be limited to Windows endpoints. With BeyondTrust AD Bridging, all configuration, rollouts, and updates are performed by a central web application. This platform makes life much easier for identity teams – allowing them to reduce the time taken to process joiners, movers and leavers, as well as reduce the overhead in producing entitlement reports for auditing and compliance purposes. Not only are IT Operations saving significant amounts of time by leveraging AD Bridge, but security teams are also enforcing a one password policy, while making it easier for the users of the system by using single sign on.

Top 10 Security Hardening Best Practices for Unix and Linux

Here are the top 10 steps System Administrators should take to harden security for their Unix and Linux environments.

1. Pick a robust, secure, reputable operating system to run your business applications
2. Apply full disk encryption to ensure files are secure at-rest to enhance physical security
3. Make use of built-in Linux Security Modules and firewalls to enhance local security
4. Stay up-to-date with patches and automate this step as much possible so you are less likely to be left with a vulnerability
5. Eliminate shared accounts--enforce one account per person with a single username, ID, password, and home directory
6. Forward all syslog events to a remote SIEM or monitoring solution
7. Manage all privileged account passwords, keys, and secrets with automatic discovery and onboarding to support environment changes
8. Implement least privilege for all users by defining and enforcing policies
9. Log and record all privileged session activity for a full audit trail
10. Use file integrity monitoring (FIM) to ensure critical files are not tampered with

Military-Grade Linux & Unix Security from BeyondTrust

When choosing a Privileged Access Management (PAM) vendor, it is critical to validate that they have a holistic view on PAM and offer robust security capabilities that not only scale, but are also platform-agnostic, practical to implement, and deliver measurable ROI.

Here are some key Unix/Linux privilege management concepts that BeyondTrust can help your team achieve:

• Manage the accounts, passwords, and keys within your *nix environment to ensure password policies are adhered to and not left static and weak. This helps protect against many types of password attacks, such as password re-use, password cracking, pass-the-hash, and more.
• Implement the least privilege principle to eliminate excessive administrative privilege This helps protect against Unix and Linux privilege escalation attacks as well as lateral movement.
• Eliminate the dangerous use of shared accounts to achieve nonrepudiation
• Control access to your *nix environment, only grant sessions for approved, authenticated individuals, from approved locations
• Provide command line filtering and protect against errant or malicious commands
• Log user activities to ensure compliance across the organization
• Manage and record privileged activity remotely for complete audit trails on problems, changes, and incidents. BeyondTrust gives you an unimpeachable audit trail of all session activity.
• Get alerts and notifications of unusual activity so you can be proactive in your security, such as adjusting configurations or pausing or terminating a suspicious in-progress session.
• Enable centralized and unified administration to help your IT staff work as effectively and securely as possible

BeyondTrust supports you with over 30 years of innovation in server privilege management, and the most complete PAM platform. With BeyondTrust, you can mature privileged access security controls across your entire IT estate at the pace that works for you.

Below, are four BeyondTrust solutions which are key to several areas in the Unix and Linux security space. It is important to note, you do not need to start in any specific order when starting your PAM journey, but BeyondTrust can certainly recommend your next steps based on where you are today. Additionally, our solutions integrate into a single platform and can be combined to unlock additional synergies.

• Privilege management for Unix/Linux, for true least privilege on Unix, Linux and network systems. Controls root access, audits user activity, and enables session monitoring and replay capabilities.
• Active Directory Bridge, for extending Microsoft Active Directory authentication to Unix and Linux systems, for single sign on and Group Policy configuration.
• DevOps Secrets Safe, for centrally securing and managing secrets used in the CI/CD tool chain.
• Password Safe, for discovering, auditing, and monitoring privileged accounts of all types, such as root accounts, SSH keys, local administrator, to name a few.