Secrets Management Overview & 7 Best Practices

Secrets management refers to the tools and methods for managing digital authentication credentials (secrets), including passwords, keys, APIs, and tokens for use in applications, services, privileged accounts and other sensitive parts of the IT ecosystem. While secrets management is applicable across an entire enterprise, the terms “secrets” and “secrets management” are referred to more commonly in IT with regard to DevOps environments, tools, and processes.

This blog will provide an overview of secrets management, explore challenges and risks to securing secrets, provide seven best practices for managing enterprise secrets, and offer additional resources.

Why Secrets Management Is Important

Passwords and keys are some of the most broadly used and important tools your organization has for authenticating applications and users and providing them with access to sensitive systems, services, and information. Because secrets have to be transmitted securely, secrets management must account for and mitigate the risks to these secrets, both in transit and at rest.

Examples of Secrets

• User- or auto-generated passwords
• API and other application keys/credentials (including within containers)
• SSH Keys
• Database and other system-to-system passwords.
• Private certificates for secure communication, transmitting, and receiving of data (TLS, SSL etc.)
• Private encryption keys for systems like PGP
• RSA and other one-time password devices

Challenges to Secrets Management

As the IT ecosystem increases in complexity and the number and diversity of secrets explodes, it becomes increasingly difficult to securely store, transmit, and audit secrets.

Common risks to secrets and some considerations include:

• Incomplete visibility and awareness of all privileged accounts, applications, tools, containers, or microservices deployed across the environment, and the associated passwords, keys, and other secrets. SSH keys alone may number in the millions at some organizations, which should provide an inkling of a scale of the secrets management challenge. This becomes a particular shortcoming of decentralized approaches where admins, developers, and other team members all manage their secrets separately, if they’re managed at all. Without oversight that stretches across all IT layers, there are sure to be security gaps, as well as auditing challenges.
• Hardcoded / embedded credentials: privileged passwords and other secrets are needed to facilitate authentication for app-to-app (A2A) and application-to-database (A2D) communications and access. Often, applications and IoT devices are shipped and deployed with hardcoded, default credentials, which are easy to crack by hackers using scanning tools and applying simple guessing or dictionary-style attacks. DevOps tools frequently have secrets hardcoded in scripts or files, which jeopardizes security for the entire automation process.
• Privileged credentials and the Cloud: Cloud and virtualization administrator consoles (as with AWS, Office 365, etc.) provide broad superuser privileges that enable users to rapidly spin up and spin down virtual machines and applications at massive scale. Each of these VM instances comes with its own set of privileges and secrets that need to be managed.
• DevOps tools: While secrets need to be managed across the entire IT ecosystem, DevOps environments are where the challenges of managing secrets seem to be particularly amplified at the moment. DevOps teams typically leverage dozens of orchestration, configuration management, and other tools and technologies (Chef, Puppet, Ansible, Salt, Docker containers, etc.) relying on automation and other scripts that require secrets to work. Again, these secrets should all be managed according to best security practices, including credential rotation, time/activity-limited access, auditing, and more.
• Third-party vendor accounts / remote access solutions: How do you ensure that the authorization provided via remote access or to a third-party is appropriately used? How do you ensure that the third-party organization is adequately managing secrets?
• Manual secrets management processes: Leaving password security in the hands of humans is a recipe for mismanagement. Poor secrets hygiene, such as lack of password rotation, default passwords, embedded secrets, password sharing, and using easy-to-remember passwords, mean secrets are not likely to remain secret, opening up the opportunity for breaches. Generally, more manual secrets management processes equate to a higher likelihood for security gaps and malpractices.

Best Practices & Solutions for Secrets Management

As noted above, manual secrets management suffers from many shortcomings. Siloes and manual processes are frequently in conflict with “good” security practices, so the more comprehensive and automated a solution the better.

While there are many tools that manage some secrets, most tools are designed specifically for one platform (i.e. Docker), or a small subset of platforms. Then, there are application password management tools that can broadly manage application passwords, eliminate hardcoded and default passwords, and manage secrets for scripts.

While application password management is an improvement over manual management processes and standalone tools with limited use cases, IT security will benefit from a more holistic approach to manage passwords, keys, and other secrets throughout the enterprise. Some secrets management or enterprise privileged credential management/privileged password management solutions go beyond just managing privileged user accounts, to manage all kinds of secrets—applications, SSH keys, services scripts, etc. These solutions can reduce risks by identifying, securely storing, and centrally managing every credential that grants an elevated level of access to IT systems, scripts, files, code, applications, etc.

In some cases, these holistic secrets management solutions are also integrated within privileged access management (PAM) platforms, which can layer on privileged security controls. Leveraging a PAM platform, for instance, you could provide and manage unique authentication to all privileged users, applications, machines, scripts, and processes, across your entire environment.

While holistic and broad secrets management coverage is best, regardless of your solution(s) for managing secrets, here are 7 best practices you should focus on addressing:

1. Discover / identify all types of passwords, keys and other secrets across your entire IT environment and bring them under centralized management. Continuously discover and onboard new secrets as they are created.
2. Eliminate hardcoded / embedded secrets in DevOps tool configurations, build scripts, code files, test builds, production builds, applications, and more. Bring hardcoded credentials under management, such as by using API calls, and enforce password security best practices. Eliminating hardcoded and default passwords effectively removes dangerous backdoors to your environment.
3. Enforce password security best practices, including password length, complexity, uniqueness expiration, rotation, and more across all types of passwords. Secrets, if possible, should never be shared. If a secret is shared, it should be immediately changed. Secrets to more sensitive tools and systems should have more rigorous security parameters, such as one-time passwords, and rotation after each use.
4. Apply privileged session monitoring to log, audit, and monitor all privileged sessions (for accounts, users, scripts, automation tools, etc.) to improve oversight and accountability. This can also entail capturing keystrokes and screens (allowing for live view and playback). Some enterprise privilege session management solutions also enable IT teams to pinpoint suspicious session activity in-progress, and pause, lock, or terminate the session until the activity can be adequately evaluated.
5. Extend secrets management to third-parties – ensure partners and vendors conform to best practices in using and managing secrets.
6. Threat analytics – continuously analyze secrets usage to detect anomalies and potential threats. The more integrated and centralized your secrets management, the better you will be able to report on accounts, keys applications, containers, and systems exposed to risk.
7. DevSecOps – With the speed and scale of DevOps, it’s crucial to build security into both the culture and the DevOps lifecycle (from inception, design, build, test, release, support, maintenance). Embracing a DevSecOps culture means that everyone shares responsibility for security, helping ensure accountability and alignment across teams. In practice, this should entail ensuring secrets management best practices are in place and that code does not contain embedded passwords in it.

By layering on other security best practices, including the principle of least privilege (PoLP) and separation of privilege, you can help ensure that users and applications have access and privileges restricted precisely to what they need and is authorized. Restriction and separation of privileges helps reduce privileged access sprawl and condense the attack surface, such as by limiting lateral movement in the event of a compromise.

The right secrets management policies, buttressed by effective processes and tools, can make it much easier to manage, transmit, and secure secrets and other privileged information. By applying the 7 best practices in secrets management, you can not only support DevOps security, but tighter security across the enterprise.