Create Encrypted Zones in HDFS

Let us understand few details about Encryption, enable encryption and then how we can create encrypted zones in HDFS.

  • When we say Data Encryption we are actually talking about business data that is stored in HDFS. This is different from log and query redaction.
  • Data encryption is mandatory for many government, financial, and regulatory entities, worldwide, to meet privacy and other security requirement.
  • Examples: Credit Card Payment Companies have to be comply with PCI DSS, Insurance and other Health Care companies need to be comply with HIPAA, etc.
  • Encrypting data stored in HDFS can help your organization comply with such regulations.

Key Capabilities

Let us go through some of the key capabilities of HDFS Encryption.

  • HDFS Clients can Encrypt or Decrypt the data.
  • Encryption and Decryption require a key. Such key management is external to HDFS. We will get the list of Cloudera Provided Key Management Solutions as part of the Wizard.
    • Cloudera Navigator Key Trustee Server
    • A file-based password-protected Java KeyStore
  • We will be using Java KeyStore approach for now.
  • HDFS uses the Advanced Encryption Standard-Counter mode (AES-CTR) encryption algorithm. AES-CTR supports a 128-bit encryption key (default).
  • It also supports 256-bit encryption key when Java Cryptography Extension (JCE) is installed.
  • HDFS Encryption can take the advantage of Hardware Encryption Accelerators such as AES-NI Instruction Set.

Enabling HDFS Encryption Using Wizard

As the curriculum talk about creating zones only we will take the simplest path to enable encryption and focus on creating encryption zones.

  • Go to the Wizard Cluster -> Set up HDFS Data At Rest Encryption
  • There are several approaches to Enable Encryption. Out of all the options we will choose A file-based password-protected Java KeyStore
  • It will actually highlight the steps need to be performed
    • Enable Kerberos – Recommended
    • Enable TLS/SSL – Recommended
    • Add Java KeyStore KMS Service
    • Restart Stale Services and Redeploy Client Configuration
    • Validate Data Encryption
  • As Kerberos and TLS/SSL are not mandatory, we will not be setting up those for learning purpose at this time. However, in actual production clusters, we need to enable both.
  • We can click on the link Add Java KeyStore KMS Service in steps and add Java KeyStore for now.
    • Add Key Management Server – bigdataserver-4
    • Add Key Admin User and Group User – itversity
    • Click on Generate ACLs
    • Click Continue to save generated XML
    • Leave to defaults and Continue
  • Make sure services are restarted and client configurations are redeployed. Ensure that there are no icons to restart or redeploy before going for the next step.

Share this post