Using helm-secrets

Helm secrets is a great plugin to avoid checking in secrets in your Source code.

Here, I am using Hashicorp vault to store secrets and retrieve them safely in helm values files while installing helm charts.

Installation

$ helm plugin install https://github.com/jkroepke/helm-secrets

Setup

  $ export VAULT_TOKEN="s.VAULT_TOKENEXAMPLEASLDKASKDASDA" 
  $ export VAULT_ADDR="https://vault.example.com" 
  $ export HELM_SECRETS_DRIVER=vault

In vault, add the secrets:

In your helm values file, refer to the secret as follows:

db:
  db:
  database:     !vault secret/misp#db_database
  username:     !vault secret/misp#db_username
  pasword:      !vault secret/misp#db_password
  rootpasword:  !vault secret/misp#db_rootpasword

Now change the helm upgrade command as follows:

$ helm secrets upgrade misp ./helm/misp --install --wait --atomic  --namespace=misp --create-namespace  --values=./helm/misp/values.yaml

The secrets plugin will fetch and update the vault references in values file before invoking the upgrade command on helm.

Note:

To check the result of decoding, you can use:

$ helm secrets dec helm/misp/values.yaml

This will result in vaules.yaml.dec with actual decoded values from Hasicorp Vault.

GitHub action with matrix (parallel) jobs for each of the item in a JSON file

I have a file

deploy-list.json:

{
    "apiList": [
      "abc",
      "def"    
    ]
}

I would like to run a GitHub action with matrix (parallel) jobs for each of the item in the apiList above.

name: API Build

on: 
  workflow_dispatch:
    inputs:
      branch:
        description: 'Github Release Branch Name'
        required: true
        default: 'release/1.5.3'

  push:
    branches:
    - 'develop'

jobs:

  build-matrix:
    runs-on: ubuntu-latest

    steps:

      - name: Checkout Branch
        uses: actions/checkout@v2
        with:          
          ref: "${{ github.event.inputs.branch }}"

      - name: Set Matrix
        id: set-matrix
        run: echo "::set-output name=api_matrix::$(cat deploy-list.json | jq -c '.apiList')"

    outputs:
      api_matrix: ${{ steps.set-matrix.outputs.api_matrix }}

  build-api:
    environment: PRD
    needs: build-matrix
    runs-on: ubuntu-latest
    strategy:
      matrix:
        api_name: ${{ fromJson(needs.build-matrix.outputs.api_matrix) }}
    steps:
      - name: Checkout Branch
        uses: actions/checkout@v2
        with:
          ref: "${{ github.event.inputs.branch }}"

      - name: Building ${{ matrix.api_name }}
        run: |
          echo ${{ matrix.api_name }}
          # Do something here

Two parallel jobs will be invoked for "abc" and "def" apis making the workflow complete in twice as fast build time. I also using maven caching to speed up the builds even further.

Preventing secret leaks

I have added secrets to git repositories more than once in my lifetime.Once I even did it in a public repo.

One way to avoid this is to use a tool to detect secrets in the source code. This will come in the category of SAST tooling.

$ brew install gitleaks
$ brew install pre-commit
$ cd /path/to/repo
$ curl https://raw.githubusercontent.com/giantswarm/apiextensions/master/.gitleaks.toml -o .gitleaks.toml
$ git add .gitleaks.toml
$ git commit .gitleaks.toml -m "Adding .gitleaks.toml"

// Run scan
$ gitleaks detect --config=.gitleaks.toml

We don't repair anything...

We don't repair anything these days, not even relationships.

- S.A. 

Today's quote...

You may have born Round but you can die SQUARE.

- S.A.

How to update Vault with ADCS issued Intermediate Cert Authority

Start Vault server

vault server -dev
export VAULT_ADDR='https://vault.example.com'
export VAULT_TOKEN="s.q3M0FGIdtVu60hLJnwrU1JC2"
export VAULT_SKIP_VERIFY=1
vault status

Enable Engine

vault secrets enable -path=pki_intermediate_ca_core pki
vault secrets tune -max-lease-ttl=87600h pki_intermediate_ca_core # 10 Years

Generate CSR

vault write pki_intermediate_ca_core/intermediate/generate/internal common_name="Example Company" ttl=87600h country="United Arab Emirates" locality="Dubai" organization="Example Company" ou="Technology Department"

 -OR-

vault write -format=json pki_intermediate_ca_core/intermediate/generate/internal common_name="Example Company" ttl=87600h country="United Arab Emirates" locality="Dubai" organization="Example Company" ou="Technology Department" | jq -r '.data.csr' > pki_intermediate.csr

Key    Value

---    -----

csr    -----BEGIN CERTIFICATE REQUEST-----
MIICWTCCAUECAQAwFDESMBAGA1UEAxMJR2FuZCBCYW5rMIIBIjANBgkqhkiG9w0B
AQEFAAOCAQ8AMIIBCgKCAQEA7e1qj67LeZCnDPKa+14YCWp8XtbG4soRs544lIJW
YipBB5eCaiRazfA5kxWUv3fOklP7/pjCkeCNhryjS5DB1GK1EdgZNFpS8odqxXwY
t4CPECGVRzSK4Cce4OKBXFMKRuTuKgWH9i9Nt+eGaxD2gOkGTruuWyTiLUnr6/mx
PyenoHzMqyeUifTv0M651KUztqPJPvSz0SSO4+jpEIrGPNYEIET1Ce/1Opkf0kCq
vtCOFzIcVqzq/bYUjtkBvKgg7kyUG/EXAMPLEKJLyVA2ij3wC5LXD2Z8OMcr
iGeSqmrOKAeAJeOwfnULIhdsXABXouWlQwi+yhS5cS7QAQIDAQABoAAwDQYJKoZI
hvcNAQELBQADggEBAMPAABu9I+ezwm//CjDiIPhhQQQSsgmXPR9SdQMDkM94hGOQ
WkWFL66RDBZp/kC+OwNDC1lj7hPLGzhhZCQY3xtzcCVhRS8C1LZYiKlZ5HyY+9GG
KwBrOsBVNTyiLTDkpuGNhmUfJbIoM2fLbKoTQ7lWjaH+Ryyd7Ud8eB6L5FLXPpQm
QjdnhXqtQ7Z1u8Q66UzR7wXHhKTZn0ZBxS0C2m85pwgVdVQepL8KyGMx6zRAveyJ
wcZ4L+Ni7op7fO6nb78cfnMSE6Ja5X0KgIU0VPbVbwFAACHkNA9fP5DNvfa5DCWq
7RxQqJ7sQflnVulQ4qUnN1Y1seqFl8W36G3V8uM=
-----END CERTIFICATE REQUEST-----

* Signed from ADCS (x509 Base64 format = PEM)

Note: Run following on ADCS:

    > certreq -submit -attrib "certificatetemplate:SubCA"

-----BEGIN CERTIFICATE-----

MIIE+zCCAuOgAwIBAgITUAAAAARrstY4ahm6yQAAAAAABDANBgkqhkiG9w0BAQsF
ADAcMRowGAYDVQQDExFHYW5kIEJhbmsgUm9vdCBDQTAeFw0yMjA0MDIwMTQ5MTJa
Fw0yMzA0MDIwMTU5MTJaMBQxEjAQBgNVBAMTCUdhbmQgQmFuazCCASIwDQYJKoZI
hvcNAQEBBQADggEPADCCAQoCggEBAO3tao+uy3mQpwzymvteGAlqfF7WxuLKEbOe
OJSCVmIqQQeXgmokWs3wOZMVlL93zpJT+/6YwpHgjYa8o0uQwdRitRHYGTRaUvKH
asV8GLeAjxAhlUc0iuAnHuDigVxTCkbk7ioFh/YvTbfnhmsQ9oDpBk67rlsk4i1J
6+v5sT8np6B8zKsnlIn079DOudSlM7ajyT70s9EkjuPo6RCKxjzWBCBE9Qnv9TqZ
H9JAqr7QjhcyHFas6v22FI7ZAbyoIO5MlBv2qEANd2Eq/8iiS8lQNoo98AuS1w9m
fDjHK4hnkqpqzigHgCXjsH51CyIXbFwAV6LlpUMIvsoUuXEu0AECAwEAAaOCATww
ggE4MB0GA1UdDgQWBBR6lWYqP/8bOMicwdXJFJ7AnqIUuzAfBgNVHSMEGDAWgBQF
vqWonZOoah+8S0tku+yRQYOkyTBQBgNVHR8ESTBHMEWgQ6BBhj9maWxlOi8vLy9X
SU4tMUlCQzRJRjlKVUYvQ2VydEVucm9sbC9HYW5kJTIwQmFuayUyMFJvb3QlMjBD
QS5jcmwwawYIKwYBBQUHAQEEXzBdMFsGCCsGAQUFBzAChk9maWxlOi8vLy9XSU4t
MUlCQzRJRjlKVUYvQ2VydEVucm9sbC9XSU4tMUlCQzRJRjlKVUZfR2FuZCUyMEJh
bmslMjBSb290JTIwQ0EuY3J0MBkGCSsGAQQBgjcUAgQMHgoAUwB1AGIAQwBBMA8G
A1UdEwEB/wQFMAMBAf8wCwYDVR0PBAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQA4
3TviPyTXM6H+G3WCzdNMhcjauoEXAMPLEGI8JfdBsZayeEtw0ZHLbiEWDvylX
CN5FOoKImfcUNDXMzQY9PiokGKo69WtIUx5V+AZwMxDFoW9tkvrtO5AVtHJLlL5l
MDqD92dDAnojHGn8BDjlrVIxvMomMRXi5p6sksSwDijgnpIJtiml+Ss5nyI7JjID
X2x5fvhRP2kqQHisdpCWyz+l8jqj3dCFsECSHkoGJjhkj8sywJK2kK9h5sXMyj0K
VNRLLli1BaWFYk0++GVK72CnzaTBXw389Pv1a+B3yOYzd+QEoprSs7RUajHPbRmF
iepFIISHGdWrtWxH9W+9R4iWWHzQ7fUNAFjtVBo7inTEtlHYH+EFCv3sgnWW+mkr
AVU/dZV+XsLzBhbd0tm21cn3hWaGMujxswGNHvKw2uvo5KL277VKrgDwEWTIMx/L
LkqCEg23eN7n5oefbULFhJVn9RBFvvjdDs2q81mp/LgeXkJVesdR1Fe83TzXRiR9
gBLRw6RRqWWuRibsGhMl16LthQMFRBbucnBwQfLCxKdV9mv+s5nUrTbUBXSQDFcE
Dyk0Z/BmEPtiRWcQzyzYR4TwWLO3ejPexfZz1rAZdfZMKSuYnz0LqXQ6l2Kjs7b3
nbjn1W7s0CSzE4HomHwKRCqlBJUb/XapqilsQ5kTpQ==
-----END CERTIFICATE-----

Set Intermediate CA in Vault

cat From_ADCS_signed_certificate.pem > full_chain.pem
cat ADCS_root.pem >> full_chain.pem
vault write pki_intermediate_ca_core/intermediate/set-signed certificate=@full_chain.pem 

// Following is to Fix CRLs

vault write pki_intermediate_ca_core/config/urls \
            issuing_certificates="https://vault.example.com/v1/pki/ca" \
            crl_distribution_points="https://vault.example.com/v1/pki/crl"

Test

# Issue a cert:

vault write pki_intermediate_ca_core/roles/generic_server_cert allowed_domains="example.com" max_ttl="43830h" allow_subdomains=true #5 years
vault write pki_intermediate_ca_core/issue/generic_server_cert common_name="testserver01.example.com" ttl="24h" > testserver01

Ref:

[1] https://www.vaultproject.io/api-docs/secret/pki#set-signed-intermediate
[2] https://www.vaultproject.io/docs/secrets/pki 

Unable to delete a Kubernetes namespace?

Try this hack:

NAMESPACE=$1
kubectl get namespace $NAMESPACE -o json > $NAMESPACE.json
sed -i -e 's/"kubernetes"//' $NAMESPACE.json
kubectl replace --raw "/api/v1/namespaces/$NAMESPACE/finalize" -f ./$NAMESPACE.json