Skip to content

nikhilsbhat/yamll

Repository files navigation

Yamll

Go Report Card shields shields shields shields

Yamll turns a pile of YAML files into one coherent config.

Introduction

Define YAML dependencies the way code manages libraries.

Yamll resolves the graph, merges the result, and catches import cycles before they turn into pain.

Features

  • Merge multiple YAML files into one
  • Resolve imports and dependencies across local, Git, and HTTP sources
  • Catch import cycles, duplicate keys, and other dependency problems early
  • Trace values back to their source
  • Generate lock files for reproducible remote imports

Authentication

  • Pass credentials through environment variables and yamll will resolve them at runtime
  • Git imports support both ssh and http URLs
  • All supported authentication parameters are defined here

Installation

  • Recommend installing released versions. Release binaries are available on the releases page.

Homebrew

Install latest version on yamll on macOS

brew tap nikshilsbhat/stable git@github.com:nikhilsbhat/homebrew-stable.git
# for latest version
brew install nikshilsbhat/stable/yamll
# for specific version
brew install nikshilsbhat/stable/yamll@0.0.3

Check repo for all available versions of the formula.

Docker

Latest version of docker images are published to ghcr.io, all available images can be found there.

docker pull ghcr.io/nikhilsbhat/yamll:latest
docker pull ghcr.io/nikhilsbhat/yamll:<github-release-tag>

Build from Source

  1. Clone the repository:
    git clone https://github.com/nikhilsbhat/yamll.git
    cd yamll
  2. Build the project:
    make local.build

Usage

Basic Usage

Point yamll at your root YAML file and it pulls the graph together:

yamll import -f import.yaml

Handling Imports

Imports live in comments that start with ##++. yamll resolves them, walks the dependency tree, and merges everything in the right order.

Handling Wildcards

Wildcard imports keep noisy file lists out of the way.

Filenames matching the pattern stay hidden in tree, import, and build. Their data is folded under the pattern itself.

For example, ##++internal/fixtures/*.test.yaml might match one.test.yaml, two.test.yaml, and three.test.yaml.

Their names disappear from the command output, and the combined content appears under the pattern import. It keeps cyclic graphs and large fixture sets readable.

The examples below show the common cases.

Example root.yaml:

##++internal/fixtures/base.yaml
##++internal/fixtures/*.test.yaml
##++git+https://github.com/nikhilsbhat/yamll@main?path=internal/fixtures/base2.yaml;{"user_name":"${GIT_USERNAME}","password":"${GITHUB_TOKEN}"}
##++http://localhost:3000/database.yaml

config2:
  test: val
  <<: *default

config3:
  - *default
  - *mysqldatabase

workflow: *mysqldatabase

Example base.yaml:

default: &default
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: base-config
  data:
    key1: value1
    key2: value2
config1: *default

Example base2.yaml retrieved from GIT source:

names:
   - john doe
   - dexter

Example base3.yaml:

organizations:
  - thoughtworks
  - google
  - microsoft

Example one.test.yaml:

editor:
  - intellij
  - visual_code

Example two.test.yaml:

movies:
  - animation
  - comedy

Example three.test.yaml:

ott:
  - netflix
  - prime_video

database.yaml retrieved from URL source:

mysqldatabase: &mysqldatabase
  hostname: localhost
  port: 3012
  username: root
  password: root

Importing root.yaml should generate final yaml file as below

---
# Source: internal/fixtures/base3.yaml
organizations:
  - thoughtworks
  - google
  - microsoft
---
# Source: internal/fixtures/base.yaml
default: &default
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: base-config
  data:
    key1: value1
    key2: value2
config1: *default
---
# Source: internal/fixtures/*.test.yaml

editor:
   - intellij
   - visual_code
ott:
   - netflix
   - prime_video
movies:
   - animation
   - comedy
---
# Source: https://github.com/nikhilsbhat/yamll@main?path=internal/fixtures/base2.yaml
names:
  - john doe
  - dexter
---
# Source: http://localhost:3000/database.yaml
mysqldatabase: &mysqldatabase
  hostname: localhost
  port: 3012
  username: root
  password: root
---
# Source: internal/fixtures/import.yaml
config2:
  test: val
  <<: *default
config3:
  - *default
  - *mysqldatabase
workflow: *mysqldatabase

Dependency Tree

Need the graph? yamll tree prints it like a filesystem tree.

Example:

yamll tree -f import.yaml
yamll tree -f import.yaml --output=json
yamll tree -f import.yaml --output=dot
yamll tree -f import.yaml --output=mermaid

yamll tree defaults to the text tree. Use --output=json for structured data, --output=dot for Graphviz, and --output=mermaid for Mermaid.

Output:

└── internal/fixtures/import.yaml
    ├── internal/fixtures/base.yaml
    │   └── internal/fixtures/base3.yaml
    ├── internal/fixtures/base2.yaml
    │   └── internal/fixtures/base3.yaml
    ├── internal/fixtures/*.test.yaml (3 files)
    │   ├── /Users/youruser/my-opensource/yamll/internal/fixtures/base.test.yaml
    │   ├── /Users/youruser/my-opensource/yamll/internal/fixtures/base2.test.yaml
    │   ├── /Users/youruser/my-opensource/yamll/internal/fixtures/base3.test.yaml
    │   ├── internal/fixtures/base4.yaml
    │   ├── internal/fixtures/*.testing.yaml (3 files)
    │   │   ├── /Users/youruser/my-opensource/yamll/internal/fixtures/one.testing.yaml
    │   │   ├── /Users/youruser/my-opensource/yamll/internal/fixtures/three.testing.yaml
    │   │   └── /Users/youruser/my-opensource/yamll/internal/fixtures/two.testing.yaml
    │   ├── internal/fixtures/base5.yaml
    │   └── internal/fixtures/base4.yaml
    ├── https://github.com/nikhilsbhat/yamll@main?path=internal/fixtures/base2.yaml
    │   └── internal/fixtures/base3.yaml
    └── http://localhost:3000/database

Impact Analysis

Need the blast radius? yamll impact walks the graph in reverse and shows every downstream file that depends on the target.

Example:

yamll impact common.yaml
yamll impact -f internal/fixtures/import.yaml internal/fixtures/base.yaml

Output:

Affected files:
  api.yaml
  ingress.yaml
  web.yaml
  jobs.yaml

Total downstream dependencies: 17

Lint

Want a fast sanity check? yamll lint scans the graph for duplicate keys, unresolved imports, unused imports, circular refs, invalid anchors, and conflicting merges.

Example:

yamll lint -f import.yaml

If issues are found, yamll prints them and exits with a non-zero status.

Trace

Need the origin of a rendered value?

yamll trace maps a generated YAML path back to the source file and line number, compiler-style.

Example:

yamll trace internal/fixtures/import.yaml:base.movies
yamll trace -f internal/fixtures/import.yaml base.movies
yamll trace -f internal/fixtures/import.yaml workflow.dbname

Output:

origin: internal/fixtures/base5.yaml:2

Lock File

Remote imports are powerful, but drift. yamll lock pins resolved commits and checksums so future runs stay reproducible.

More details: LOCKFILE.md

Example:

yamll lock -f internal/fixtures/import.yaml
yamll import -f internal/fixtures/import.yaml
yamll build -f internal/fixtures/import.yaml

To ignore the lock file for a run:

yamll import -f internal/fixtures/import.yaml --no-lock

Preventing Import Cycles

yamll detects and prevents import cycles. If an import cycle is detected, it will report an error and stop the merging process.

Documentation

Updated documentation on all available commands and flags can be found here.

Packages

 
 
 

Contributors