Python yaml module

Introduction to yaml files

yaml is a specialized language for writing configuration files.

yaml file rules

• Case-sensitive.
• Use of indentation to indicate hierarchical relationships.
• Use the space bar to indent, not the Tab key to indent
• The number of indented spaces is not fixed, only the left-hand alignment of elements at the same level is required.
• Strings in a file do not need to be marked with quotation marks, but if they contain special characters they need to be marked with quotation marks.
• Comments marked with #

yaml file data structure

• Object: a collection of key-value pairs (referred to as a "mapping or dictionary")

  Key-value pairs are represented by a colon ":" structure, with a space separating the colon from the value

• Array: a set of values arranged in order (referred to as a "sequence or list")
  Arrays are preceded by the "-" symbol, with a space separating the symbol from the value

• scalars: single, non-divisible values (e.g., strings, bool values, integers, floating point numbers, time, date, null, etc.)
  None values can be represented by null or ~

Reading yaml configuration files in python

Prerequisites

Before reading yaml files in python you need to install pyyaml and import the yaml module.

• The module you need to install to use yaml is pyyaml (pip3 install pyyaml);
• The imported module is yaml (import yaml)

Read yaml file data

python reads the file data by open and then converts the data into a list or dictionary by the load function.

import yaml
import os

def get_yaml_data(yaml_file):
    # Open yaml file
    print("***Get yaml file data***")
    file = open(yaml_file, 'r', encoding="utf-8")
    file_data = file.read()
    file.close()

    print(file_data)
    print("Type:", type(file_data))

    # Converting strings to dictionaries or lists
    print("***Transform yaml data into dictionaries or lists***")
    data = yaml.load(file_data)
    print(data)
    print("Type:", type(data))
    return data
current_path = os.path.abspath(".")
yaml_path = os.path.join(current_path, "config.yaml")
get_yaml_data(yaml_path)

"""
***Get yaml file data***
# yaml key-value pairs: i.e. dictionaries in python
usr: my
psw: 123455
Type：<class 'str'>
***Transform yaml data into dictionaries or lists***
{'usr': 'my', 'psw': 123455}
Type：<class 'dict'>
"""

yaml file data as key-value pairs

(1) The contents of the yaml file are key-value pairs.

# yaml key-value pairs: i.e. dictionaries in python
usr: my
psw: 123455
s: " abc\n"

Data obtained by python after parsing the yaml file.

{'usr': 'my', 'psw': 123455, 's': ' abc\n'}

(2) The content of the yaml file is "key-value pairs’ nested in "key-value pairs"

# yaml key-value pair nesting: i.e. dictionary nested dictionaries in python
usr1:
  name: a
  psw: 123
usr2:
  name: b
  psw: 456

Data obtained by python after parsing the yaml file.

{'usr1': {'name': 'a', 'psw': 123}, 'usr2': {'name': 'b', 'psw': 456}}

(3) nested "arrays" in "key-value pairs" in the yaml file

# Nested arrays in yaml key-value pairs
usr3:
  - a
  - b
  - c
usr4:
  - b

Data obtained by python after parsing the yaml file.

{'usr3': ['a', 'b', 'c'], 'usr4': ['b']}

yaml file data as an array

(1) The contents of the yaml file are arrays

# yaml arrays
- a
- b
- 5

Data obtained by python after parsing the yaml file.

['a', 'b', 5]

(2) yaml file "array" nested in "key-value pairs"

# Nested "key-value pairs" in "arrays" in yaml
- usr1: aaa
- psw1: 111
  usr2: bbb
  psw2: 222

Data obtained by python after parsing the yaml file:

[{'usr1': 'aaa'}, {'psw1': 111, 'usr2': 'bbb', 'psw2': 222}]

Basic data types in yaml files

# Pure quantity
s_val: name              # String: {'s_val': 'name'}
spec_s_val: "name\n"    # Special string: {'spec_s_val': 'name\n'}
num_val: 31.14          # Number: {'num_val': 31.14}
bol_val: true           # Boolean value: {'bol_val': True}
nul_val: null           # null value: {'nul_val': None}
nul_val1: ~             # null value: {'nul_val1': None}
time_val: 2018-03-01t11:33:22.55-06:00     # Time value: {'time_val': datetime.datetime(2018, 3, 1, 17, 33, 22, 550000)}
date_val: 2019-01-10    # Date value: {'date_val': datetime.date(2019, 1, 10)}

Reference in yaml file

The contents of the yaml file

animal3: &animal3 fish
test: *animal3

The data read by python

{'animal3': 'fish', 'test': 'fish'}

Reading multiple yaml documents in Python

Multiple documents in one yaml file, segmented using — separations

For example: data in yaml file

# Segmenting multiple documents in a yaml file
---
animal1: dog
age: 2
---
animal2: cat
age: 3

python script to read multiple documents in a yaml file method

python to get yaml data when you need to use the load_all function to parse all the documents, and then read the data from the object

# yaml file contains more than one document, get the data in the document separately
def get_yaml_load_all(yaml_file):
    # Open yaml file
    file = open(yaml_file, 'r', encoding="utf-8")
    file_data = file.read()
    file.close()
    all_data = yaml.load_all(file_data)
    for data in all_data:
        print(data)
current_path = os.path.abspath(".")
yaml_path = os.path.join(current_path, "config.yaml")
get_yaml_load_all(yaml_path)
"""Results
{'animal1': 'dog', 'age': 2}
{'animal2': 'cat', 'age': 3}
"""

Python object generation yaml documentation

Direct import yaml (i.e. import yaml) generated yaml document

The yaml.dump() method does not transform the list or dictionary data into the yaml standard schema, it only generates the data into a yaml document

# Generating python objects into yaml documents
import yaml
def generate_yaml_doc(yaml_file):
    py_object = {'school': 'zhang',
                 'students': ['a', 'b']}
    file = open(yaml_file, 'w', encoding='utf-8')
    yaml.dump(py_object, file)
    file.close()
current_path = os.path.abspath(".")
yaml_path = os.path.join(current_path, "generate.yaml")
generate_yaml_doc(yaml_path)
"""Results
school: zhang
students: [a, b]
"""

Generate standard yaml documents using the yaml method in the ruamel module

(1) Use the yaml prerequisites in the ruamel module

• Modules that need to be installed to use yaml: ruamel.yaml (pip3 install ruamel.yaml);
• Imported modules: from ruamel import yaml

(2) ruamel module to generate yaml documents

def generate_yaml_doc_ruamel(yaml_file):
    from ruamel import yaml
    py_object = {'school': 'zhang',
                 'students': ['a', 'b']}
    file = open(yaml_file, 'w', encoding='utf-8')
    yaml.dump(py_object, file, Dumper=yaml.RoundTripDumper)
    file.close()
current_path = os.path.abspath(".")
yaml_path = os.path.join(current_path, "generate.yaml")
generate_yaml_doc_ruamel(yaml_path)
"""Results
school: zhang
students:
- a
- b
"""

(3) ruamel module reads yaml documents

# Read yaml file by from ruamel import yaml
def get_yaml_data_ruamel(yaml_file):
    from ruamel import yaml
    file = open(yaml_file, 'r', encoding='utf-8')
    data = yaml.load(file.read(), Loader=yaml.Loader)
    file.close()
    print(data)
current_path = os.path.abspath(".")
yaml_path = os.path.join(current_path, "dict_config.yaml")
get_yaml_data_ruamel(yaml_path)