Parsing yaml, json, xml files using golang

WeChat Public Number: Operations and Maintenance Development Story, Author::wanger

Parsing and generating yaml files

YAML (YAML is not a markup language) is a human-readable data serialization language. It is commonly used for configuration files, but also for data storage or transfer.
YAML itself supports three basic data types: scalars (such as strings, integers, and floating-point numbers), lists, and maps (dictionaries/hashes).
We use the yaml.v3 package to parse yaml files

go get gopkg.in/yaml.v3

Parsing yaml

func Unmarshal(in []byte, out interface{}) (err error)

We use Unmarshal to parse yaml
The yaml file is as follows:

- name: wanger
  age: 24
  address: beijing
  hobby:
    - literature
    - social
- name: Tonko
  age: 30
  address: chengdu
  hobby:
    - basketball
    - guitar
- name: Huazi
  age: 27
  address: shenzhen
  hobby:
    - Glory of Kings
- name: Choke
  age: 29
  address: chongqing
  hobby:
    - Read
    - Glory of Kings
- name: Teacher Xia
  age: 27
  address: chengdu
  hobby:
    - Eat, drink and drink
- name: Total Ginger
  age: 25
  address: shanghai
  hobby:
    - talkshow
- name: Zheng Ge
  age: 30
  address: beijing
  hobby:
    - Read
    - Repeater

Read test.yaml

package main

import (
	"fmt"
	"gopkg.in/yaml.v2"
	"io/ioutil"
	"gopkg.in/yaml.v3"
	"log"
)



type Users struct {
	Name   string      `yaml:"name"`
	Age    int8        `yaml:"age"`
	Address  string `yaml:"address"`
	Hobby []string `yaml:"hobby"`
}


func main() {

	file, err := ioutil.ReadFile("test.yaml")
	if err != nil {
		log.Fatal(err)
	}
	var data [7]Users
	err2 := yaml.Unmarshal(file, &data)

	if err2 != nil {
		log.Fatal(err2)
	}
	for _, v := range data {
		fmt.Println(v)
	}
}

The output is as follows

{wanger 24 beijing [literature social]}
{Dong Ge 30 chengdu [basketball guitar]}
{Huazi 27 shenzhen [Glory of Kings]}
{Choke 29 chongqing [Read King Glory]}
{Teacher Xia 27 chengdu [Eat, drink and drink]}
{Ginger total 25 shanghai [Fishing Music Gourmet Wine talkshow]}
{Zheng Ge 30 beijing [Reading Rereader]}

Generate yaml

func Marshal(in interface{}) (out []byte, err error)

Let's use Marshal to generate yaml, a yaml file of information about our team
You can customize the key name of the output yaml file by defining the structure yaml tag

package main

import (
	"fmt"
	"gopkg.in/yaml.v3"
)



type Users struct {
	Name   string      `yaml:"name"`
	Age    int8        `yaml:"age"`
	Address  string `yaml:"address"`
	Hobby []string `yaml:"hobby"`
}


func main() {
	wanger := Users{
		Name: "wanger",
		Age:  24,
		Address: "beijing",
		Hobby: []string{"literature", "social"},
	}
	dongdong := Users{
		Name: "Tonko",
		Age:  30,
		Address: "chengdu",
		Hobby: []string{"basketball", "guitar"},
	}
	xialaoshi := Users{
		Name: "Teacher Xia",
		Age:  29,
		Address: "chengdu",
		Hobby: []string{"Eat, drink and drink"},
	}
	huazai := Users{
		Name: "Huazi",
		Age:  28,
		Address: "shenzhen",
		Hobby: []string{"Glory of Kings"},
	}
	qiaoke := Users{
		Name: "Choke",
		Age:  30,
		Address: "chongqing",
		Hobby: []string{"Read", "Glory of Kings"},
	}
	jiangzong := Users{
		Name: "Total Ginger",
		Age:  25,
		Address: "shanghai",
		Hobby: []string{"Go fishing","Music","Delicious food","Alcohol"},
	}
	zhengge := Users{
		Name: "Zheng Ge",
		Age:  30,
		Address: "beijing",
		Hobby: []string{"Read", "Repeater"},
	}
    userlist:=[7]Users{wanger,dongdong,huazai,qiaoke,xialaoshi,jiangzong,zhengge}

	yamlData, err := yaml.Marshal(&userlist)

	if err != nil {
		fmt.Printf("Error while Marshaling. %v", err)
	}

	fmt.Println(string(yamlData))
    fileName := "test.yaml"
    err = ioutil.WriteFile(fileName, yamlData, 0644)
    if err != nil {
        panic("Unable to write data into the file")
    }
}    

The yaml information generated is as follows

- name: wanger
  age: 24
  address: beijing
  hobby:
    - literature
    - social
- name: Tonko
  age: 30
  address: chengdu
  hobby:
    - basketball
    - guitar
- name: Huazi
  age: 27
  address: shenzhen
  hobby:
    - Glory of Kings
- name: Choke
  age: 29
  address: chongqing
  hobby:
    - Read
    - Glory of Kings
- name: Teacher Xia
  age: 27
  address: chengdu
  hobby:
    - Eat, drink and drink
- name: Total Ginger
  age: 25
  address: shanghai
  hobby:
    - Go fishing  
    - Music
    - Delicious food 
    - Alcohol
- name: Zheng Ge
  age: 30
  address: beijing
  hobby:
    - Read
    - Repeater

Parsing and Generating json Files

We use the encoding/json standard library package to parse and generate JSON files

Read and parse json files

func Unmarshal(data []byte, v interface{}) error

I've defined a user.json file here

{
  "users": [
    {
      "name": "wanger",
      "address": "beijing",
      "age": 24,
      "social": {
        "mobile": "111111111",
        "email": "wanger@163.com"
      }
    },
    {
      "name": "dongdong",
      "address": "chengdu",
      "age": 30,
      "social": {
        "mobile": "2222222222222222",
        "emial": "dongdong@163.com"
      }
    },
    {
      "name": "Teacher Xia",
      "address": "chengdu",
      "age": 29,
      "social": {
        "mobile": "2232222222222222",
        "emial": "xialaoshi@163.com"
      }
    },
    {
      "name": "Zheng Ge",
      "address": "beijing",
      "age": 30,
      "social": {
        "mobile": "12222211111",
        "email": "zhengge@163.com"
      }
    },
    {
      "name": "Total Ginger",
      "address": "shanghai",
      "age": 25,
      "social": {
        "mobile": "111122211",
        "email": "jaingzong@163.com"
      }
    },
    {
      "name": "Choke",
      "address": "chongqing",
      "age": 30,
      "social": {
        "mobile": "11333331111111",
        "email": "qiaoke@163.com"
      }
    },
    {
      "name": "Watson",
      "address": "shenzhen",
      "age": 28,
      "social": {
        "mobile": "113311111",
        "email": "huazai@163.com"
      }
    }
  ]
}

Read user.json file

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"os"
	"strconv"
)
func main() {
	jsonFile,err:=os.Open("user.json")
	if err != nil {
		fmt.Println(err)
	}
	fmt.Println("Successfully Opened users.json")
	defer jsonFile.Close()
	byteValue,_:=ioutil.ReadAll(jsonFile)
	var users Users
	json.Unmarshal(byteValue,&users)
	for i :=0;i<len(users.Users);i++ {
		fmt.Println("User Type: "+ users.Users[i].Address)
		fmt.Println("User Age: "+strconv.Itoa(users.Users[i].Age))
		fmt.Println("User Name: "+users.Users[i].Name)
		fmt.Println("User Email: "+users.Users[i].Social.Email)
	}
	var result Users
	json.Unmarshal(byteValue,&result)
}
type Users struct {
	Users []User `json:"users"`
}
type User struct {
	Name string `json:"name"`
	Address string `json:"address"`
	Age int `json:"Age"`
	Social Social `json:"social"`
}
type Social struct {
	Mobile string `json:"mobile"`
	Email string `json:"email"`
}

The output is as follows

Successfully Opened users.json
User Type: beijing
User Age: 24
User Name: wanger
User Email: wanger@163.com
User Type: chengdu
User Age: 30
User Name: dongdong
User Email: 
User Type: chengdu
User Age: 28
User Name: Teacher Xia
User Email: 
User Type: beijing
User Age: 30
User Name: Zheng Ge
User Email: zhengge@163.com
User Type: shanghai
User Age: 25
User Name: Total Ginger
User Email: jaingzong@163.com
User Type: chongqing
User Age: 29
User Name: Choke
User Email: qiaoke@163.com
User Type: shenzhen
User Age: 28
User Name: Watson
User Email: huazai@163.com

Sometimes, of course, we may not know the JSON data structure to read, so we can't predefine the structure, so we can use the **map[string]interface{}** type to parse the json.

var result map[string]interface{}
err = json.Unmarshal(byteValue, &result)
fmt.Printf("%+v\n", result)

The output information is as follows:

map[users:[map[address:beijing age:24 name:wanger social:map[email:wanger@163.com mobile:111111]] map[address:chengdu age:30 name:dongdong social:map[emial:dongdong@163.com mobil222222222222222]] map[address:chengdu age:28 name:Teacher Xia social:map[emial:xialaoshi@163.cmobile:2232222222222222]] map[address:beijing age:30 name:Zheng Ge social:map[email:zhengge@1com mobile:12222211111]] map[address:shanghai age:25 name:Total Ginger social:map[email:jaingzong3.com mobile:111122211]] map[address:chongqing age:29 name:Choke social:map[email:qiaoke@1com mobile:11333331111111]] map[address:shenzhen age:28 name:Watson social:map[email:huazai3.com mobile:113311111]]]]

Generate json file

func Marshal(v interface{}) ([]byte, error)
package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
)

func main() {
	wanger:=User{Address:"beijing",Name:"wanger",Age:24,Social:Social{Email:"wanger@163.com",Mobile:"111111111111"}}
	huazi:=User{Address:"shenzhen",Name:"huazai",Age:28,Social:Social{Email:"huazai@163.com",Mobile:"111122211111"}}
	qiaoke:=User{Address:"chongqing",Name:"qiaoke",Age:30,Social:Social{Email:"qiaoke@163.com",Mobile:"13332211111"}}
	xialaoshi:=User{Address:"chengdu",Name:"Teacher Xia",Age:29,Social:Social{Email:"xialaoshi@163.com",Mobile:"11144445411111"}}
	jiangzong:=User{Address:"shanghai",Name:"Total Ginger",Age:25,Social:Social{Email:"jiangzong@163.com",Mobile:"111222445211111"}}
	dongdong:=User{Address:"chengdu",Name:"Tonko",Age:30,Social:Social{Email:"dongdong@163.com",Mobile:"1155555211111"}}
	zhengge:=User{Address:"beijing",Name:"Zheng Ge",Age:24,Social:Social{Email:"zhengge@163.com",Mobile:"1112224566211111"}}
	result:=Users{Users: []User{wanger,huazi,jiangzong,xialaoshi,qiaoke,dongdong,zhengge}}
	bytearray,err:=json.Marshal(result)
	if err!=nil {
		fmt.Println(err)
	}
	fmt.Println(string(bytearray))
	fileName := "user.json"
	err = ioutil.WriteFile(fileName, bytearray, 0644)
	if err != nil {
		panic("Unable to write data into the file")
	}
}
type Users struct {
	Users []User `json:"users"`
}
type User struct {
	Name string `json:"name"`
	Address string `json:"address"`
	Age int `json:"Age"`
	Social Social `json:"social"`
}
type Social struct {
	Mobile string `json:"mobile"`
	Email string `json:"email"`
}

The output is as follows

{"users":[{"name":"wanger","address":"beijing","Age":24,"social":{"mobile":"111111111111","email":"wanger@163.com"}},{"name":"huazai","address":"shenzhen","Age":28,"social":{"mobile":"111122211111","email":"huazai@163.com"}},{"name":"Total Ginger","address":"shanghai","Age":25,"social":{"mobile":"111222445211111","email":"jiangzong@163.com"}},{"name":"Teacher Xia","address":"chengdu","Age":29,"social":{"mobile":"11144445411111","email":"xialaoshi@163.com"}},{"name":"qiaoke","address":"chongqing","Age":30,"social":{"mobile":"13332211111","email":"qiaoke@163.com"}},{"name":"Tonko","address":"chengdu","Age":30,"social":{"mobile":"1155555211111","email":"dongdong@163.com"}},{"name":"Zheng Ge","address":"beijing","Age":24,"social":{"mobile":"1112224566211111","email":"zhengge@163.com"}}]}

You can see that the JSON output above is not very beautiful. You can use the more readable function **json.MarshalIndent()** which defines the prefix and indentation of the output.

bytearray,err:=json.MarshalIndent(result,""," ")
if err!=nil {
	fmt.Println(err)
}
fmt.Println(string(bytearray))

The output is as follows, which looks much better than before

{
 "users": [
  {
   "name": "wanger",
   "address": "beijing",
   "Age": 24,
   "social": {
    "mobile": "111111111111",
    "email": "wanger@163.com"
   }
  },
  {
   "name": "huazai",
   "address": "shenzhen",
   "Age": 28,
   "social": {
    "mobile": "111122211111",
    "email": "huazai@163.com"
   }
  },
  {
   "name": "Total Ginger",
   "address": "shanghai",
   "Age": 25,
   "social": {
    "mobile": "111222445211111",
    "email": "jiangzong@163.com"
   }
  },
  {
   "name": "Teacher Xia",
   "address": "chengdu",
   "Age": 29,
   "social": {
    "mobile": "11144445411111",
    "email": "xialaoshi@163.com"
   }
  },
  {
   "name": "qiaoke",
   "address": "chongqing",
   "Age": 30,
   "social": {
    "mobile": "13332211111",
    "email": "qiaoke@163.com"
   }
  },
  {
   "name": "Tonko",
   "address": "chengdu",
   "Age": 30,
   "social": {
    "mobile": "1155555211111",
    "email": "dongdong@163.com"
   }
  },
  {
   "name": "Zheng Ge",
   "address": "beijing",
   "Age": 24,
   "social": {
    "mobile": "1112224566211111",
    "email": "zhengge@163.com"
   }
  }
 ]
}

Parsing and generating xml files

Parsing xml files

func Unmarshal(data []byte, v interface{}) error

Define a user.xml file

<?xml version="1.0" encoding="UTF-8"?>
<users>
    <user address="beijing">
        <name>wanger</name>
        <age>24</age>
        <social>
            <email>wanger@163.com</email>
            <mobile>1233455464</mobile>
        </social>
    </user>
    <user address="chengdu">
        <name>Tonko</name>
        <age>30</age>
        <social>
            <email>dongge@163.com</email>
            <mobile>12245555464</mobile>
        </social>
    </user>
    <user address="chengdu">
        <name>Teacher Xia</name>
        <age>29</age>
        <social>
            <email>xialaoshi@163.com</email>
            <mobile>12335677464</mobile>
        </social>
    </user>
    <user address="beijing">
        <name>Zheng Ge</name>
        <age>30</age>
        <social>
            <email>zhengge@163.com</email>
            <mobile>12334355464</mobile>
        </social>
    </user>
    <user address="shanghai">
        <name>Total Ginger</name>
        <age>25</age>
        <social>
            <email>jiangzong@163.com</email>
            <mobile>123565455464</mobile>
        </social>
    </user>
    <user address="chongqing">
        <name>Choke</name>
        <age>29</age>
        <social>
            <email>qiaoke@163.com</email>
            <mobile>124676755464</mobile>
        </social>
    </user>
    <user address="shenzhen">
        <name>Watson</name>
        <age>28</age>
        <social>
            <email>huazai@163.com</email>
            <mobile>1238655464</mobile>
        </social>
    </user>
</users>

Parsing xml files
address,attr means that the address field is an XML attribute, not a nested element.
If the structure has a field of type Name named XMLName, Unmarshal records the element name in that field.

For proper parsing, the go language xml package requires that all fields in the struct definition must be exportable (i.e., capital letters)

package main

import (
	"encoding/xml"
	"fmt"
	"io/ioutil"
	"os"
)
func main() {

	xmlFile,err:=os.Open("users.xml")
	if err!=nil {
		fmt.Println(err)
	}
	fmt.Println("successfully opened users.xml")
	defer xmlFile.Close()
	byteValue,_:=ioutil.ReadAll(xmlFile)
	var users Users
	xml.Unmarshal(byteValue,&users)
	for i :=0;i<len(users.Users);i++ {
		fmt.Println("User Address: "+users.Users[i].Address)
		fmt.Println("User Name: "+users.Users[i].Name)
		fmt.Println("Facebook Url: "+users.Users[i].Social.Email)
	}
}
type Users struct {
	XMLName xml.Name `xml:"users"`
	Users []User `xml:"user"`
}

type User struct {
	XMLName xml.Name `xml:"user"`
	Address string `xml:"address,attr"`
	Name string `xml:"name"`
	Social Social `xml:"social"`
}
type Social struct {
	XMLName xml.Name `xml:"social"`
	Mobile string `xml:"mobile"`
	Email string `xml:"email"`
}

The output is as follows:

successfully opened users.xml
User Address: beijing
User Name: wanger
Facebook Url: wanger@163.com
User Address: chengdu
User Name: Tonko
Facebook Url: dongge@163.com
User Address: chengdu
User Name: Teacher Xia
Facebook Url: xialaoshi@163.com
User Address: beijing
User Name: Zheng Ge
Facebook Url: zhengge@163.com
User Address: shanghai
User Name: Total Ginger
Facebook Url: jiangzong@163.com
User Address: chongqing
User Name: Choke
Facebook Url: qiaoke@163.com
User Address: shenzhen
User Name: Watson
Facebook Url: huazai@163.com

Generate xml file

func Marshal(v interface{}) ([]byte, error)
func MarshalIndent(v interface{}, prefix, indent string) ([]byte, error)

You can use the Marshal() function and the MarshalIndent() function, which were also used in previous json and yaml packages, except that MarshalIndent() can be added
Prefixes and indents, which look a little more beautiful, allow Marshal and MarshalIndent to process all other data by writing one or more XML elements that contain data.

package main

import (
	"encoding/xml"
	"fmt"
	"io/ioutil"
)
func main() {
	wanger:=User{Address:"beijing",Name:"wanger",Age:24,Social:Social{Email:"wanger@163.com",Mobile:"111111111111"}}
	huazi:=User{Address:"shenzhen",Name:"huazai",Age:28,Social:Social{Email:"huazai@163.com",Mobile:"111122211111"}}
	qiaoke:=User{Address:"chongqing",Name:"qiaoke",Age:30,Social:Social{Email:"qiaoke@163.com",Mobile:"13332211111"}}
	xialaoshi:=User{Address:"chengdu",Name:"Teacher Xia",Age:29,Social:Social{Email:"xialaoshi@163.com",Mobile:"11144445411111"}}
	jiangzong:=User{Address:"shanghai",Name:"Total Ginger",Age:25,Social:Social{Email:"jiangzong@163.com",Mobile:"111222445211111"}}
	dongdong:=User{Address:"chengdu",Name:"Tonko",Age:30,Social:Social{Email:"dongdong@163.com",Mobile:"1155555211111"}}
	zhengge:=User{Address:"beijing",Name:"Zheng Ge",Age:24,Social:Social{Email:"zhengge@163.com",Mobile:"1112224566211111"}}
    v:=&Users{Users: []User{wanger,huazi,qiaoke,xialaoshi,zhengge,jiangzong,dongdong}}

	result, err := xml.MarshalIndent(v, "  ", "    ")
	if err != nil {
		fmt.Printf("error: %v\n", err)
	}
	fmt.Println(string(result))
	fileName := "users.xml"
	err = ioutil.WriteFile(fileName, result, 0644)
	if err != nil {
		panic("Unable to write data into the file")
	}
}
type Users struct {
	XMLName xml.Name `xml:"users"`
	Users []User `xml:"user"`
}

type User struct {
	XMLName xml.Name `xml:"user"`
	Age int64 `xml:"age"`
	Address string `xml:"address,attr"`
	Name string `xml:"name"`
	Social Social `xml:"social"`
}
type Social struct {
	XMLName xml.Name `xml:"social"`
	Mobile string `xml:"mobile"`
	Email string `xml:"email"`
}

The output information is as follows

  <users>
      <user address="beijing">
          <age>24</age>
          <name>wanger</name>
          <social>
              <mobile>111111111111</mobile>
              <email>wanger@163.com</email>
          </social>
      </user>
      <user address="shenzhen">
          <age>28</age>
          <name>huazai</name>
          <social>
              <mobile>111122211111</mobile>
              <email>huazai@163.com</email>
          </social>
      </user>
      <user address="chongqing">
          <age>30</age>
          <name>qiaoke</name>
          <social>
              <mobile>13332211111</mobile>
              <email>qiaoke@163.com</email>
          </social>
      </user>
      <user address="chengdu">
          <age>29</age>
          <name>Teacher Xia</name>
          <social>
              <mobile>11144445411111</mobile>
              <email>xialaoshi@163.com</email>
          </social>
      </user>
      <user address="beijing">
          <age>24</age>
          <name>Zheng Ge</name>
          <social>
              <mobile>1112224566211111</mobile>
              <email>zhengge@163.com</email>
          </social>
      </user>
      <user address="shanghai">
          <age>25</age>
          <name>Total Ginger</name>
          <social>
              <mobile>111222445211111</mobile>
              <email>jiangzong@163.com</email>
          </social>
      </user>
      <user address="chengdu">
          <age>30</age>
          <name>Tonko</name>
          <social>
              <mobile>1155555211111</mobile>
              <email>dongdong@163.com</email>
          </social>
      </user>
  </users>

Tags: Go

Posted on Thu, 14 Oct 2021 12:52:50 -0400 by Roscoe