Python XML Parser

By /
Xml Parsing Python

Ever stuck with an annoying XML file that you need to parse to get important values? Let’s learn how to create a Python XML parser.

<page>
    <header>
        <type heading="XML Parsing in Python"/>
        <type text="Hello from AskPython. We'll be parsing XML"/>
    </header>
</page>

We’ll look at how we can parse XML files like these using Python to get the relevant attributes and values.

Let’s get started!

Method 1: Using ElementTree (Recommended)

We can use the ElementTree Python library to achieve this task.

This is the simplest and recommended option for building a Python XML parser, as this library comes in bundled with Python by default.

Not only does it provide ease of access, since it is already installed, but it is also quite fast. Let’s look at exactly how we can extract attributes from our test file.

<page>
    <header>
        <type heading="XML Parsing in Python"/>
        <type text="Hello from AskPython. We'll be parsing XML"/>
    </header>
</page>

We’ll be using the xml.etree.ElementTree interface inside the core xml package.

import xml.etree.ElementTree as ET

Build the Python XML Parser Tree

Let’s first construct the root node of this parse tree. This is the topmost node of the tree, and is necessary for us to get started with the parsing.

Thankfully for us, this API already has the below method for us:

import xml.etree.ElementTree as ET
root_node = ET.parse('sample.xml').getroot()
print(root_node)

This will automatically read the XML input file and get the root node for us.

Output

<Element 'page' at 0x7f885836b2f0>

Okay, so it seems that is has parsed. But we cannot verify it yet. So let’s parse the other attributes and try to get its value.

Get the Values of Relevant Attributes

So now, our task is to get the value inside the <heading> attribute, with the use of our Python XML Parser.

Its position from the root node <page> is <header/type>, so we need to loop through all matches at that level of the tree.

We can do that using root_node.findall(level), where level is the desired position (<header/type> in our case).

for tag in root_node.find_all(level):
    value = tag.get(attribute)
    if value is not None: print(value)

The tag.get(attribute) will get the value of our <attribute> tag at the levels which we are searching at. So, we simply need to do this at <header/type>, and get the values of the <heading> and the <text> attributes. That’s it!

import xml.etree.ElementTree as ET

# We're at the root node (<page>)
root_node = ET.parse('sample.xml').getroot()

# We need to go one level below to get <header>
# and then one more level from that to go to <type>
for tag in root_node.findall('header/type'):
    # Get the value of the heading attribute
    h_value = tag.get('heading')
    if h_value is not None:
        print(h_value)
    # Get the value of the text attribute
    t_value = tag.get('text')
    if t_value is not None:
        print(t_value)

Output

XML Parsing in Python
Hello from AskPython. We'll be parsing XML

We’ve retrieved all the values at that level of our XML parse tree! We’ve successfully parsed our XML file.

Let’s take another example, in order to clear up everything.

Now, assume that the XML file looks like this:

<data>
    <items>
        <item name="item1">10</item>
        <item name="item2">20</item>
        <item name="item3">30</item>
        <item name="item4">40</item>
    </items>
</data>

Here, we must not only get the attribute values of name, but also get the text values 10, 20, 30, and 40 for every element at that level.

To get the attribute value of name, we can do the same as before. We can also use tag.attrib[name] to get the value. This is the same as tag.get(name), except that it uses dictionary lookups.

attr_value = tag.get(attr_name)
# Both methods are the same. You can
# choose any approach
attr_value = tag.attrib[attr_name]

To get the text value, it’s simple! Just get it using:

tag.text

So, our complete program for this parser will be:

import xml.etree.ElementTree as ET

# We're at the root node (<page>)
root_node = ET.parse('sample.xml').getroot()

# We need to go one level below to get <items>
# and then one more level from that to go to <item>
for tag in root_node.findall('items/item'):
    # Get the value from the attribute 'name'
    value = tag.attrib['name']
    print(value)
    # Get the text of that tag
    print(tag.text)

Output

item1
10
item2
20
item3
30
item4
40

You can extend this logic to any number of levels for arbitrarily long XML files too! You can also write a new parse tree to another XML file.

But, I’ll leave that for you to figure out from the documentation, since I’ve provided a starting point for you to build upon!

Method 2: Using BeautifulSoup (Reliable)

This is also another good choice, if, for some reason, the source XML is badly formatted. XML may not work very well if you don’t do some pre-processing to the file.

It turns out that BeautifulSoup works very well for all these types of files, so if you want to parse any kind of XML file, use this approach.

To install it, use pip and install the bs4 module:

pip3 install bs4

I’ll give you a small snippet for our previous XML file:

<data>
    <items>
        <item name="item1">10</item>
        <item name="item2">20</item>
        <item name="item3">30</item>
        <item name="item4">40</item>
    </items>
</data>

I’ll be passing this file then parsing it using bs4.

from bs4 import BeautifulSoup

fd = open('sample.xml', 'r')

xml_file = fd.read()

soup = BeautifulSoup(xml_file, 'lxml')

for tag in soup.findAll("item"):
    # print(tag)
    print(tag["name"])
    print(tag.text)

fd.close()

The syntax is similar to our xml module, so we’re still getting the attribute names using value = tag['attribute_name'] and text = tag.text. Exactly the same as before!

Output

item1
10
item2
20
item3
30
item4
40

We’ve now parsed this using bs4 too! If your source XML file is badly formatted, this method is the way to go since BeautifulSoup has different rules for handling such files.

Conclusion

Hopefully, you’ve now got a good grasp on how to build a Python XML parser easily. We showed you two approaches: One using the xml module, and another one using BeautifulSoup.

References