Bs4.FeatureNotFound: Couldn't Find A Tree Builder With The Features ...

1. 1. Home
   2. Questions
   3. Tags
   4. Users
   5. Companies
   6. Labs
   7. Jobs
   8. Discussions
   9. Collectives

   10. Communities for your favorite technologies. Explore all Collectives

2. Teams

   Ask questions, find answers and collaborate at work with Stack Overflow for Teams.

   Try Teams for free Explore Teams
3. Teams

4. Ask questions, find answers and collaborate at work with Stack Overflow for Teams. Explore Teams

Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

Get early access and see previews of new features.

Learn more about Labs bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library? Ask Question Asked 10 years, 6 months ago Modified 1 year, 1 month ago Viewed 803k times 448 ... soup = BeautifulSoup(html, "lxml") File "/Library/Python/2.7/site-packages/bs4/__init__.py", line 152, in __init__ % ",".join(features)) bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?

The above outputs on my Terminal. I am on Mac OS 10.7.x. I have Python 2.7.1, and followed this tutorial to get Beautiful Soup and lxml, which both installed successfully and work with a separate test file located here. In the Python script that causes this error, I have included this line: from pageCrawler import comparePages And in the pageCrawler file I have included the following two lines: from bs4 import BeautifulSoup from urllib2 import urlopen

How can this problem be solved?

• python
• python-2.7
• beautifulsoup
• lxml

Share Improve this question Follow edited Apr 6, 2023 at 22:50 Dharman♦ 33.2k2727 gold badges9999 silver badges146146 bronze badges asked Jun 25, 2014 at 0:12 user3773048user3773048 6,20944 gold badges2020 silver badges2323 bronze badges 2

• 2 see this answer - stackoverflow.com/questions/17766725/how-to-re-install-lxml – Md. Mohsin Commented Nov 1, 2014 at 8:34
• 1 Is html a url or a the html contents? – tommy.carstensen Commented Jan 10, 2018 at 15:57

Add a comment |

22 Answers 22

Sorted by: Reset to default Highest score (default) Trending (recent votes count more) Date modified (newest first) Date created (oldest first) 476

I have a suspicion that this is related to the parser that BS will use to read the HTML. They document is here, but if you're like me (on OSX) you might be stuck with something that requires a bit of work:

You'll notice that in the BS4 documentation page above, they point out that by default BS4 will use the Python built-in HTML parser. Assuming you are in OSX, the Apple-bundled version of Python is 2.7.2 which is not lenient for character formatting. I hit this same problem, so I upgraded my version of Python to work around it. Doing this in a virtualenv will minimize disruption to other projects.

If doing that sounds like a pain, you can switch over to the LXML parser:

pip install lxml

And then try:

soup = BeautifulSoup(html, "lxml")

Depending on your scenario, that might be good enough. I found this annoying enough to warrant upgrading my version of Python. Using virtualenv, you can migrate your packages fairly easily.

Share Improve this answer Follow edited May 6, 2018 at 13:31 user124384 41011 gold badge1111 silver badges2222 bronze badges answered Nov 11, 2014 at 3:16 James ErricoJames Errico 6,21611 gold badge2121 silver badges1616 bronze badges 5

• 4 To test after pip install : python -c 'import requests ; from bs4 import BeautifulSoup ; r = requests.get("https://www.allrecipes.com/recipes/96/salad/") ; soup = BeautifulSoup(r.text, "lxml") ' – ViFI Commented Mar 2, 2019 at 19:40
• 3 in my virtual env, I needed to install requests, bs4 and lxml before BeautifulSoup would parse my webpage content. – noobninja Commented Nov 25, 2019 at 19:22
• 2 Uff! Mad Mac, I dont know when I'll stop regretting my decision of buying Mac! – Iqra. Commented May 4, 2020 at 23:45
• 4 The first time I had to run lxml I added the line import lxml into my script then it ran – TobyPython Commented Feb 10, 2021 at 19:22
• This didn't work for me on my MacAir MacOS 14.3 (Sonoma), M2 Chip using JupyterLab. Wound up having to do: import html5lib Then: soup = BeautifulSoup(c, "html5lib") as suggested by Tim Seed below. – Winston Lee Commented Feb 19 at 16:28

Add a comment | 122

I'd prefer the built in python html parser, no install no dependencies

soup = BeautifulSoup(s, "html.parser")

Share Improve this answer Follow edited Jan 8, 2021 at 12:59 answered May 10, 2017 at 8:55 ErnstErnst 1,26311 gold badge88 silver badges44 bronze badges 3

• 2 Although this answer doesn't answer question directly, it does provide potentially a better alternative. I had no preference for xlml and i changed everything to html.parser and it worked. I'd rather carry forward with something that works out of the box , than drag on the unnecessary technical debt. – donkz Commented Mar 25, 2021 at 14:48
• Sometimes the html parser doesn't do the job. Some page requires the XML parser to do the job. – Luís Henrique Martins Commented Mar 17, 2022 at 15:53
• html.parser does not preserve the case for XML elements and makes them all lowercase. :( – VahidNaderi Commented Feb 8 at 11:26

Add a comment | 65

For basic out of the box python with bs4 installed then you can process your xml with

soup = BeautifulSoup(html, "html5lib")

If however you want to use formatter='xml' then you need to

pip3 install lxml soup = BeautifulSoup(html, features="xml") Share Improve this answer Follow answered Feb 10, 2017 at 4:24 Tim SeedTim Seed 5,27922 gold badges3131 silver badges2727 bronze badges 2

• 7 On a newly spun up remote server, html5lib didn't work out of the box for me. I still had to do a pip install html5lib, after which everything worked fine. – petercoles Commented Dec 14, 2019 at 14:00
• 2 Didn't work for me: bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library? If I change it to html.parser it works – 8bitjunkie Commented May 22, 2020 at 20:29

Add a comment | 65

Run these three commands to make sure that you have all the relevant packages installed:

pip install bs4 pip install html5lib pip install lxml

Then restart your Python IDE, if needed.

That should take care of anything related to this issue.

Share Improve this answer Follow edited May 25, 2020 at 3:17 answered Feb 12, 2020 at 8:22 Pikamander2Pikamander2 8,27944 gold badges5454 silver badges7373 bronze badges 1

• 5 The key for me on this was restarting the IDE that triggered this all to work successfully – KVSEA Commented Jun 26, 2022 at 4:20

Add a comment | 50

Actually 3 of the options mentioned by other work.

# 1. soup_object= BeautifulSoup(markup,"html.parser") #Python HTML parser # 2. pip install lxml soup_object= BeautifulSoup(markup,'lxml') # C dependent parser # 3. pip install html5lib soup_object= BeautifulSoup(markup,'html5lib') # C dependent parser Share Improve this answer Follow edited Jul 2, 2022 at 17:02 JayRizzo 3,59633 gold badges3737 silver badges5252 bronze badges answered Sep 1, 2020 at 20:14 33Anika3333Anika33 63155 silver badges22 bronze badges 1

• In my case lxml used to work but when I switched to html.parser it froze. – Yan King Yin Commented Dec 13, 2021 at 18:01

Add a comment | 20

Install LXML parser in python environment.

pip install lxml

Your problem will be resolve. You can also use built-in python package for the same as:

soup = BeautifulSoup(s, "html.parser")

Note: The "HTMLParser" module has been renamed to "html.parser" in Python3

Share Improve this answer Follow answered May 28, 2020 at 12:00 Shankar VishnuShankar Vishnu 25122 silver badges44 bronze badges Add a comment | 19

I am using Python 3.6 and I had the same original error in this post. After I ran the command:

python3 -m pip install lxml

it resolved my problem

Share Improve this answer Follow edited Jan 22, 2018 at 7:33 Kinght 金 18.3k55 gold badges6262 silver badges7777 bronze badges answered Jan 22, 2018 at 4:48 BasharBashar 19111 silver badge22 bronze badges 2

• 4 In Docker it's also necessary to apt install python-lxml – user7075574 Commented Oct 30, 2019 at 12:41
• I don't need to run apt install python-lxml, but perhaps this is image-dependent. It suffices for me to do python3 -m pip install lxml. – Jan-Åke Larsson Commented Sep 9, 2022 at 9:03

Add a comment | 14

Instead of using lxml use html.parser, you can use this piece of code:

soup = BeautifulSoup(html, 'html.parser') Share Improve this answer Follow answered Feb 13, 2018 at 12:28 YogeshYogesh 1,43211 gold badge1313 silver badges1616 bronze badges 1

• 2 vendor.bs.bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html.parser. Do you need to install a parser library? – alex Commented Apr 18, 2018 at 17:27

Add a comment | 8

Although BeautifulSoup supports the HTML parser by default If you want to use any other third-party Python parsers you need to install that external parser like(lxml).

soup_object= BeautifulSoup(markup, "html.parser") #Python HTML parser

But if you don't specified any parser as parameter you will get an warning that no parser specified.

soup_object= BeautifulSoup(markup) #Warnning

To use any other external parser you need to install it and then need to specify it. like

pip install lxml soup_object= BeautifulSoup(markup, 'lxml') # C dependent parser

External parser have c and python dependency which may have some advantage and disadvantage.

Share Improve this answer Follow edited Jan 3, 2022 at 11:42 ah bon 9,9792020 gold badges7979 silver badges180180 bronze badges answered Mar 24, 2018 at 11:06 Projesh BhoumikProjesh Bhoumik 1,0781414 silver badges1717 bronze badges Add a comment | 6

pip install lxml then keeping xml in soup = BeautifulSoup(URL, "xml") did the job on Mac.

Share Improve this answer Follow answered Dec 29, 2022 at 20:41 zabopzabop 7,78244 gold badges5353 silver badges103103 bronze badges Add a comment | 5

In my case I had an outdated version of the lxml package. So I just updated it and this fixed the issue.

sudo python3 -m pip install lxml --upgrade Share Improve this answer Follow answered Feb 17, 2022 at 3:25 blizzblizz 4,16866 gold badges3838 silver badges6262 bronze badges 1

• 1 thank you! this is what I needed to do also – Luther Commented Aug 3, 2022 at 6:53

Add a comment | 3

I encountered the same issue. I found the reason is that I had a slightly-outdated python six package.

>>> import html5lib Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/site-packages/html5lib/__init__.py", line 16, in <module> from .html5parser import HTMLParser, parse, parseFragment File "/usr/local/lib/python2.7/site-packages/html5lib/html5parser.py", line 2, in <module> from six import with_metaclass, viewkeys, PY3 ImportError: cannot import name viewkeys

Upgrading your six package will solve the issue:

sudo pip install six=1.10.0 Share Improve this answer Follow answered Mar 4, 2017 at 6:17 Qiao YangQiao Yang 3922 bronze badges 1

• sudo pip install six==1.10.0 – Pyd Commented Nov 28, 2017 at 16:56

Add a comment | 2

BS4 by default expects an HTML document. Therefore, it parses an XML document as an HTML one. Pass features="xml" as an argument in the constructor. It resolved my issue.

Share Improve this answer Follow answered Jul 3, 2022 at 4:41 AyanabhaAyanabha 5177 bronze badges 1

• 5 you need to install lxml, with pip install lxml – titusfx Commented Aug 13, 2022 at 16:38

Add a comment | 1

In some references, use the second instead of the first:

soup_object= BeautifulSoup(markup,'html-parser') soup_object= BeautifulSoup(markup,'html.parser') Share Improve this answer Follow edited Apr 2, 2018 at 14:07 nj2237 1,27833 gold badges2323 silver badges2929 bronze badges answered Apr 2, 2018 at 13:28 abhishekPakrashiabhishekPakrashi 2344 bronze badges 1

• You should provide a bit more detail in your answer – Michael Commented Apr 2, 2018 at 13:50

Add a comment | 1

The error is coming because of the parser you are using. In general, if you have HTML file/code then you need to use html5lib(documentation can be found here) & in-case you have XML file/data then you need to use lxml(documentation can be found here). You can use lxml for HTML file/code also but sometimes it gives an error as above. So, better to choose the package wisely based on the type of data/file. You can also use html_parser which is built-in module. But, this also sometimes do not work.

For more details regarding when to use which package you can see the details here

Share Improve this answer Follow answered Jan 24, 2020 at 3:07 Pranav BhendawadePranav Bhendawade 31533 silver badges33 bronze badges Add a comment | 1

Blank parameter will result in a warning for best available. soup = BeautifulSoup(html)

---------------/UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("html5lib"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.----------------------/

python --version Python 3.7.7

PyCharm 19.3.4 CE

Share Improve this answer Follow edited Mar 30, 2020 at 21:11 answered Mar 30, 2020 at 20:01 user176105user176105 1966 bronze badges Add a comment | 1

My solution was to remove lxml from conda and reinstalling it with pip.

Share Improve this answer Follow answered Nov 9, 2021 at 19:47 MJimitaterMJimitater 93955 gold badges1515 silver badges2929 bronze badges Add a comment | 1

I am using python 3.8 in pycharm. I assume that you had not installed "lxml" before you started working. This is what I did:

1. Go to File -> Settings
2. Select " Python Interpreter " on the left menu bar of settings, select "Python Interpreter."
3. Click the "+" icon over the list of packages.
4. Search for "lxml."
5. Click "Install Package" on the bottom left of the "Available Package" window.

Share Improve this answer Follow answered Jan 17, 2022 at 20:42 Jd_mahmudJd_mahmud 4166 bronze badges 0 Add a comment | 1

I fixed with below changes

Before changes

soup = BeautifulSoup(r.content, 'html5lib' ) print (soup.prettify())

After change

soup = BeautifulSoup(r.content, features='html') print(soup.prettify())

my code works properly

Share Improve this answer Follow edited Mar 11, 2022 at 3:46 answered Mar 6, 2022 at 14:00 Shivam BaldhaShivam Baldha 5688 bronze badges 2

• Are you sure of the syntax? The string in the second block of code doesn't seem to be valid Python syntax – aaossa Commented Mar 10, 2022 at 2:48
• Nowi it is work – Shivam Baldha Commented Mar 11, 2022 at 4:14

Add a comment | 1

You may want to double check that you're using the right interpreter if you have multiple versions of Python installed.

Once I chose the correct version of Python, lxml was found.

Share Improve this answer Follow answered Jul 30, 2022 at 23:21 Akira RorschachAkira Rorschach 9511 silver badge99 bronze badges Add a comment | 1

Important for Jupyternotebook-Users: If you decide for the lxml parser make sure to restart the jupyternotebook kernel after installing it with pip install lxml. Otherwise the parser can not be found as it is not yet inititalized properly. Restarting the kernel is possible via the jupyternotebook web/pycharm/vscode GUI.

Share Improve this answer Follow answered Nov 3, 2023 at 9:27 BrianBrainBrianBrain 5866 bronze badges Add a comment | 0

This method worked for me. I prefer to mention that I was trying this in the virtual environment. First:

pip install --upgrade bs4

Secondly, I used:

html.parser

instead of

html5lib Share Improve this answer Follow answered Feb 27, 2022 at 17:07 abbas abaeiabbas abaei 6333 gold badges33 silver badges88 bronze badges Add a comment | Highly active question. Earn 10 reputation (not counting the association bonus) in order to answer this question. The reputation requirement helps protect this question from spam and non-answer activity.

Not the answer you're looking for? Browse other questions tagged

• python
• python-2.7
• beautifulsoup
• lxml

or ask your own question.

• The Overflow Blog
• Why do developers love clean code but hate writing documentation?
• This developer tool is 40 years old: can it be improved?
• Featured on Meta
• The December 2024 Community Asks Sprint has been moved to March 2025 (and...
• Stack Overflow Jobs is expanding to more countries

Linked

2 How to install a parser library for python 3 Do I need to install a different parser library for Beauitfulsoup4? LXML is not working 124 Can existing virtualenv be upgraded gracefully? 22 How to re-install lxml? 7 Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library? 9 lxml is not found within Beautiful Soup 8 "bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml" after installing lxml 2 bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html5lib 1 Multiple errors when scraping premier league tables 0 How to solve Selenium exception: " invalid argument 'url' must be a string " See more linked questions

Related

0 How to split the tags from html tree 1 Non-recursive find with lxml builder 1 python beautifulsoup : lxml html.parser 0 bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html5lib. Do you need to install a parser library? 2 bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html5lib 2 Scraping XML data with BS4 "lxml" 0 bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: html-parser. Do you need to install a parser library? 3 Display XML tree structure with BeautifulSoup 1 TypeError: object of type 'lxml.etree._ElementTree' has no len() 1 Already got html-parser for BS4 : Couldn't find a tree builder ... html-parser

Hot Network Questions

• Nut allergy and I need a substitution
• Dative in front of accusative
• Best Practices for Managing Open-Source Vulnerabilities in Enterprise Deployments
• How to simplify/refactor this code even more?
• Thoughts and analogy in cognition
• Translation of "Nulla dies sine linea" into English within Context Given
• 2010s-era Analog story referring to something like the "bouba/kiki" effect
• C++ code reading from a text file, storing value in int, and outputting properly rounded float
• Finitely generated left ideals of operator algebras
• How to account for disproportionate group sizes?
• ping from script launched by cron
• Under epistemological pluralism, how can one determine the most suitable epistemology to apply in a given context?
• Kodaira-Thurston manifold
• What keyboard shortcuts disable the keyboard?
• What do "messy" weapons do, exactly?
• Why is my LED burning out?
• Looking for a time travel short story about a woman who makes small changes
• Help in identifying this dot-sized insect crawling on my bed
• Exploiting MSE for fast search
• Custom implementation of `std::unique_ptr<T>`
• Is it possible to translate/rotate the camera in geometry nodes?
• Why is there no AES-512 for CTR & variants to have good large nonces?
• Can Bob send a stone into Alice's future?
• Permanent night on a portion of a planet

more hot questions Question feed Subscribe to RSS Question feed

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

lang-py