Remove a Tag But Keep Its Contents Using BeautifulSoup

The problem this handbook solves

When you scrape a web page, the markup is rarely as tidy as you would like. Useful text is often buried inside layers of styling tags — a stray <span>, a <font> wrapper, an <a> link, or a decorative <b> that adds nothing to the data you actually want. The goal is to delete the wrapper element itself while keeping every bit of text and every child node that lived inside it. BeautifulSoup, the popular Python HTML parser, has a method built precisely for this, and a couple of close alternatives that behave differently. This handbook walks through each one so you pick the right tool the first time.

The short answer: unwrap()

If you only remember one thing, remember unwrap(). Call it on a tag and BeautifulSoup removes that tag, then lifts everything that was inside it up into the parent in the same position. The wrapper is gone; the contents stay. It is the cleanest expression of "remove the tag but keep what is inside".

A minimal working example

Suppose you have a paragraph where an editor wrapped a phrase in a <b> tag and you want the words but not the bold tag. The pattern looks like this:

from bs4 import BeautifulSoup

html = "<p>Prices start <b>very low</b> this week</p>"
soup = BeautifulSoup(html, "html.parser")

bold = soup.find("b")
bold.unwrap()

print(soup.p)
# <p>Prices start very low this week</p>

The <b> wrapper has vanished, but the words "very low" remain exactly where they were. That is the behaviour you almost always want when tidying scraped content.

How unwrap() actually works under the hood

Conceptually, unwrap() takes the tag's children — both text strings and nested tags — and re-parents them onto the tag's parent at the tag's former position. It then removes the now-empty tag from the tree and returns it. Because the children are moved rather than copied, their internal structure and any attributes on nested tags are left completely intact. Only the single wrapper you targeted disappears.

unwrap() versus replace_with()

Some people reach for replace_with() to achieve the same effect, but it is a different tool. replace_with() swaps one node for another node or string that you supply. You can mimic unwrapping by replacing a tag with its contents, but you have to gather those contents yourself, and edge cases around multiple children make it fiddly. For the plain "remove wrapper, keep inside" task, unwrap() is purpose-built and far less error-prone. Save replace_with() for when you genuinely want to substitute different content.

unwrap() versus decompose() and extract()

It is worth being precise here because mixing these up causes silent data loss. decompose() destroys a tag and all of its contents and frees the memory — nothing survives. extract() pulls a tag (and its contents) out of the tree and hands it back to you, so you could re-insert it elsewhere, but its contents leave with it. Neither keeps the inner data in place. Use them when you want a whole branch gone, such as removing a navigation block, a script tag, or an advertisement container.

• unwrap() — keep the inside, drop the wrapper.
• decompose() — destroy the tag and everything in it, permanently.
• extract() — remove the tag and its contents, returning them for reuse.
• replace_with() — substitute a tag with new content you provide.

Removing many tags of the same type

Often you want to strip every <span> or every <font> on a page. Collect them with find_all(), then loop. The important detail is to iterate over the list find_all() returns, calling unwrap() on each item, so you do not skip elements while the tree shifts:

for span in soup.find_all("span"):
    span.unwrap()

This leaves all the text where it was while quietly deleting every span wrapper across the document.

Targeting tags by attribute or class

You rarely want to remove all tags of a kind — usually just those matching a class or attribute. Pass selectors to find_all() and only those will be unwrapped:

for tag in soup.find_all("span", class_="highlight"):
    tag.unwrap()

The same approach works for any attribute, letting you surgically remove only the decorative wrappers a particular site adds while leaving structural tags alone.

Handling nested wrappers

Some pages nest the same wrapper several levels deep — a <span> inside a <span> inside another. A single pass of find_all() followed by unwrap() handles this correctly because find_all() already collected every level before you began unwrapping. If you build your own recursive cleaner instead, be careful to re-scan after each change, or you may leave inner wrappers behind.

Why text sometimes runs together afterwards

When a wrapper is removed, its text strings become direct neighbours of the surrounding text. If you later call get_text() you may find words joined with no space between them. This is not a bug — there simply was no whitespace between the nodes. The fix is to pass a separator to get_text(), for example soup.get_text(" ", strip=True), or to normalise whitespace after extraction.

Cleaning before versus after extraction

You can either unwrap tags first and then pull text, or extract text and clean the string afterwards. Unwrapping first keeps the document structure usable if you still need to navigate it; cleaning the string afterwards is simpler when you only want the final text. Choose based on whether you will keep working with the parsed tree.

A reusable tag-stripping helper

If you strip the same set of wrappers across many pages, wrap the logic in a small function so your scraper stays readable:

def strip_wrappers(soup, names):
    for name in names:
        for tag in soup.find_all(name):
            tag.unwrap()
    return soup

strip_wrappers(soup, ["span", "font", "b", "i"])

This keeps your parsing code declarative — you state which wrappers to remove and the helper does the rest.

Where this fits in a scraping pipeline

Removing wrappers is a post-processing step. The typical flow is: fetch the page with an HTTP client, parse it with BeautifulSoup, navigate to the part you want, clean the markup by unwrapping noise tags, then extract structured fields. Keeping these stages separate makes each one easier to test and debug.

Where proxies enter the picture

BeautifulSoup never touches the network, so it has no concept of proxies. Proxies matter one step earlier, during fetching, when you request many pages and a single IP address is likely to be rate limited or blocked. Proxies route your requests through different IPs so the traffic looks like separate visitors. They are configured in your HTTP client — such as requests or httpx — not inside BeautifulSoup.

Proxy types that suit HTML scraping

The right proxy type depends on how defensive the target site is. For tolerant sites, fast and affordable options may be enough; for sites with stronger anti-bot defences, residential-grade IPs tend to perform better. Worth considering across typical projects:

• Datacenter / IPv4 proxies — fast and cost-effective for sites that do not scrutinise IPs heavily.
• Residential proxies — real consumer IPs that may help with sites that flag datacenter ranges.
• ISP proxies — a middle ground that can pair datacenter speed with consumer-grade trust.
• Mobile proxies — cellular IPs that may suit the most defensive targets, usually at a higher cost.

Value and cost considerations

Proxy pricing varies widely by type and provider, and the cheapest option is not always the best fit for a given target. It is worth testing a small sample of pages on a candidate proxy before committing to a larger plan, and matching the proxy type to the site's tolerance rather than over-buying. Conditional, measured spending tends to give the best value.

Common mistakes when stripping tags

• Reaching for decompose() when you meant unwrap() — and silently deleting the data you wanted.
• Iterating over a live result set and skipping elements as the tree mutates.
• Forgetting that removed wrappers can leave text strings touching, producing joined words later.
• Unwrapping structural tags you actually needed for navigation.
• Assuming proxies belong inside BeautifulSoup rather than the HTTP client.

Best practices recap

Favour unwrap() for the "keep contents" task, collect targets with find_all() before looping, use selectors to be surgical, normalise whitespace afterwards, and keep fetching, parsing and cleaning as distinct stages. These habits keep scraping code predictable and maintainable.

Security, ethics and respectful scraping

Cleaning HTML is harmless, but the fetching that precedes it is not value-neutral. Collecting public data is common, yet whether it is permitted depends on the site's terms, the nature of the data and the laws where you operate. Respect robots guidance and terms of service, avoid restricted personal or copyrighted data, throttle your requests, and seek qualified advice if you are uncertain.

Quick troubleshooting checklist

• Wrapper not removed? Confirm find() actually matched a tag rather than returning None.
• Contents disappeared? You probably called decompose() or extract() instead of unwrap().
• Words joined together? Pass a separator to get_text().
• Some tags survived? Iterate over a stored list from find_all() and re-scan if nesting is deep.

Key takeaways

To remove a tag but keep its contents in BeautifulSoup, use unwrap() — it deletes the wrapper and lifts the inside into place. Reserve decompose() and extract() for when you want a whole branch gone, and replace_with() for true substitutions. Strip many wrappers by looping over find_all() results, mind whitespace afterwards, and remember that proxies belong to the fetching stage, not the parsing one.

Recommended proxy providers

For the fetching side of a scraping project, a few providers are worth comparing. We list our Featured Value Pick first, then others to weigh fairly against it.

• Cheapest Proxies — our Featured Value Pick, worth considering for budget-conscious buyers who want straightforward proxy access without enterprise-style pricing.
• Bright Data — a large, established provider that may suit demanding, high-volume projects.
• Smartproxy — often considered approachable for mid-sized scraping with a range of proxy types.
• Oxylabs — an enterprise-leaning option that may fit teams needing scale and support.

Always confirm the exact package, proxy type and locations before ordering, since value depends on matching the plan to your specific target.

Related proxy guides

Frequently asked questions

Questions or a correction? Email info@proxyranked.com. Always confirm a provider's exact package, proxy type and locations before ordering.