Python using Playwright with proxy
Playwright is a web automation framework provided by Microsoft, similar to Selenium. We can use these web automation frameworks to scrape (extract) data from a website. Sometimes we have to use a proxy to bypass blocking from a website.
What is Playwright?
Playwright is a web autotest (automation) framework provided by Microsoft. It is similar to Selenium.
We may use these web automation frameworks to scrape (extract) data from a website. Sometimes, we have to use a proxy to bypass blocking from a website. The source code below shows how to use a proxy for Playwright.
Using a Proxy with Playwright
Pass the proxy configuration (server address, username, and password) directly to the launch() call:
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.webkit.launch(
headless=False,
proxy={
"server": "server-address:port",
"username": "My_user",
"password": "My_password",
},
)
context = browser.new_context()
page = context.new_page()
page.goto("https://whoer.net")
page.screenshot(path="whoer.png")
browser.close()
- Replace
server-address:portwith your proxy server’s host and port. - Replace
My_userandMy_passwordwith your proxy credentials. - The screenshot saved as
whoer.pnglets you verify the proxy IP is being used correctly.
Installing Playwright
If you haven’t installed Playwright yet:
pip install playwright
playwright install
If you have any question, don’t hesitate to contact me.
Good luck!
See also:
- Best Proxies for Web Scraping — choosing the right proxy provider