πŸ•ΈοΈ Python Tutorial β€” Selenium (Web Automation & Web Scraping)

Introduction 🌟

Selenium is a browser automation tool used for web testing, automation, and scraping dynamic JavaScript websites. It controls real browsers like Chrome, Firefox, Edge, Safari through WebDriver.

Note

πŸ’‘ Best for dynamic sites (JS-rendered content)
πŸ’‘ Can fill forms, click buttons, extract data
πŸ’‘ Works with ChromeDriver, GeckoDriver, etc.

1. Install Selenium πŸ“¦

install.sh

pip install selenium webdriver-manager

βœ” webdriver-manager automatically downloads drivers

2. Launching a Browser (Chrome) 🌐

launch_browser.py

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

driver = webdriver.Chrome(ChromeDriverManager().install())

driver.get("https://google.com")
print(driver.title)

driver.quit()

βœ” Opens Chrome
βœ” Loads URL
βœ” Closes browser

3. Selecting Elements πŸ”

Find by ID

find_id.py

element = driver.find_element("id", "username")

Find by Name

find_name.py

element = driver.find_element("name", "q")

Find by Class

find_class.py

driver.find_element("class name", "btn-login")

Find by CSS Selector

find_css.py

driver.find_element("css selector", "div.header > a.link")

Find by XPath

find_xpath.py

driver.find_element("xpath", "//button[@type='submit']")

Note

πŸ’‘ XPath is the most powerful but can become long.

4. Sending Text (Typing) ⌨️

typing.py

search = driver.find_element("name", "q")
search.send_keys("Python Selenium Tutorial")

5. Clicking Elements πŸ–±οΈ

click.py

button = driver.find_element("css selector", ".btn-search")
button.click()

6. Extracting Text & Attributes πŸ“

extract.py

text = driver.find_element("id", "title").text
link = driver.find_element("tag name", "a").get_attribute("href")

print(text, link)

7. Handling Waits (Important!) ⏳

Implicit Wait

implicit_wait.py

driver.implicitly_wait(5)

Explicit Wait

explicit_wait.py

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)

element = wait.until(EC.presence_of_element_located((By.ID, "product")))

Note

βœ” Prefer explicit waits for consistency

8. Scrolling the Page πŸ“œ

scroll.py

driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

9. Handling Forms 🧾

form.py

username = driver.find_element("id", "user")
password = driver.find_element("id", "pass")

username.send_keys("admin")
password.send_keys("secret")

driver.find_element("id", "login").click()

10. Handling Alerts ⚠️

alerts.py

alert = driver.switch_to.alert
print(alert.text)
alert.accept()

11. Handling Dropdowns πŸ”½

dropdown.py

from selenium.webdriver.support.ui import Select

select = Select(driver.find_element("id", "country"))
select.select_by_visible_text("India")

12. Switching Windows & Tabs πŸͺŸ

switch_window.py

driver.find_element("css selector", "a.new-tab").click()

tabs = driver.window_handles
driver.switch_to.window(tabs[1])

13. Taking Screenshots πŸ“Έ

screenshot.py

driver.save_screenshot("page.png")

14. Running Headless Mode πŸ€– (Without UI)

headless.py

from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--headless")

driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)

βœ” Useful for automation & servers

15. Selenium for Web Scraping 🧩

scrape_dynamic.py

from selenium import webdriver
from bs4 import BeautifulSoup
import time

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get("https://quotes.toscrape.com/js/")

time.sleep(3)  # wait for JS

soup = BeautifulSoup(driver.page_source, "html.parser")
quotes = soup.select(".quote")

for q in quotes:
    print(q.text.strip())

driver.quit()

16. Selenium With Actions (Mouse/Keyboard) πŸ–±οΈβŒ¨οΈ

actions.py

from selenium.webdriver import ActionChains

actions = ActionChains(driver)

element = driver.find_element("id", "drag")
target = driver.find_element("id", "drop")

actions.drag_and_drop(element, target).perform()

17. File Uploads & Downloads πŸ“€πŸ“₯

Upload

upload.py

file_input = driver.find_element("id", "file")
file_input.send_keys("/path/to/file.png")

Download (Chrome)

download.py

options.add_experimental_option("prefs", {
    "download.default_directory": "/my/download/path"
})

18. Closing Browser Properly πŸ”š

close.py

driver.quit()

Selenium Locators Cheat Sheet πŸ“˜

LocatorExample
IDfind_element("id", "username")
Namefind_element("name", "q")
Classfind_element("class name", "item")
Tagfind_element("tag name", "a")
CSS Selectorfind_element("css selector", ".title")
XPathfind_element("xpath", "//div[@class='info']")

Best Practices πŸ’‘

  • βœ” Use explicit waits instead of sleep()
  • βœ” Avoid XPath when CSS selector works
  • βœ” Use headless mode for automation
  • βœ” Keep ChromeDriver updated
  • βœ” Do not overload websites with rapid requests

Conclusion πŸŽ‰

>>β€œSelenium lets you control browsers like a human β€” perfect for automation, testing, and scraping dynamic websites.” ✨

You now understand Selenium in Python! Want the next topic? Try Scrapy Framework, Async Selenium (undetected_chromedriver), FastAPI Automation, or Browser Automation Bots. Just tell me! 😊