[selenium]被识别如何解决？爬虫上线的selenium如何配置

1. 解决Selenium被识别

Selenium被监测出来一般是在JS代码里判断，加上execute_cdp_cmd后面的这句话就好了

from selenium import webdriver

# 实例化driver
driver = webdriver.Chrome(
    executable_path='C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe',
)
# 防止被监测
driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
  "source": """
    Object.defineProperty(navigator, 'webdriver', {
      get: () => undefined
    })
  """
})

2. Selenium的options配置

from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')  # 解决DevToolsActivePort文件不存在的报错
chrome_options.add_argument('window-size=1920x1080')  # 指定浏览器分辨率
chrome_options.add_argument('--disable-gpu')  # 谷歌文档提到需要加上这个属性来规避bug
chrome_options.add_argument('--hide-scrollbars')  # 隐藏滚动条, 应对一些特殊页面
chrome_options.add_argument('blink-settings=imagesEnabled=false')  # 不加载图片, 提升速度
chrome_options.add_argument('--headless')  # 浏览器不提供可视化页面. linux下如果系统不支持可视化不加这条会启动失败

3. 爬虫上线的Selenium配置

from selenium import webdriver


chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')  
chrome_options.add_argument('--disable-gpu')  
chrome_options.add_argument('blink-settings=imagesEnabled=false') 
chrome_options.add_argument('--headless') 

driver = webdriver.Chrome(
	executable_path='C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe', # 不是Window不需要加
    options = chrome_options,
)

driver.execute_cdp_cmd("Page.addScriptToEvaluateOnNewDocument", {
  "source": """
    Object.defineProperty(navigator, 'webdriver', {
      get: () => undefined
    })
  """
})