Why is it a browser crawler? Where is this method suitable for?
Because this method fully utilizes js features to do crawler, it is suitable for those without verification code ajax loading
For example xx site reviews
HTML({
onreadystatechange:function(xhr){
var i=xhr.xhr.responseURL.indexOf("http://hotels.ctrip.com/Domestic/tool/AjaxHotelCommentList.aspx");
if(i!=-1)
{
console.log("评论分页内容",xhr.xhr.responseText);
}
}
})
We have hooked all http communication in the browser. Generally, websites will return json for one step analysis and do whatever you want, but you need to use your own brain to save it to your own database and turn pages.
