使用浏览器控制台抓取shodan、搜索引擎信息

0x00 前言

使用JavaScript提取自己想要截图的示例

0x01 提取Shodan结果ip

Shadan
//方法1：获取标签提取
console.log(document.getElementsByClassName("ip")[0].firstChild.innerHTML)
function getIpByTag()
{
    var ip = document.getElementsByClassName("ip");
    var str = "\n";
    
    for(var i=0; i < ip.length; i++)
    {
        var node = ip[i].firstChild;
        str = str + node.innerHTML + "\n";
    }
    return str;
}
getIpByTag();


//方法2：正则提取
function getIpByTag(){
    var tag = document.getElementsByClassName("span9")[0];
    var re = new RegExp();
    var str = tag.innerHTML; 
    var re = /\d+\.\d+\.\d+\.\d+/g;
    var arr = str.match(re);
    console.log(arr);
    //数组去重
    arr.sort();
    for(var i = 0; i < arr.length-1;) {
        //用当前的元素与他的前一个元素进行对比
        if(arr[i] == arr[i + 1]) {
        //如果相同的话,就删除掉第i个元素
        arr.splice(i, 1);
    }else{ i++;}
}
console.log(arr);

0x02 抓取Google结果

Google
console.log(document.getElementsByClassName("r")[0.].firstChild.href)
function getIpByTag()
{
    var r = document.getElementsByClassName("r");
    var str = '\n';
    
    for(var i=0;i< r.length;i++)
    {
        str = str + r[i].firstChild.href + '\n';
    }
    
    return str;
}
getIpByTag();

0x03 提取百度结果

Baidu
console.log(document.getElementsByClassName("t")[0].getElementsByTagName('a')[0].href)
function getIpByTag()
{
    var t = document.getElementsByClassName("t");
    var str = '\n';
    
    for(var i=0;i< t.length;i++)
    {
        str = str + '"' + t[i].getElementsByTagName('a')[0].href + '",\n';
    }
    str = str.substring(0,str.length-2)
    str = str + '\n'
    return str;
}
getIpByTag();

将百度加密后的url转成真正的地址：

import requests
urlList = [
"http://www.baidu.com/link?url=M1SN1OPmF9xM43i4jwjeDVvn-uD-i7xOf1nDxZDdIh4iCQRPXnmJnpzEFaRpcLNbSzXJGnlGiRClt_kX_KjXo_",
"http://www.baidu.com/link?url=9J00kAi9Fu07zxr4q4v_WZ2b0lW6WM-eIuzzcRtKQSS8Hd2u7hqAyBYyDOm1JbAwGgrUAubK8cR3V2_7RFJ1j_",
"http://www.baidu.com/link?url=kJuAmhEDNtu9VT5tpF_Grdi5fv246Dyf6ESnWqyBrR9HZD8BniQXVqOEinUox_hn",
"http://www.baidu.com/link?url=9Zlb9C0SnpP01To84341TBe2Tr1888CY8vkv86ZJAB94GoDO0II9m19lJpAKmSlm",
"http://www.baidu.com/link?url=e-jNCpBOgKCFOAGakRSt7jsqeKM4Z7kAKxmzFXyizOybrMP3Ig5MVmIHd6cwgsug",
"http://www.baidu.com/link?url=mhPGHLye4mCUdZOKGZz-RY_d7vzNThy_ifVZ8qGpAkvEGYUspJKT5wvHX0LSvPVd",
"http://www.baidu.com/link?url=NOw1rL9Juxdl-_FYexMJq8n1I3vliWRPjMAVZT8YQ-S9nHOXn-EuI8YnIz6-8EXF",
"http://www.baidu.com/link?url=3iCH4yJeE6UA_Pura3WMiNcoLBOYKePK0teNAwELb3667oy-RXOSuanprur6GjUN",
"http://www.baidu.com/link?url=w_7v1e_uvw8YSQyZEA-SN1vnIIljpmknKaVLTtdZqxM8qLXi0C0LwLAUQJyrZYTY8aU4DjPnXtQeUQlP-zqzXK",
"http://www.baidu.com/link?url=DjXRm8KwbnSAOaPtt3NtR7XzwCnWizbgJDxeC9DPB0GBeSJIiYb2ObZHQ5mLsYjP"
]
for someurl in urlList:
    response = requests.get(someurl)
    if response.history:
        print(response.url)
    else:
        print("Request was not redirected")

参考

[1] 使用Chrome console提取页面数据
https://www.cnblogs.com/liun1994/p/7265828.html

[2] Google浏览器URL采集的一种思路
https://blog.csdn.net/qq_29647709/article/details/84379170

[3]使用浏览器控制台抓取信息
https://lufe1.cn/2017/09/20/%E4%BD%BF%E7%94%A8%E6%B5%8F%E8%A7%88%E5%99%A8%E6%8E%A7%E5%88%B6%E5%8F%B0%E6%8A%93%E5%8F%96%E4%BF%A1%E6%81%AF/