Crawler: URL resolution link

catalogue urlparse()   : Realize the identification and segmentation of URL urlparse() method other API usage urlunparse(): realize URL splicing urlsplit():   Parse the URL and add params and to the path urlunsplit()   : Complete link splicing urljoin(): completes the merge of links urlencode(): serialized as GET request ...

Posted on Sun, 28 Nov 2021 11:39:33 -0500 by DamienRoche

US News university ranking data acquisition

  obtaining the university ranking data of US News is a little complicated. US News 2022 Best Global Universities Rankings Web pages are dynamically refreshed, and the amount of data is relatively large. Data acquisition is divided into three steps: 1. Access to basic university information; 2. Obtain details of each university; 3. Ranking ...

Posted on Wed, 10 Nov 2021 15:10:45 -0500 by aruns