Skip to content

Commit 85260cf

Browse files
Added scraper
1 parent 86580ca commit 85260cf

File tree

2 files changed

+29
-1
lines changed

2 files changed

+29
-1
lines changed

README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -26,4 +26,5 @@
2626
23. LinkedIn Scraper: Scrapes user profile (v1)
2727
24. Global Timer: Enter location/city name to get the associated time depending upon time zone.
2828
25. Codechef Submission CLI Bot: Codechef bot to submit the solution of problems and get the result in CLI.
29-
26. Indeed Scraper: Scrapes job listings on indeed.com and stores them in csv file.
29+
26. Indeed Scraper: Scrapes job listings on indeed.com and stores them in csv file.
30+
27. Wallpaper Scraper: Scrapes images from webpage and stores them in current directory

wallpaper-scraper/scrape.py

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
import urllib3
2+
import shutil
3+
from bs4 import BeautifulSoup as bs
4+
5+
6+
def main():
7+
8+
ur = input("Input URL Here ")
9+
10+
http = urllib3.PoolManager()
11+
12+
page = http.request('GET', ur)
13+
14+
soup = bs(page.data, 'html.parser')
15+
16+
link = ["http:"+ls.get('href') for ls in soup.find_all('a', {"class" : "zoom"})]
17+
for vd in soup.find_all('source'):
18+
link.append('http:'+vd.get('src'))
19+
print(link)
20+
21+
for ln in link:
22+
index = ln.rindex('/')
23+
filename = ln[index+1:len(ln)]
24+
with http.request('GET', ln, preload_content=False)as resp, open(filename, 'wb') as out_file:
25+
shutil.copyfileobj(resp, out_file)
26+
if __name__ == "__main__":
27+
main()

0 commit comments

Comments
 (0)