Write multiple .txt files / Python loop

Question

I'm scraping multiple URL's of a website with BeautifulSoup and want to generate a file for each URL.

categories = ["NEWS_AND_MAGAZINES", "ART_AND_DESIGN",...,"FAMILY"]
subcategories = ["topselling_free",...,"topgrossing"]
urls = []

for i in range (0,len(categories)):
    for j in range (0,len(subcategories)):
        url = categories_url_prefix + categories[i]+'/collection/'+subcategories[j]
        urls.extend([url])

for i in urls:
response = get(i)
html_soup = BeautifulSoup(response.text, 'html.parser')
app_container = html_soup.find_all('div', class_="card no-rationale square-cover apps small")
file = open("apps.txt","a+")
for i in range(0, len(app_container)):
    print(app_container[i].div['data-docid'])
    file.write(app_container[i].div['data-docid'] + "\n")

file.close()

I'm generating a unique file 'app.txt' how can I generate a file for each URL ? Thank you

Kian · Accepted Answer · 2018-11-27 17:13:55Z

2

Just replace this:

for n, i in enumerate(urls):
  response = get(i)
  html_soup = BeautifulSoup(response.text, 'html.parser')
  app_container = html_soup.find_all('div', class_="card no-rationale square-cover apps small")
  with open("file{}.txt".format(n),"a+") as f:
    for i in range(0, len(app_container)):
      print(app_container[i].div['data-docid'])
      f.write(app_container[i].div['data-docid'] + "\n")

edited Nov 27, 2018 at 17:13

answered Nov 27, 2018 at 15:02

Kian

1,3481 gold badge13 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

userHG Over a year ago

"No such file or directory: URL.txt'"

Kian Over a year ago

Directory error? It's weird! If you run your program, you should have file0.txt, file1.txt ... in your working directory. Make sure of .txt after file{}

Collectives™ on Stack Overflow

Write multiple .txt files / Python loop

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related