How to get json string from a website using Python?

Question

I am trying to get image links of the products from the website. I can get image info on some of the products. However, I can't get some of them. In the code URL1 is working but URL2 throws "json.decoder.JSONDecodeError". I think the problem is I cant parse the JSON string. I am not good at regular expression. How can I get JSON string?

Screenshot

Code

import re,json,requests
url1 =  "https://www.trendyol.com/samsung/akilli-smart-air-sihirli-led-tv-televizyon-kumandasi-yerine-tuslu-kumanda-1078-p-43447565?boutiqueId=61&merchantId=384846"
url2 = "https://www.trendyol.com/samsung/k-ve-m-serisi-uyumlu-led-lcd-tv-akilli-kumandasi-bn59-01259b-p-45735139?boutiqueId=61&merchantId=115135"
r = requests.get(url2)
data = json.loads(re.search(r'PRODUCT_DETAIL_APP_INITIAL_STATE__=(.*?);', r.text).group(1))
images = ['https://www.trendyol.com' + img for img in data['product']['images']]
print(images)

QHarr · Accepted Answer · 2021-07-18 13:58:00Z

1

The following regex is a better match for your given urls as it terminates at the end of the nested dictionaries and before the start of the next block.

import re,json,requests

url1 =  "https://www.trendyol.com/samsung/akilli-smart-air-sihirli-led-tv-televizyon-kumandasi-yerine-tuslu-kumanda-1078-p-43447565?boutiqueId=61&merchantId=384846"
url2 = "https://www.trendyol.com/samsung/k-ve-m-serisi-uyumlu-led-lcd-tv-akilli-kumandasi-bn59-01259b-p-45735139?boutiqueId=61&merchantId=115135"

for url in [url1, url2]:
    r = requests.get(url)
    data = json.loads(re.search(r'PRODUCT_DETAIL_APP_INITIAL_STATE__=(.*?\}\});', r.text).group(1))
    images = ['https://www.trendyol.com' + img for img in data['product']['images']]
    print(images)
    print("")

answered Jul 18, 2021 at 13:58

QHarr

84.5k14 gold badges58 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Matteo Bianchi · Accepted Answer · 2021-07-18 14:39:39Z

1

You can try this:

import requests
from bs4 import BeautifulSoup

headers = {
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0',
}

r = requests.get('https://www.trendyol.com/samsung/k-ve-m-serisi-uyumlu-led-lcd-tv-akilli-kumandasi-bn59-01259b-p-45735139?boutiqueId=61&merchantId=115135')
soup = BeautifulSoup ((r.text).encode('utf-8'))

img = soup.findAll ('img')
for x in img:
    print(x['src'])

answered Jul 18, 2021 at 14:39

Matteo Bianchi

4421 gold badge6 silver badges20 bronze badges

1 Comment

HASAN HÜSEYİN YÜCEL Over a year ago

Thank you. I tried that way before. It is getting low-resolution images. I need high-resolution images of the product.

Collectives™ on Stack Overflow

How to get json string from a website using Python?

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related