0

I'm trying to automate filling a car insurance quote form on a site: (following the same format as the site URL lets call it: "https://secure.examplesite.com/css/car/step1#noBack")

I'm stuck on the rego section as once the rego has been added, a button needs to be clicked to perform the search and it seems this is heavy Javascript and I know mechanizer can't handle this. I'm not versed in JavaScript, but I can see that when clicking the button a POST request is made to this URL: ("https://secure.examplesite.com/css/car/step1/searchVehicleByRegNo") Please see image also.

How can I emulate this POST request in Mechanize to run the javascript? So I can see the response / interact with the response? Or is this not possible? Can I consider bs4/requests/robobrowser instead. I'm only ~4 months into learning! Thanks

# Mechanize test
import mechanize

br = mechanize.Browser()
br.set_handle_robots(False)   # ignore robots
br.set_handle_refresh(False)  # can sometimes hang without this
res = br.open("https://secure.examplesite.com/css/car/step1#noBack")

br.select_form(id = "quoteCollectForm")
br.set_all_readonly(False)    # allow everything to be written to

controlDict = {}

# List all form controls
for control in br.form.controls:
    controlDict[control.name] = control.value
    print("type = %s, name = %s, value = %s" %(control.type, control.name, control.value))

# Enter Rego etc "example"
br.form["vehicle.searchRegNo"] = "example"

# Now for control name = vehicle.searchRegNo, value = example

# BUT Now how do I click the button?? Simulate POST? The post url is formatted like:
# https://secure.examplesite.com/css/car/step1/searchVehicleByRegNo

Javascript POST

1 Answer 1

1

Solved my own problem-

Steps:

  1. open dev tools in browser
  2. Go to network tab and clear
  3. interact with form element (in my case car rego finder)
  4. click on the event that occurs from interaction
  5. copy the exact URL, Request header data, and payload
  6. I used Postman to quickly test the request and responses were correct / the same as the Webform and found the relevant headers
  7. in postman convert to python requests code

Now I can interact completely with the form

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.