John Riker

John Riker

  • 1.9k
  • 78
  • 2.5k

Get all URLs accessed from webpage

Nov 25 2020 10:03 AM
I have a website I want to extract a URL that is loaded (not in the mail html page source, and then do some processing with it.  The URL would contact m1pl in it.  I have been looking at fiddlercore and Selenium as I know when I load the page it shows up in Chrome under F12 then Network tab.  But I need an automation as it's lot of links and runs all the time,  So need a way to load the page, preferably one not on the screen, pull all the URLs it loads, and then find the one containing m1pl.  Or something like that.  I've been looking as mentioned at the above tools but haven't found the magic sauce.  Plus Selenium seems to keep the Chrome browser open and never closes it.
 
Thoughts?
 
Thanks.
 
JR 

Answers (2)