Set cookies for crawl
Sometimes preprod environments can be protected by cookies : the user is redirected if he doesn't have the cookie. Some other websites can be concerned (big SEO problem if cookies are mandatory to navigate). Therefore it would be a good idea to add a feature that can let us set some custom cookies to allow the crawl.
FYI, Screaming Frog's "allow cookie"'s feature is not enough since we need to set custom cookies. I had a good experience with Apify with this feature.
Thank you !
Under consideration Suggested by: Maxime Guernion / Thomas Cerdan • Upvoted: 03 Nov, '22 • Comments: 0