please process urls like any browser does

1 votes

A browser ignores double slashes like in http://somesite//subset/subpage.nl. This is processed exactly like http://somesite/subset/subpage.nl. They return the same contents. But sitebulb treats them as different, and later complains about them being identical. If some wrong link refers back using something like ..//subpage.nl then sitebulb runs into a loop processing subpage.nl again and again with an increasing number of slashes. Causing it to never terminate scanning (except when exceeding the scandepth).

Not planned Suggested by: Jan de Ruiter Upvoted: 11 Dec, '21 Comments: 0

Comments: 0