Substring index out of range??

Topics: Developer Forum
Feb 16, 2008 at 8:13 AM
Edited Feb 16, 2008 at 8:15 AM
I am using version IonicIsapiRewriter-1.2.12c and have just added the following rules to prevent image hotlinking
{"
#blocks any hotlinking, except for google, msn, yahoo crawlers, google image search
#also allows hotlinking/viewing of logos
RewriteCond %{HTTPREFERER} ^(?!HTTPREFERER)
RewriteCond %{HTTP_REFERER} ^(?!http?://(?:www\.)mysite\.com/) I
RewriteCond %{HTTP_REFERER} ^(?!http?://(?:images\.|www\.)?(cache|google|googlebot|yahoo|msn|ask|picsearch|alexa)\..*) I
RewriteCond %{HTTPUSERAGENT} ^(?!google|yahoo|msn|ask|picsearch|alexa) I
#apply restrictions above to all images, except logo images
RewriteRule (?!logo).*\.(?:gif|jpg|jpeg|png)$ /images/noshow.jpg I,L
"}

Every 30 minutes or so, access from google image search seems to crash isapi with the following error when comparing against HTTP_REFERER

Sat Feb 16 19:32:31 2008 - Rule 1 : 1 matches
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%2)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%26)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%26)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%26)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%26)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%26)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%26)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%26)
Sat Feb 16 19:32:31 2008 - GenerateReplacementString: Substring index out of range (%3)


Now I am pretty sure it is because the google images referring url has these chars (which are maybe used for internal purposes??)
&prev=/images%3Fq%3word1%2Bword12%26start%3D90%26ndsp%3D18%26um%3D1%26hl%3Dit%26sa%3DN&frame=small

How can I fix this problem? Is it a development issue or can it be fixed with the ini file? :)
Feb 16, 2008 at 12:33 PM
I've since then tested with 1.2.13 beta and the dev build 1.2.14a

Same problem...crashes IIS and forces it to restart.
Coordinator
Feb 17, 2008 at 4:48 AM
Thanks for the testing.
This is obviously a problem in the filter, not in the ini file.
The filter shouldn't crash IIS.

Can you try the updated v1.2.14a ?
I just uploaded an update.

I found and fixed a problem but I'm not sure the problem I fixed is the one you had.

When you include the rules, can you include them in double curly braces ?

Also, what logging level do you use? Can you move it to 4 ? and re-test ?
thanks
let me know.
Feb 17, 2008 at 11:28 AM
Edited Feb 17, 2008 at 11:51 AM
ah double braces! the markup guide said to use one brace - no wonder it didn't work!...here are the rules...
    #blocks any hotlinking, except for google, msn, yahoo crawlers, google image search
    #also allows hotlinking/viewing of logo images
    RewriteCond %{HTTP_REFERER} 		^(?!HTTP_REFERER) 
    RewriteCond %{HTTP_REFERER} 		^(?!http?://(?:www\.)mysite\.com/)   [I]
    RewriteCond %{HTTP_REFERER} 		^(?!http?://(?:images\.|www\.)?(cache|google|googlebot|yahoo|msn|ask|picsearch|alexa)\..*)   [I]  
    RewriteCond %{HTTP_USER_AGENT}		^(?!.*google|yahoo|msn|ask|picsearch|alexa|clush|botw.*) [I]
    #apply restrictions above to all images, except logo images
    RewriteRule .*(?!logo).*\.(?:gif|jpg|jpeg|png)$ 	/images/nogo.jpg   [I,L]
 
    RewriteRule ^images/nogo\.jpg$/ 	images/nogo.jpg [I,L]

I am testing the updated version now with debug level 4, will let you know how it goes...
Feb 17, 2008 at 11:28 AM
Edited Feb 17, 2008 at 3:05 PM
Unfortunately, same problem...here is the full log
Sun Feb 17 23:57:11 2008 - ReplaceServerVariables: VariableName='HTTP_REFERER' Value='http://images.google.pl/imgres?imgurl=http://www.mysite.com/images/2873hjh3_l.jpg&imgrefurl=http://www.mysite.com/results&h=700&w=525&sz=51&tbnid=_nLARpEjxQdrEM:&tbnh=140&tbnw=105&hl=pl&um=1&prev=/results%3Fq%3DSEARCHTERM1%2BSEARCHTERM2%2B%26start%3D100%26ndsp%3D20%26um%3D1%26hl%3Dpl%26lr%3D%26sa%3DN&frame=small'
Sun Feb 17 23:57:11 2008 - ReplaceServerVariables: InputString='%{HTTP_REFERER}' out='http://images.google.pl/imgres?imgurl=http://www.mysite.com/images/2873hjh3_l.jpg&imgrefurl=http://www.mysite.com/results&h=700&w=525&sz=51&tbnid=_nLARpEjxQdrEM:&tbnh=140&tbnw=105&hl=pl&um=1&prev=/results%3Fq%3DSEARCHTERM1%2BSEARCHTERM2%2B%26start%3D100%26ndsp%3D20%26um%3D1%26hl%3Dpl%26lr%3D%26sa%3DN&frame=small'
Sun Feb 17 23:57:11 2008 - EvalCondition: ts1 'http://images.google.pl/imgres?imgurl=http://www.mysite.com/images/2873hjh3_l.jpg&imgrefurl=http://www.mysite.com/results&h=700&w=525&sz=51&tbnid=_nLARpEjxQdrEM:&tbnh=140&tbnw=105&hl=pl&um=1&prev=/results%3Fq%3DSEARCHTERM1%2BSEARCHTERM2%2B%26start%3D100%26ndsp%3D20%26um%3D1%26hl%3Dpl%26lr%3D%26sa%3DN&frame=small'
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: src='/images/2873hjh3_l.jpg','http://images.google.pl/imgres?imgurl=http://www.mysite.com/images/2873hjh3_l.jpg&imgrefurl=http://www.mysite.com/results&h=700&w=525&sz=51&tbnid=_nLARpEjxQdrEM:&tbnh=140&tbnw=105&hl=pl&um=1&prev=/results%3Fq%3DSEARCHTERM1%2BSEARCHTERM2%2B%26start%3D100%26ndsp%3D20%26um%3D1%26hl%3Dpl%26lr%3D%26sa%3DN&frame=small' ReplacePattern='http://images.google.pl/imgres?imgurl=http://www.mysite.com/images/2873hjh3_l.jpg&imgrefurl=http://www.mysite.com/results&h=700&w=525&sz=51&tbnid=_nLARpEjxQdrEM:&tbnh=140&tbnw=105&hl=pl&um=1&prev=/results%3Fq%3DSEARCHTERM1%2BSEARCHTERM2%2B%26start%3D100%26ndsp%3D20%26um%3D1%26hl%3Dpl%26lr%3D%26sa%3DN&frame=small' vec=[[  [  0, 37] [  0, 0] ] counts=1,1
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%2)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%2)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%26)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%26)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%26)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%26)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%26)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%26)
Sun Feb 17 23:57:11 2008 - GenerateReplacementString: WARNING: (maybe a problem) Substring index out of range (%3)

again I am fairly certain it is the '%' chars followed by a number, in the google url

&prev=/results%3Fq%3DSEARCHTERM1%2BSEARCHTERM2%2B%26start%3D100%26ndsp%3D20%26um%3D1%26hl%3Dpl%26lr%3D%26sa%3DN&frame=small
Coordinator
Feb 17, 2008 at 3:07 PM
Edited Feb 17, 2008 at 3:12 PM
ok, thank you, I will have another look.

But that log does not look like the FULL LOG?
Where is the beginning of the log?

There is a bunch of stuff missing.
Feb 18, 2008 at 12:28 AM
oh yeah the whole log is about 10mb... shouldn't that HTTP_REFERRER string be enough?
The bit I pasted was the end of the log, after which iis crashed and restarted. I can't paste any more than the above due to the character limit. If you need the full log, let me know and I will pm you with a link to it on our server.

Coordinator
Feb 18, 2008 at 2:22 AM
Was it the very first time that URL was applied that it crashed?
Was there a loop in the logic?
and so on.

These are questions I can answer with the full log.
Feb 18, 2008 at 2:33 AM
have sent u a pm with the full log link