IIRF is not rewriting

Topics: Developer Forum, User Forum
Dec 12, 2009 at 3:51 PM

Hello,

i am dealing with a case of IIRF freezing (not processing incoming urls).

I cannot really tell what is the problem because the log (set a level 5) does not show actual errors ... (plus it stops logging once it freezes..)

Let me state my observations and i am open to discussion about this..

  1. Windows Server 2003 R2 (SP2)
  2. IIS 6.0
  3. IIRF v1.2.16 R8
  4. The testing i have done to understand that it freezes is that i made my main page log each visit to a text file, and once the IIRF stop responding i got no more log in that file.
  5. While IIRF is frozen, the StatusUrl is working (i have it on RemoteOk) (is that explainable ?)
  6. No rules display any warning or errors when parsed
  7. If i recycle the pool of the site ( a dedicated pool ) it comes back to life ( i suppose that recycling the pool also restarts the IIRF )
  8. The freezing happens after undefined conditions (no specific number of  rewrites, no specific time passed, no specific last URL processed...)
  9. The only registered error (if it is one) from IIRF is this
    • Sat Dec 12 14:33:23 -  2084 - DoRewrites
    • Sat Dec 12 14:33:23 -  2084 - GetServerVariable_AutoFree: getting 'url'
    • Sat Dec 12 14:33:23 -  2084 - GetServerVariable_AutoFree - no joy (GetLastError()=1413)
    • Sat Dec 12 14:33:23 -  2084 - GetServerVariable_AutoFree: 128 bytes
    • Sat Dec 12 14:33:23 -  2084 - GetServerVariable_AutoFree: result ''
  10. The second weird thing logged is a large number of continuous (with different proccess IDs of course and time)
    • Sat Dec 12 14:45:20 -  6012 - DllMain THREAD_DETACH
    • Sat Dec 12 14:45:40 -  5216 - DllMain THREAD_DETACH
    ending in
    • TerminateFilter
    • Sat Dec 12 14:45:40 - 3856 - DllMain PROCESS_DETACH
    • Sat Dec 12 14:45:40 - 3856 - AwaitWatcherTermination: closing dir handle.
    • Sat Dec 12 14:45:40 - 3856 - AwaitWatcherTermination: Waiting 150 ms ...
    • Sat Dec 12 14:45:40 - 1824 - AwaitIniChangeAndReinit: Pre-empted...
    • Sat Dec 12 14:45:40 - 1824 - FileChangeWatcher(): Await returns (TerminateWatch= TRUE)
    • Sat Dec 12 14:45:40 - 1824 - FileChangeWatcher: return()...
    • Sat Dec 12 14:45:40 - 3856 - AwaitWatcherTermination: watcher thread is terminated...
Not sure if the above provide any hint ...
I could really use some help with this .. (if it might be a rule thing let me know and i will post those too... they are just  10)
regards
Gabriele
Coordinator
Dec 12, 2009 at 5:04 PM

Hello Gabriele,

The first thing I would suggest is to move to IIRF V2.0. 

If you can't do that, I have some other questions.

  1. You said the freezing happens after undefined conditions.  Is there a "normal" amount of time that the filter works successfully, before freezing?
  2. Can you characterize the load on the system when IIRF freezes?
  3. Can you post more of the log?  (all?) I cannot really diagnose the problem without looking at more of the IIRF log. The error you noticed can occur in normal processing.

-Cheeso

 

Dec 12, 2009 at 5:59 PM
Edited Dec 12, 2009 at 6:09 PM

thanks for the prompt reply..

  1. No there is no patterns (that i have identified..) for when the freeze occurs... 
  2. it can happen whether there is one person online or many ...  ( i have seen both scenarios occur..).. from bots as well..
  3. What parts of the log you need ? they are quite huge.. (being in level 5 logging, they can get to 200 MBs..)

Important question: will the rules (from IIRF 1.2.16) work out of the box for IIRF V2.0 ? or do i have to make alterations ?

 

the ruleset is the following (just in case you spot an error, a loop or something that might be occuring..)

  • RewriteLogLevel 5
  • RewriteLogFile c:\temp\iirf
  • RewriteEngine ON
  • StatusUrl /iirf_Status RemoteOk
  • RewriteRule ^/(template|index)\.asp(.+)$ - [L]
  • RewriteCond %{HTTP_HOST} ^domain\.gr [I,OR]
  • RewriteCond %{HTTP_HOST} ^new\.domain\.gr [I,OR]
  • RewriteCond %{HTTP_HOST} ^www\.new\.domain\.gr [I]
  • RedirectRule ^/(.*) http://www.domain.gr/$1 [I,R]
  • RewriteRule ^/(image|rendering|layout|skin|font\sfiles|code)/(.+)$ - [L]
  • RewriteRule ^/(?!(template|index)\.asp)([^\?\/]+)/([^\?]+)$ /$3?lang=$2
  • RewriteRule ^/(?!(template|index)\.asp)([^\?\/]+)/([^\?]+)/?\?([^lang].+)$ /$3?lang=$2&$4
  • RewriteRule ^/(?!(template|index)\.asp)([^\?\/]+)/?\?([^lang].+)$ /?lang=$2&$3
  • RewriteRule ^/(?!(template|index)\.asp)([^\?\/]+)/([^\?]+)\?(.+)$ /$3?$4&v=$2
  • RewriteRule ^/([^\?\/]+)/?\?(.+)$ /template.asp?$2&v=$1 [L]
  • RewriteRule ^/\?(.+)$ /template.asp?$1 [L]
  • RewriteRule ^/([^\?\/]+) /index.asp?lang=$1 [L]

 

Coordinator
Dec 12, 2009 at 6:31 PM
Edited Dec 12, 2009 at 6:36 PM

The rules work the same way in v2.0, as in v1.2.16. 

There is a basic difference in the install and deployment.  In IIRF v1.2, the INI file is called IsapiRewrite4.ini and is placed in the same directory as the DLL.  In v2.0, the ini file is called IIRF.ini and is placed into the document root for each virtual directory or site on the server, as many as you like.  There are new directives supported in v2.0 as compared to v1.2.  You can learn more in the documentation.

Yes, I do have some useful comments on your rules:

  1. The RedirectRule - it is unnecessary to place [R] in the modifiers.  You can place R=301 or R=302 if you want to be explicit about the HTTP return code for the redirect.  Lacking an R=30x, IIRF will use 302 (Moved Temporarily). 
  2. For all rules following the RedirectRule, where you use (?!(template|index)\.asp) to exclude some URLs from matching: You have already excluded URL paths matching that pattern, with the very first RewriteRule.  Therefore, that negative lookahead is unnecessary.  It's not an error, but it's unnecessary and makes the rules harder to read.
  3. For this rule:
    RewriteRule ^/(?!(template|index)\.asp)([^\?\/]+)/([^\?]+)$ /$3?lang=$2 
    there is no $3. The 3 refers to the 3rd capturing group, but there are only 2 possible capturing groups. This rule won't work the way it is intended. The negative lookahead, which is unnecessary as I pointed out above, is non-capturing. It doesn't capture anything and it doesn't increment the capture group index.   I think what you want here is more like: 
    RewriteRule ^/([^\?\/]+)/([^\?]+)$ /$2?lang=$1 
    But I don't know for sure. It helps to insert comments into the ruleset to describe an example of what you're trying to do.
  4. There's a similar problem with all RewriteRules following that one, that have the negative-lookahead - (?!...).  The indexes are off-by-one.  $3 should be $2,  etc.   The rules without the negative lookahead don't exhibit that problem.
  5. According to my comments above, this rule:
    RewriteRule  ^/(?!(template|index)\.asp)([^\?\/]+)/([^\?]+)/?\?([^lang].+)$  /$3?lang=$2&$4
    should be rewritten as
    RewriteRule  ^/([^\?\/]+)/([^\?]+)/?\?([^lang].+)$  /$2?lang=$1&$3 
    ...to remove the unnecessary negative lookahead and to fix the off-by-one error.    But even then, it is probably incorrect.  Without documentation, it's hard to know what is intended, but I think this rule, even modified this way to correct the two problems I described above, is not correct. I think It will not do what is intended. It seems to want to NOT match "lang" in the query string. But [^lang] is not the correct way to do that. In regex, square brackets denote a character range, and a circumflex (^) as the first character within the range indicates a negation of the range. Therefore, [^lang] means, in english, "any single character that is not one of l, a, n, or g." I can only guess what you truly intend by this rule. If I had to guess, from the context, I would say you are trying to exclude any URL that includes lang=xxx in the query string. Is that right? If so, then the rule ought to be like this:
    RewriteRule  ^/([^\?\/]+)/([^\?]+)/?\?(?!lang)(.+)$  /$2?lang=$1&$3 
    
    The subexpression (?!lang) is a (non-capturing) negative look-ahead. It says "match only if the text at this point in the subject is not 'lang'. " I am only guessing that this is what you want.
  6. The rule following that one, also exhibits the problem with [^lang] versus (?!lang).

Are you sure the rules are working the way you think?  Do you have a test suite, using the testdriver, to verify that the rules do what you want?  I would be surprised that they work the way you want.

Now, is any of this related to IIRF "freezing"?  It's hard to tell.  It could be that IIRF is behaving correctly and the problem you are observing is due to incorrect rules.  I would suggest you fix the rules first. Also I strongly suggest using the testdriver to test input URLs and verify your ruleset.

The use of the testdriver.exe is described in the IIRF documentation.

 

Dec 13, 2009 at 4:38 PM

Thank you very much for the pointers cheeso, i really appreciate this.

Looks like there is a long way ahead of me in the regular expressions studying :)

Funny part is that the rules have worked as expected in terms of the final outcome from the site .. but i have incorporated all the changes you mention and it still works as expected which is excellent.

I will keep trying to track down the issue ( i now start to have some ideas to something else than IIRF being the culprit... )

 

Thanks for the time, and thank you for an excellent product.

Gabriele

Coordinator
Dec 13, 2009 at 5:19 PM

Glad to hear that the changes seem to work.  I strongly suggest you look into the testdriver.exe to remove some of the mystery behind the rules. As I say in the documentation, you should think of the rules as "computer code" and rewrite rulesets should be tested as rigorously as any other computer code.  I provide the testdriver to help with that.  It's also a good way to gain some additional insight into how the rewrites work.

Good luck.