ProxyPass issues concerning headers

Topics: Developer Forum
Feb 7, 2010 at 10:00 PM

Hi,

I'm facing some issues with ProxyPass. Here is my config: IIS 6.0, IIRF 2.1.0.11, Win 2003 Server.

I want IIS to handle authentication and then pass the request down to my app server, via ProxyPass.

In fact I just want to do the same as the following directives which comes from Helicon ISAPI_Rewrite3:

RewriteProxy (.*) http://localhost:8001/$1 [H,A]

The magic is the A flag, which adds these headers to the proxied server (see http://www.helicontech.com/isapi_rewrite/doc/RewriteProxy.htm )

X-ISRW-Proxy-AUTH-TYPE,
X-ISRW-Proxy-AUTH-USER,
X-ISRW-Proxy-LOGON-USER,
X-ISRW-Proxy-REMOTE-USER

which comes from SERVER variables:

AUTH_TYPE,
AUTH_USER,
LOGON_USER,
REMOTE_USER

 

So, to achieve this with IIRF, I tried to add a new header through the RewriteHeader directive, like so:

RewriteHeader HTTP_X_REMOTE_USER ^$ %{REMOTE_USER}
ProxyPass ^/(?!home)(.*)$ http://localhost:8001/$2

When IIS is authenticating the request I see in the log that the header is set accordingely, BUT this new header is NOT sent to the proxied server (the one at localhost 8001).
I'll call this the first bug. Could someone confirm this? If yes, I may file a bug in the tracker.

 

Another issue: when IIS doesn't authenticate the request (anonymous access), then there is no REMOTE_USER header and the previous directive will loop until max depth reaches. This is because the REMOTE_USER is blank and the pattern to match the HTTP_X_REMOTE_USER is also blank => ^$. So what came to my mind is to protect this RewriteHeader directive with a RewriteCond, like this:

RewriteCond %{HTTP_X_REMOTE_USER} ^$
RewriteHeader HTTP_X_REMOTE_USER ^$ %{REMOTE_USER}
ProxyPass ^/(?!home)(.*)$ http://localhost:8001/$2

Looking at the log shows a weird behaviour, here is the interesting part:

Sun Feb 07 23:18:20 -  2324 - EvaluateRules: depth=0
Sun Feb 07 23:18:20 - 2324 - GetHeader_AutoFree: getting 'HTTP_X_REMOTE_USER:'
Sun Feb 07 23:18:20 - 2324 - GetHeader_AutoFree: 128 bytes ptr:0x000C9AA8
Sun Feb 07 23:18:20 - 2324 - GetHeader_AutoFree: 'HTTP_X_REMOTE_USER:' = ''
Sun Feb 07 23:18:20 - 2324 - EvaluateRules: Rule 1 : 1 matches
Sun Feb 07 23:18:20 - 2324 - GetServerVariable: getting 'HTTP_X_REMOTE_USER'
Sun Feb 07 23:18:20 - 2324 - GetServerVariable: cannot find that variable
Sun Feb 07 23:18:20 - 2324 - GetServerVariable: 128 bytes
Sun Feb 07 23:18:20 - 2324 - GetServerVariable: result 'HTTP_X_REMOTE_USER'
Sun Feb 07 23:18:20 - 2324 - ReplaceServerVariables: VariableName='HTTP_X_REMOTE_USER' Value='HTTP_X_REMOTE_USER'
Sun Feb 07 23:18:20 - 2324 - ReplaceServerVariables: in='%{HTTP_X_REMOTE_USER}' out='HTTP_X_REMOTE_USER'
Sun Feb 07 23:18:20 - 2324 - EvalCondition: ts1 'HTTP_X_REMOTE_USER'
Sun Feb 07 23:18:20 - 2324 - GenerateReplacementString: result 'HTTP_X_REMOTE_USER'
Sun Feb 07 23:18:20 - 2324 - EvalCondition: checking 'HTTP_X_REMOTE_USER' against pattern '^$'
Sun Feb 07 23:18:20 - 2324 - EvalCondition: match result: -1 (No match)
Sun Feb 07 23:18:20 - 2324 - EvalCondition: Cond %{HTTP_X_REMOTE_USER} ^$ => FALSE
Sun Feb 07 23:18:20 - 2324 - EvalConditionList: rule 1, FALSE, Rule does not apply

because the HTTP_X_REMOTE_USER header doesn't exist, it is created with a VALUE of HTTP_X_REMOTE_USER which is the header name. Of course at this point we can't rely on the fact that this header is blank, which it is no more. (see the next line of the log, EvalCondition)

I'll call this the second bug. IMHO a non existant header should be considered as having a blank value, not the name of itself.

 

Thanx for listening, best regards,

Pascal

Coordinator
Feb 7, 2010 at 11:44 PM

Pascal,

thanks for opening a new thread .

On the first issue, you may be right that the new header may not be properly sent along to the proxied request .  I'll have to look into that, to investigate further.  I wouldn't mind if you opened a workitem for that.

On the second issue, the value of a variable that is not set, is somewhat arbitrary.  At one time it was empty, but I decided that was too susceptible to problems with unknown variables.  So I made it evaluate to the name of the variable.  Given the documented behavior, what you can do is invert the boolean test.  Rather than testing for empty, test for a value that is not something reasonable.  

Another option is to test for empty by specifying the name of the variable.  Like so:  RewriteCond %{HTTP_X_REMOTE_USER} ^HTTP_X_REMOTE_USER$

Regardless what you decide to do, the capability available to you in IIRF is complete.  You don't need an empty variable to evaluate to an empty string, in order to get the behavior you want. 

 

 

Feb 8, 2010 at 12:24 AM

Cheeso,

I'm pleased to see a so quick reply, thanx.

I'll get into the tracker for the first issue.

For the second one, yes I did try what you meant and I also had issues with it, let's see the log:

Mon Feb 08 02:02:27 -  2396 - EvaluateRules: depth=0
Mon Feb 08 02:02:27 - 2396 - GetHeader_AutoFree: getting 'HTTP_X_REMOTE_USER:'
Mon Feb 08 02:02:27 - 2396 - GetHeader_AutoFree: 128 bytes ptr:0x000C94A0
Mon Feb 08 02:02:27 - 2396 - GetHeader_AutoFree: 'HTTP_X_REMOTE_USER:' = ''
Mon Feb 08 02:02:27 - 2396 - EvaluateRules: Rule 1 : 1 matches
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: getting 'HTTP_X_REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: cannot find that variable
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: 128 bytes
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: result 'HTTP_X_REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - ReplaceServerVariables: VariableName='HTTP_X_REMOTE_USER' Value='HTTP_X_REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - ReplaceServerVariables: in='%{HTTP_X_REMOTE_USER}' out='HTTP_X_REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - EvalCondition: ts1 'HTTP_X_REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - GenerateReplacementString: result 'HTTP_X_REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - EvalCondition: checking 'HTTP_X_REMOTE_USER' against pattern '^HTTP_X_REMOTE_USER$'
Mon Feb 08 02:02:27 - 2396 - EvalCondition: match result: 1 (match)
Mon Feb 08 02:02:27 - 2396 - EvalCondition: Cond %{HTTP_X_REMOTE_USER} ^HTTP_X_REMOTE_USER$ => TRUE
Mon Feb 08 02:02:27 - 2396 - EvalConditionList: rule 1, TRUE, Rule will apply
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: getting 'REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: 1 bytes
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: result ''
Mon Feb 08 02:02:27 - 2396 - ReplaceServerVariables: VariableName='REMOTE_USER' Value=''
Mon Feb 08 02:02:27 - 2396 - ReplaceServerVariables: in='%{REMOTE_USER}' out=''
Mon Feb 08 02:02:27 - 2396 - GenerateReplacementString: result ''
Mon Feb 08 02:02:27 - 2396 - EvaluateRules: Result (length 0):
Mon Feb 08 02:02:27 - 2396 - EvaluateRules: Setting Header: 'HTTP_X_REMOTE_USER:' = ''
Mon Feb 08 02:02:27 - 2396 - EvaluateRules: depth=1
Mon Feb 08 02:02:27 - 2396 - GetHeader_AutoFree: getting 'HTTP_X_REMOTE_USER:'
Mon Feb 08 02:02:27 - 2396 - GetHeader_AutoFree: 128 bytes ptr:0x000C9528
Mon Feb 08 02:02:27 - 2396 - GetHeader_AutoFree: 'HTTP_X_REMOTE_USER:' = ''
Mon Feb 08 02:02:27 - 2396 - EvaluateRules: Rule 1 : 1 matches
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: getting 'HTTP_X_REMOTE_USER'
Mon Feb 08 02:02:27 - 2396 - GetServerVariable: cannot find that variable

We see that everything is fine before the last 7 lines, the red line shows that we are re-starting an iteration, fine.

But reading the last line (of the stripped log) shows that the new header (HTTP_X_REMOTE_USER) is not preserved across iterations, so IIRF thinks again that this header doesn't exist and re-do exactly the same scenario. So I end up reaching the max iteration count.

Maybe this behaviour is a bug.

 

Regards,

Pascal

Coordinator
Feb 9, 2010 at 9:55 PM
Edited Feb 10, 2010 at 12:15 AM

Hello Pascal,

regarding your second issue, please pardon me for not noticing this earlier but...

The problem is you are setting a HEADER and retrieving a SERVER VARIABLE.   In IIS, "Request HEADERS" are different than "Server Variables", though there is a linkage: For any given header, its value can be retrieved as a server variable by prefixing the name with HTTP_ and replacing - (dash) with _ (underscore) .

Therefore, if you set header X, you can retrieve it as HTTP_X.  If you set header with the name X-Y-Z, you can retrieve it with HTTP_X_Y_Z. 

In your case, you should set header X-Remote-User, and then retrieve or test it as a server variable with HTTP_X_REMOTE_USER.  Eg 

## If the request header X-Remote-User is not present...
RewriteCond %{HTTP_X_REMOTE_USER}  ^HTTP_X_REMOTE_USER$
RewriteHeader X-Remote-User:  Whatever

This is not a feature of IIRF.  It's a documented behavior in IIS.    I looked briefly for the doc page on it, but didn't find it.  I'll post that a little later, when I have  more time to search.   It is documented in the IIRF manual, as well.

 

Coordinator
Feb 10, 2010 at 12:23 AM

IIS Documentation: http://msdn.microsoft.com/en-us/library/ms524602.aspx

Coordinator
Feb 10, 2010 at 12:40 AM

I looked further in the IIRF doc and could not find any description of this behavior.  so I have now added it.

Thanks for the tip.

 

Feb 10, 2010 at 6:36 PM

Thanx a lot for noticing my mistake,

I've done very little development on the IIS side, so yes I mixed server variables with plain http headers (I was fooled by the way we access them as you outlined it).

I'll modify my ini file.

 

Best Regards and keep the good work going on. Thanx.

 

Pascal

Feb 10, 2010 at 8:37 PM

I'm back again ;-)

Okay, I modified my ini file with your suggestion, but sadly the behaviour is unchanged.

I can see in the log that the 'X-Remote-User' header is set accordingely, then a new iteration begins, and the corresponding server variable (HTTP_X_REMOTE_USER) is still undefined.

Maybe IIS doesn't parse the new headers on a new iteration, thus the server variable HTTP_X_REMOTE_USER reflects the state of the initial request.

Something to investigate.

BTW, the current online doc for IIRF 2.1 is still missing your changes concerning the default value of a missing header. url: http://cheeso.members.winisp.net/Iirf20Help/html/bdaf0dbe-e539-4bf5-9231-1dae3b0e27a5.htm
Excerpt (RewriteRule at the end after the available server variables table)

 

If you use something other than one of these values, then the result will be the empty string.
If you have an error in spelling in the name of the server variable, then it could be that the reference always evaluates to a blank.
A blank is also the result when the server variable has no value. For example, if you are not using HTTPS, but you reference the variable HTTPS_KEYSIZE, you'll get a blank string.

 

Which should states that the default value is the name of the access server variable.

 

Best Regards,

Pascal

 

Coordinator
Feb 11, 2010 at 12:14 AM

Yes, I caught that clear documentation error.  IIRF previously behaved that way but I changed the bbehaior due to the fact that typos in variable names became a very common problem.  I had forgotten to update the documentation to reflect the updated behavior.

Regarding your new observation - can you please provide the ini file and the IIRF log that exhibits this problem?

I just tried the situation you described.  I think.  Here's my ini file:

RewriteLog  c:\inetpub\iirfLogs\iirf-v2.0.winisp
RewriteLogLevel 3

# Set this header if it is unset
RewriteCond  %{HTTP_X_SOMEHEADER}  ^HTTP_X_SOMEHEADER$
RewriteHeader X-SomeHeader: ^$ YaHuh

RewriteCond  %{QUERY_STRING}  ^$
RewriteHeader Query-Is-Empty: ^$ YaHuh

RewriteCond %{QUERY_STRING} ^.+$
RewriteHeader Query-Is-Not-Empty: ^$ YaHuh

and here's the relevant log file lines:

Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: c:\dinoch\webs\winisp\Iirf.ini(11): IIRF Status Inquiry is disabled.
Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: c:\dinoch\webs\winisp\Iirf.ini(14): RewriteCond   %{HTTP_X_SOMEHEADER}  ^HTTP_X_SOMEHEADER$ '(null)'
Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: c:\dinoch\webs\winisp\Iirf.ini(15): RewriteHeader (rule 1)  'X-SomeHeader:'  '^$'  'YaHuh'   (null)
Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: c:\dinoch\webs\winisp\Iirf.ini(17): RewriteCond   %{QUERY_STRING}  ^$ '(null)'
Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: c:\dinoch\webs\winisp\Iirf.ini(18): RewriteHeader (rule 2)  'Query-Is-Empty:'  '^$'  'YaHuh'   (null)
Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: c:\dinoch\webs\winisp\Iirf.ini(20): RewriteCond   %{QUERY_STRING}  ^.+$ '(null)'
Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: c:\dinoch\webs\winisp\Iirf.ini(21): RewriteHeader (rule 3)  'Query-Is-Not-Empty:'  '^$'  'YaHuh'   (null)
Wed Feb 10 20:05:20 -  2648 - ReadVdirConfig: Done reading, found 3 rules (1 errors, 0 warnings) on 277 lines
Wed Feb 10 20:05:20 -  2648 - HttpFilterProc: SF_NOTIFY_URL_MAP
Wed Feb 10 20:05:20 -  2648 - HttpFilterProc: cfg= 0x00EF3E48
Wed Feb 10 20:05:20 -  2648 - HttpFilterProc: SF_NOTIFY_AUTH_COMPLETE
Wed Feb 10 20:05:20 -  2648 - DoRewrites
Wed Feb 10 20:05:20 -  2648 - DoRewrites: Url (no decoding): '/winisp/info.aspx?erg'
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: depth=0
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Rule 1 : 1 matches
Wed Feb 10 20:05:20 -  2648 - GetServerVariable: cannot find that variable
Wed Feb 10 20:05:20 -  2648 - EvalCondition: Cond %{HTTP_X_SOMEHEADER} ^HTTP_X_SOMEHEADER$ => TRUE
Wed Feb 10 20:05:20 -  2648 - EvalConditionList: rule 1, TRUE, Rule will apply
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Result (length 5): YaHuh
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: depth=1
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Rule 1 : -1 (No match)
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Rule 2 : 1 matches
Wed Feb 10 20:05:20 -  2648 - EvalCondition: Cond %{QUERY_STRING} ^$ => FALSE
Wed Feb 10 20:05:20 -  2648 - EvalConditionList: rule 2, FALSE, Rule does not apply
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Rule 3 : 1 matches
Wed Feb 10 20:05:20 -  2648 - EvalCondition: Cond %{QUERY_STRING} ^.+$ => TRUE
Wed Feb 10 20:05:20 -  2648 - EvalConditionList: rule 3, TRUE, Rule will apply
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Result (length 5): YaHuh
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: depth=2
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Rule 1 : -1 (No match)
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Rule 2 : 1 matches
Wed Feb 10 20:05:20 -  2648 - EvalCondition: Cond %{QUERY_STRING} ^$ => FALSE
Wed Feb 10 20:05:20 -  2648 - EvalConditionList: rule 2, FALSE, Rule does not apply
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: Rule 3 : -1 (No match)
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: returning 0
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: returning 0
Wed Feb 10 20:05:20 -  2648 - EvaluateRules: returning 0
Wed Feb 10 20:05:20 -  2648 - DoRewrites: No Rewrite

You can see the first line in red above indicates that the header is being set. The second line in red above indicates where IIRF applies the ruleset again, on the rewritten request, the request with the new header. And as you can see, in this case, the header is set, and the rule does not apply, in the 2nd iteration.  (The logfile line just above the 2nd red line above, shows that IIRF is iterating on the ruleset).

It's possible that I don't understand what you're doing; It's also possible that you're donig something that isn't quite right. Once I have a look at your ini file and log file, I think we'll know more

Feb 11, 2010 at 9:02 AM

Okay here is what I'm trying to do:

I use IIRF as a reverse proxy for my Rails backend, I want IIS to handle NTLM authentication only for a specific URL (/auth), others URL should simply be proxied (without authentication), except URL starting with /home or /prof which will be served by a legacy ASP code in IIS.

(for the authentication scheme described above, I have anonymous access set for the root (/), and NTLM authentication for /auth, this behaves okay on the IIS side)

Here is my ini file:

RewriteLog D:\InetPub\rewrite.log
RewriteLogLevel 5

RewriteCond %{HTTP_X_REMOTE_USER} ^HTTP_X_REMOTE_USER$
RewriteHeader X-Remote-User: ^$ %{REMOTE_USER}

ProxyPass ^/(?!(home|prof))(.*)$ http://localhost:8001/$2

Here is the log file for getting /yop (which should be proxied with the X-Remote-User set to blank (it's a url in the anonymous land))

Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: actual log file 'D:\InetPub\rewrite.log.1028.log'
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: ini file: 'c:\inetpub\wwwroot\Iirf.ini'
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: ini file timestamp: 2010/02/11 10:40:42 Europe de l'Ouest
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: cfg(0x00F84340)
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: LogLevel = 5
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: pass 2
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: c:\inetpub\wwwroot\Iirf.ini(19): RewriteCond   %{HTTP_X_REMOTE_USER}  ^HTTP_X_REMOTE_USER$ '(null)'
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: c:\inetpub\wwwroot\Iirf.ini(20): RewriteHeader (rule 1)  'X-Remote-User:'  '^$'  '%{REMOTE_USER}'   (null)
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: not a duplicate rule...
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: c:\inetpub\wwwroot\Iirf.ini(24): ProxyPass (rule 2)  '^/(?!(home|prof))(.*)$'  'http://localhost:8001/$2'   (null)
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: not a duplicate rule...
Thu Feb 11 10:42:05 -  3984 - ReadVdirConfig: Done reading, found 2 rules (0 errors, 0 warnings) on 25 lines
Thu Feb 11 10:42:05 - 3984 - DoRewrites
Thu Feb 11 10:42:05 - 3984 - GetServerVariable_AutoFree: getting 'url'
Thu Feb 11 10:42:05 - 3984 - GetServerVariable_AutoFree: 128 bytes
Thu Feb 11 10:42:05 - 3984 - GetServerVariable_AutoFree: result ''
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: getting 'url'
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 5 bytes ptr:0x000C8F58
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 'url' = '/yop'
Thu Feb 11 10:42:05 - 3984 - GetServerVariable_AutoFree: getting 'QUERY_STRING'
Thu Feb 11 10:42:05 - 3984 - GetServerVariable_AutoFree: 1 bytes
Thu Feb 11 10:42:05 - 3984 - GetServerVariable_AutoFree: result ''
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: getting 'method'
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 4 bytes ptr:0x000C9068
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 'method' = 'GET'
Thu Feb 11 10:42:05 - 3984 - DoRewrites: New Url, before decoding: '/yop'
Thu Feb 11 10:42:05 - 3984 - DoRewrites: Url (no decoding): '/yop'
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: depth=0
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: getting 'X-Remote-User:'
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 128 bytes ptr:0x000C90F0
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 'X-Remote-User:' = ''
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: Rule 1 : 1 matches
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: getting 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: cannot find that variable
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: 128 bytes
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: result 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: VariableName='HTTP_X_REMOTE_USER' Value='HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: in='%{HTTP_X_REMOTE_USER}' out='HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - EvalCondition: ts1 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - GenerateReplacementString: result 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - EvalCondition: checking 'HTTP_X_REMOTE_USER' against pattern '^HTTP_X_REMOTE_USER$'
Thu Feb 11 10:42:05 - 3984 - EvalCondition: match result: 1 (match)
Thu Feb 11 10:42:05 - 3984 - EvalCondition: Cond %{HTTP_X_REMOTE_USER} ^HTTP_X_REMOTE_USER$ => TRUE
Thu Feb 11 10:42:05 - 3984 - EvalConditionList: rule 1, TRUE, Rule will apply
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: getting 'REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: 1 bytes
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: result ''
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: VariableName='REMOTE_USER' Value=''
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: in='%{REMOTE_USER}' out=''
Thu Feb 11 10:42:05 - 3984 - GenerateReplacementString: result ''
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: Result (length 0):
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: Setting Header: 'X-Remote-User:' = ''
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: depth=1
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: getting 'X-Remote-User:'
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 128 bytes ptr:0x000C9178
Thu Feb 11 10:42:05 - 3984 - GetHeader_AutoFree: 'X-Remote-User:' = ''
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: Rule 1 : 1 matches
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: getting 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: cannot find that variable

Thu Feb 11 10:42:05 - 3984 - GetServerVariable: 128 bytes
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: result 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: VariableName='HTTP_X_REMOTE_USER' Value='HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: in='%{HTTP_X_REMOTE_USER}' out='HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - EvalCondition: ts1 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - GenerateReplacementString: result 'HTTP_X_REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - EvalCondition: checking 'HTTP_X_REMOTE_USER' against pattern '^HTTP_X_REMOTE_USER$'
Thu Feb 11 10:42:05 - 3984 - EvalCondition: match result: 1 (match)
Thu Feb 11 10:42:05 - 3984 - EvalCondition: Cond %{HTTP_X_REMOTE_USER} ^HTTP_X_REMOTE_USER$ => TRUE
Thu Feb 11 10:42:05 - 3984 - EvalConditionList: rule 1, TRUE, Rule will apply
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: getting 'REMOTE_USER'
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: 1 bytes
Thu Feb 11 10:42:05 - 3984 - GetServerVariable: result ''
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: VariableName='REMOTE_USER' Value=''
Thu Feb 11 10:42:05 - 3984 - ReplaceServerVariables: in='%{REMOTE_USER}' out=''
Thu Feb 11 10:42:05 - 3984 - GenerateReplacementString: result ''
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: Result (length 0):
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: Setting Header: 'X-Remote-User:' = ''
Thu Feb 11 10:42:05 - 3984 - EvaluateRules: depth=2

You see 2 complete iterations which are the same, the others 6 are also the same, then it reaches the limit of 8 iterations.

That's it.

Cheers!

Feb 11, 2010 at 9:25 AM

I did a little experiment,

I changed the RewriteHeader rule with this:

RewriteHeader X-Remote-User: ^$ Kool

And it works as expected. So another experiment, I changed to this variant (I want the header value to be an empty string):

RewriteHeader X-Remote-User: (^$) $1

And it fails like in my previous post. 

My thoughts: when the header gets set with a non empty string, in the next iteration the server variable reflect the previously set header.
But when setting the header to an empty string, then the server variables is either not set or is detected by IIRF as being blank and considered inexistant which is wrong. It is set with blank (difference between NULL and empty).

 

So long,

Pascal

Feb 11, 2010 at 9:06 PM
Edited Feb 11, 2010 at 9:29 PM

Investigating a bit further, the log line you outlined in red in your previous post

EvaluateRules: Rule 1 : -1 (No match)

simply shows that the RULE will not apply (the RewriteHeader) because it doesn't match the pattern ^$ , it has nothing to do with the previous condition (RewriteCond) which is never evaluated in the second round.

In fact, you may completely remove the RewriteCond (for X-SomeHeader) and the behaviour will be exactly the same for your scenario.

I keep thinking that IIS server variables are not updated regarding new headers set in the filter. This assumption is also based on the information or maybe the lack of information in the MSDN doc at http://msdn.microsoft.com/en-us/library/ms826755.aspx

There's no indication that setting a header will update the corresponding server variable. So I assume it doesn't. Any opinion on this? Or maybe you have a contact that could confirm this.

 

Cheers,

Pascal

 

Coordinator
Feb 12, 2010 at 3:36 AM
Edited Feb 12, 2010 at 3:37 AM

Hi Pascal,

I think you're right about my prior post.  The Rewritecond is never evaluated.  It's unnecessary.

Regarding the behavior you observed... Your RewriteHeader directive "works" when you use a non-blank string, but you say it "does not work" when you set the header to the empty string.  I don't understand this.  The rule as you have written it, is this;

RewriteHeader X-Remote-User: ^$ %{REMOTE_USER}

In English, I think that means, when the header X-Remote-User is not set (or empty), then set it to the value of the server variable named REMOTE_USER.

According to the log file you provided, that server variable is blank. Is that right? It contains the empty string. You then set the header to that value - the empty string. According to the documented behavior of IIRF, it iterates on the ruleset. Again, the RewriteHeader directive is evaluated. Once again, X-Remote-User is compared to the empty string. It is the empty string, so the directive fires again. It is set to the empty string. And so on, until the iteration limit is reached.

If the server variable with the name REMOTE_USER has the value of the empty string, then I believe the behavior you described and observed (and documented with your log) is correct.

Once again, maybe I'm not understanding something, in which case, please clarify.

Feb 12, 2010 at 8:33 AM
Cheeso wrote:

Regarding the behavior you observed... Your RewriteHeader directive "works" when you use a non-blank string, but you say it "does not work" when you set the header to the empty string.  I don't understand this.  The rule as you have written it, is this;

RewriteHeader X-Remote-User: ^$ %{REMOTE_USER}

In English, I think that means, when the header X-Remote-User is not set (or empty), then set it to the value of the server variable named REMOTE_USER.

Yes, that is what I want.

According to the log file you provided, that server variable is blank. Is that right? It contains the empty string. You then set the header to that value - the empty string. According to the documented behavior of IIRF, it iterates on the ruleset. Again, the RewriteHeader directive is evaluated. Once again, X-Remote-User is compared to the empty string. It is the empty string, so the directive fires again. It is set to the empty string. And so on, until the iteration limit is reached.

You are right, it is exactly because of this recursive behaviour that I want to protect this rule with a RewriteCond. The condition is here to skip the rule if the header has been set (either to blank or with a "real" string) by the previous iteration.

If the server variable with the name REMOTE_USER has the value of the empty string, then I believe the behavior you described and observed (and documented with your log) is correct.

Once again, maybe I'm not understanding something, in which case, please clarify.

The real behaviour I need is simply set a header to the value of a server variable, whatever the value of that server variable.

This could be "translated" to something like:

SetHeader X-Remote-User: %{REMOTE_USER}

BUT, the SetHeader rule doesn't exist in IIRF, so I try to emulate this behaviour (note that SetHeader should not be considered as a rule that triggers a new iteration).

I'm a bit lost with those combinations of RewriteHeader and RewriteCond... Did you catch what I need to do in the end?

Thanx,

 

Pascal

Coordinator
Feb 12, 2010 at 10:43 PM

Yes, I understand now.  You want to skip the rule if the header has been set (in other words, if the rule has fired once), either to a blank or non-blank value.

I think you may be able to get what you want by appending a tag to the header, and then removing it at the end of all iterations.

like this:

# After the following rule fires, the header will have a value prefixed with xxx-
RewriteHeader X-Remote-User: ^(?!xxx-) xxx-%{REMOTE_USER}

#... other rules here ...

# remove the tag inserted by the top rule. This could result in an "empty" header. 
RewriteHeader X-Remote-User: ^xxx-(.*) $1   [L]

 

Feb 12, 2010 at 10:51 PM

I don't think it will do the trick.

Because the real last rule I need is the ProxyPass not the RewriteHeader to strip out the magic. If I understand the [L] modifier correctly, it states that this is the last rule, so I may not have my ProxyPass after it.

I have to say that I got the sources and I'm trying some things out.

 

Coordinator
Feb 13, 2010 at 12:04 PM
Edited Feb 13, 2010 at 12:15 PM

ok, I understand.  One possible general solution to this is to extend the patterns supported by the RewriteHeader to include a special one that implies "not set".

Today a directive like the this :

 RewriteHeader  X-Remote-User: ^$  Foo 

...matches if X-Remote-User is blank, and if X-Remote-User is empty. What you want is to distinguish between those two conditions.

The filter uses the ISAPI callback function named GetHeader to get the value of the header to match against the pattern, and the return value from that thing DOES distinguish between those conditions. But the IIRF source code masks the difference.  It would be a small change to expose that distinction to the RewriteHeader rule.

Coordinator
Feb 13, 2010 at 3:01 PM

Ok, I looked into the code and it looks like I could support your scenario with a straightforward change. 

I propose to add a "well known string"  to be used with RewriteHeader as the pattern which matches only when the header is not set.  The well-known string would be *NOT_SET*  (or *UNSET* or something similar, we can choose it to be anything we like).  It would look like this in the ini file:

RewriteHeader  X-Remote-User:  *NOT_SET*   Foo 

The RewriteHeader rule would then be applied when the header was not previously set. 

Doing it this way would be a simple extension of the existing model, and I think would accomplish what you want.    It's also a low-impact change, just a few lines of code and one new paragraph in the documentation.

Actually I've made the change already and am testing it out.  I can make it permanent if you agree that it works for you.

 

Feb 13, 2010 at 7:47 PM

Hi Cheeso,

Yes, your modification seems to match the behaviour I look for. If it's ok for you, just commit the changes.

Note that in my own experiment, I took another approach to overcome this issue. I let you know, maybe you are interested in this change too:

I added a modifier flag that applies to any rule, I called this flag NoIteration (I didn't choose a letter, just picked X). The changes in the source are also very low, below ten lines of code in total.

This new flag permits that a rule applies, but instead of recursing it goes to the next rule (like if it didn't apply).

So for my wanted behaviour I had:

RewriteHeader X-Remote-User: ^$ %{REMOTE_USER} [X]

ProxyPass ^/(?!(home|prof))(.*)$ http://localhost:8001/$2

Maybe this new flag is against the philosophy of a rewriter, I can't judge, I have too little experience in this field.

 

BTW, I also dug in the code for the other issue for which I set up a WorkItem (26146), passing headers to the proxied server.

Because of my specific need, I turned to the way Helicon ISAPI_Rewrite3 works (described in the first post of this thread), adding a [A] flag to ProxyPass to set authentication headers. I get this way because it was simplier. For a general header forwarding mechanism for ProxyPass, the amount of work is a bit higher. The filter must track all the headers that are rewritten, maybe also get all the extra headers comming from the initial request. Like I saw, for verbs other thant POST and PUT, it is done in the filter, but for these two verbs, you created an Extension. Well, like I said, this smells like a certain amount of work.

Tell me if you are interested in the diffs I made to add this [A] flag.

Once again, thanx for your support.

Pascal

Coordinator
Feb 14, 2010 at 1:14 PM

Hello Pascal,

I'm not sure the changes I've described to you, will work. The problem is that setting a request header to the empty string, deletes the header.  This is according to Microsoft's documentation for SetHeader, the function IIRF uses to set the request header when the RewriteHeader directive fires. 

Therefore in your case, setting any header to the value of %{REMOTE_USER} , when REMOTE_USER is blank, will cause the named header to be deleted.

I tried special-casing the case where the value is blank, and replacing the blank with a single space character.  In this case, though, the GetHeader function still indicates that the header is unset. 

I also read in the RFC 2616, and the meaning of "unset" and "blank" for a header are the same, for the protocol.  There is no distinction between a header that is not present and a header that is blank.  So I think the very idea of discriminating between those cases is hopeless.

I am interested in your modifications for [X] flag and the [A] flag.  I have to think further about the [X] flag, and whether it makes sense.  I will also have to read more about the authentication headers, and the Helicon approach.  If you could post the mods to the relevant workitems, that would be good.  Workitem 26146 for the [A] flag, and I don't know if there is a workitem yet for the X flag.

 

 

Feb 14, 2010 at 8:25 PM

Hi,

I created a new work item 26212 for the NoIteration flag which I renamed to [NI] instead of [X]. Note that I first though to create a new rule (SetHeader) but duplicating code that is merely the same (as RewriteHeader) is not the way I like. Moreover this new rule also had to not iterate and do its work in place. So I got the NoIteration flag way which has a very low code impact (as you can see in the diff).

For the [A] flag, I'll post the diff tomorow. Note that I could test it for the GET verb, it's ok. I also did the modifications for the POST and PUT verbs, modifying the Extension part. But I couldn't test it, I also have some doubt. Maybe because the Extension is serving a new request (/proxy.iirf) the server variables containing the authentication may be either wrong or not set. So please, take this part of the mods with caution.

Regards,

Pascal

Coordinator
Feb 14, 2010 at 10:17 PM

I like the idea; I'll look through your diffs.

Coordinator
Mar 24, 2010 at 4:36 PM

Hello Pascal,

Following up on this old issue.  I've posted a new update of IIRF - v2.1.0.15 - which handles HTTP headers generally.  I think it may solve the problem you reported, with the auth headers.  The workitem to track this was workitem 26146

The updated filter is now available on http://iirf.codeplex.com/releases/view/36814 .

If you get a chance to test this update, I would appreciate it if you could share your results with me.

- Cheeso 

 

Mar 24, 2010 at 5:27 PM

Hi,

I'll check this new version in the following weeks (Easter holidays).

 

Thanx for the good work.

Pascal