Redirect on empty User Agent

Oct 13, 2011 at 12:58 PM

I'm trying to eliminate the bot traffic that hits my site with a blank http_user_agent but the rules in the "Blacklist by User Agent" example don't seem to work for me. 

 

RewriteCond %{HTTP_USER_AGENT} ^$ 
RewriteRule ^/(.*)$ /$1 [F]

 

When I test in FF 3.6 using the "User Agent Switcher" (I created a custom user agent that is blank), no redirection happens I'm able to get to whatever page I want on the site. To confirm the agent is indeed blank and that Firebug wasn't lying, this tool came in handy. http://rocket-city.us/Dunavant/user-agent.html

Any ideas what I might be doing wrong? Thanks.

Using:

IIRF 2.x
IIS 7
WinServ 2008 

Coordinator
Oct 13, 2011 at 4:48 PM

well I would loook in the IIRF log, to see what IIRF is doing about those requests.

If you turn up the log level to 4, temporarily and for the purposes of diagnosis only, you'll clearly see the evaluation of the RewriteCond and why it is failing.  

At that point you can adjust the rule to fit the circumstances, and then turn the loglevel back to its normal level.

 

Oct 13, 2011 at 5:15 PM

Cheeso-thanks for replying. 

Here's what I see in the log

Thu Oct 13 13:02:18 -  9484 - ReplaceServerVariables: in='%{HTTP_USER_AGENT}' out='HTTP_USER_AGENT'
Thu Oct 13 13:02:18 -  9484 - GenerateReplacementString: alloc 17 bytes
Thu Oct 13 13:02:18 -  9484 - ApplyUrlEncoding: in 'HTTP_USER_AGENT'
Thu Oct 13 13:02:18 -  9484 - ApplyUrlEncoding: out 'HTTP_USER_AGENT'
Thu Oct 13 13:02:18 -  9484 - GenerateReplacementString: result 'HTTP_USER_AGENT'
Thu Oct 13 13:02:18 -  9484 - EvalCondition: Cond t(%{HTTP_USER_AGENT}) op(~) p(^$) => FALSE
Thu Oct 13 13:02:18 -  9484 - EvalConditionList: rule 1, FALSE, Rule does not apply
Thu Oct 13 13:02:18 -  9484 - EvaluateRules: Rule 2: pattern: ^/(.*)  subject: /
Thu Oct 13 13:02:18 -  9484 - EvaluateRules: Rule 2: 2 matches
Thu Oct 13 13:02:18 -  9484 - EvaluateRules: Rule 2: evaluating condition

This line should catch the blank user agent, right? ->

EvalCondition: Cond t(%{HTTP_USER_AGENT}) op(~) p(^$) => FALSE

Coordinator
Oct 13, 2011 at 6:43 PM
Edited Oct 13, 2011 at 6:44 PM

what version of IIRF are you running?  There was a change in the way it handled empty headers (as HTTP_USER_AGENT is in this case for you).  Only the more recent builds of IIRF will work with the rule you have.

you can either:

  • update to the latest IIRF, and keep your rule as it is
  • modify the rule and keep your older version of IIRF as you have it.

if you choose the latter option, the modified rule should look like this:

RewriteCond %{HTTP_USER_AGENT} ^HTTP_USER_AGENT$ 
RewriteRule ^/(.*)$ /$1 [F]