Hello again,

I would like to report that it was needed to copy /etc/nsswitch.conf,
/lib/libresolv.so* and /lib/libns*.so* to the RootJail because of the last
directive enabled. Pound depended on the DNS lookups in order to
RewriteLocation successfully.

So problem does not lie between lines 1060 and 1070 at all, these are all
right.

Please regard this email as the ticket closing one. I apologise for the
false alarm.

Warm regards,
Martins
[...]

in the end ... can i do that ?

i can use rinetd for port redirect but this way i lose the IP source ....
and my web application use this IP source for some sort of filters and for
some sort of authentification

regards,
mihai


On 6/28/07, cosmih <cosmih(at)gmail.com> wrote:[...]

On Thu, 2007-06-28 at 12:46 -0700, Ed Sawicki wrote:[...]

Thanks for your help Ed. 2.4b had indeed introduced a problem with
chunked transfer-encoding. This should be fixed in the just-released
2.4c - give it a try and let me know.[...]

On Wed, 2007-06-27 at 16:51 -0400, Albert wrote:[...]

I guess that depends on how you can identify the crawlers. I expect you
already looked into the simple answers (firewall for source addresses,
HeadDeny for header-based identification, traffic shaping for many quick
requests from the same source address). Is there anything I missed?[...]

Robert Segall wrote:[...][...][...]

It works!

I was using version 2.3.2, so it's been broken for a while
I guess.

I'm an author and I cover pound in my Guide to Apache book:

http://apachebook.alcpress.com

which is used in college courses. I'm expanding the Pound
section in the Second Edition so it's nice that the chunked
transfer encoding problem is fixed.

Thank you,
Ed Sawicki

[...]
Yes we have, and there is no simple answer.  We're planning to limit 
number of page views any C-class IP address can have for a period of 
time.  But the problem is with scrappers using ISPs.  The other issues 
is real crawlers, we don't want to limit number of pages a crawler sees, 
so in essence we need to create an "authorized" list of IP addresses, 
without any limitations.

Some scrappers tone down their bots to do page requests every few 
seconds (mimicking human like speed).  Other scrappers seems to employ 
cheap human labor, anywhere from India to China to Africa, so you have 
somebody who looks like browsing your data, but really scrapping.

One idea I had would be to allow pound to call a plugin.  But this would 
make pound more complicated, and slower, not to mention the plugin will 
need to be implemented. 

We'll be trying to tackle this problem in-house in the new few weeks, 
and I'll share our experiences.

Albert

Would you mind sharing your ISAPI filter or give a pointer in the right 
direction for writing one?

Thanks!

Albert wrote:[...][...][...]

Leonard Bethea wrote:[...]

Since pound speaks HTTP only to the backend, all traffic appears to be 
HTTP traffic to Apache.  You'll want to use a header as your indicator. 
  Something like this works:

===
RewriteEngine On
RewriteCond %{HTTP:X-Forwarded-Proto} =https 
 

RewriteRule .* - [L]

# otherwise redirect to SSL
RewriteRule .* https://www.example.com%{REQUEST_URI} [R,L]
===

Be sure you're adding the 'X-Forwarded-Proto: https' header in pound.

Regards,[...]

Thanks Dave,

I was able to get the redirect to work. I tweaked your code a little and
added the header. If it's a SSL cert then everything works now. If the
header is not https then I redirect to the secure link; however, the
redirect does not pass the form information. I was wondering if this was
because of mod_rewrite or pound?

-----Original Message-----
From: Dave Steinberg [mailto:dave(at)redterror.net] 
Sent: Saturday, July 28, 2007 10:13 AM
To: pound(at)apsis.ch
Subject: Re: [Pound Mailing List] mod_rewrite

Leonard Bethea wrote:[...]
It[...]
RewriteEngine[...]
with[...]

Since pound speaks HTTP only to the backend, all traffic appears to be 
HTTP traffic to Apache.  You'll want to use a header as your indicator. 
  Something like this works:

===
RewriteEngine On
RewriteCond %{HTTP:X-Forwarded-Proto} =https 
 

RewriteRule .* - [L]

# otherwise redirect to SSL
RewriteRule .* https://www.example.com%{REQUEST_URI} [R,L]
===

Be sure you're adding the 'X-Forwarded-Proto: https' header in pound.

Regards,[...]

Leonard Bethea wrote:[...]

Neither - if your application is submitting a form to a HTTP url and you 
expect to receive variables, the redirector is going to need to turn the 
POST variables into the query string of the redirect URL (thereby 
turning them into "GET variables".  In this case you're looking at a 
custom redirector rather than using mod_rewrite for it - at least if I'm 
understanding you correctly.

Naturally I'd recommend avoiding this if possible, as any large forms 
won't be too happy.

Regards,[...]