Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

eika from Ru-Board
Hi folks!

I come across with issue I can't fix. I have Apache/2.4.43 OpenSSL/1.1.1g
with mod_autoindex (showing directory listings instead of index.html) and
with mod_deflate.

I was able to get content gzipped, but only if they are files with
extensions (e.g. .css, .html, etc.) But when I want to get gzipped page
coming from mod_autoindex, I can't get it. E.g. https://domain.dom,
https://domain.dom/somedir/, https://domain.dom/test/, etc. These URLs comes
without Content-encoding: gzip header.

I think that I am not far from the reason why, because I found that
directory indexes sent by chunks (transfer-encoding: chunked). But files
with extensions came with these headers:

Accept-Ranges: bytes
Content-Encoding: gzip
Content-Length: value

But I have no ideas what to do next. There is no way to force Content-Length
header. And I don't to force HTTP 1.0 protocol instead of HTTP 1.1.

More details: https://stackoverflow.com/questions/62942077/ (I hope that
external links are OK here).

Please advise.

Thanks!

--
Regards,
Art





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

Rainer Canavan-2
On Fri, Jul 31, 2020 at 9:01 PM eika from Ru-Board
<[hidden email]> wrote:

>
> Hi folks!
>
> I come across with issue I can't fix. I have Apache/2.4.43 OpenSSL/1.1.1g
> with mod_autoindex (showing directory listings instead of index.html) and
> with mod_deflate.
>
> I was able to get content gzipped, but only if they are files with
> extensions (e.g. .css, .html, etc.) But when I want to get gzipped page
> coming from mod_autoindex, I can't get it. E.g. https://domain.dom,
> https://domain.dom/somedir/, https://domain.dom/test/, etc. These URLs comes
> without Content-encoding: gzip header.
>
> I think that I am not far from the reason why, because I found that
> directory indexes sent by chunks (transfer-encoding: chunked). But files
> with extensions came with these headers:

I suspect that you have simply configured mod_gzip to be only active
for URLs ending in .html etc, and the transfer-encoding just happens
to correlate with that because it's not a local "file" but generated
dynamically. You should check the outgoing content-type instead of the
URL. We've been using the following for quite some time:

   FilterProvider buffer           BUFFER   "%{CONTENT_TYPE} =~ m|^text/|"
   FilterProvider buffer           BUFFER   "%{CONTENT_TYPE} =~ m|^model/|"
[... more content types ]
   FilterProvider gzip_compression DEFLATE  "%{CONTENT_TYPE} =~ m|^text/|"
   FilterProvider gzip_compression DEFLATE  "%{CONTENT_TYPE} =~ m|^model/|"
[...]
   FilterChain buffer gzip_compression

rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

eika from Ru-Board
Hi Rainer!

Thanks for your response.

Could you please help a bit more?

Here is my Apache Deflate configured:

        DeflateFilterNote Input instream
        DeflateFilterNote Output outstream
        DeflateFilterNote Ratio ratio
        AddOutputFilterByType DEFLATE text/plain
        AddOutputFilterByType DEFLATE text/html
        AddOutputFilterByType DEFLATE text/css
        AddOutputFilterByType DEFLATE text/javascript
        AddOutputFilterByType DEFLATE text/xml
        AddOutputFilterByType DEFLATE text/php
        AddOutputFilterByType DEFLATE image/x-icon
        AddOutputFilterByType DEFLATE application/javascript
        AddOutputFilterByType DEFLATE application/x-javascript
        AddOutputFilterByType DEFLATE application/xml

And these directives works like charm for html, css, js, files, etc.

I tried to add:

        FilterDeclare COMPRESS
        FilterProvider COMPRESS DEFLATE "%{Content_Type} = 'text/html'"
        FilterProvider COMPRESS DEFLATE "%{Content_Type} = 'text\/html.*$'"
        FilterProvider COMPRESS DEFLATE "%{Content_Type} = 'text/plain'"
        FilterProvider COMPRESS DEFLATE "%{Content_Type} = 'text/php'"
        FilterChain COMPRESS gzip_compression
        FilterProtocol COMPRESS DEFLATE change=yes;byteranges=no

But still getting chunked responses. I am sending requests to URI '/' and '/TEST/'. And Content-Type for these addresses are "text/html;charset=UTF-8".

Also, I have no ideas why Content-Type is 'text/html;charset=UTF-8', but not just 'text/html'.

Here is header example from LRT plugin for Chrome:

------------------------------------------------------
[
  {
    "url": "https://domain.dom/",
    "redirect": false,
    "redirectType": "",
    "statusCode": 200,
    "statusLine": "HTTP/1.1 200 OK",
    "headers": [
      {
        "name": "Date",
        "value": "Mon, 03 Aug 2020 19:18:19 GMT"
      },
      {
        "name": "Server",
        "value": "Apache/2.4"
      },
      {
        "name": "Accept-Ranges",
        "value": "none"
      },
      {
        "name": "Vary",
        "value": "Accept-Encoding,User-Agent"
      },
      {
        "name": "Cache-Control",
        "value": "max-age=86400, private, must-revalidate"
      },
      {
        "name": "X-Permitted-Cross-Domain-Policies",
        "value": "none"
      },
      {
        "name": "X-Content-Type-Options",
        "value": "nosniff"
      },
      {
        "name": "X-XSS-Protection",
        "value": "1; mode=block"
      },
      {
        "name": "X-Frame-Options",
        "value": "deny"
      },
      {
        "name": "Content-Language",
        "value": "en"
      },
      {
        "name": "Keep-Alive",
        "value": "timeout=5, max=95"
      },
      {
        "name": "Connection",
        "value": "Keep-Alive"
      },
      {
        "name": "Transfer-Encoding",
        "value": "chunked"
      },
      {
        "name": "Content-Type",
        "value": "text/html;charset=UTF-8"
      }
    ],
  }
]
------------------------------------------------------

Please advise what could be wrong.

Thanks in advance!

-----Original Message-----
From: Rainer Canavan [mailto:[hidden email]]
Sent: 03 August 2020 12:41 PM
To: [hidden email]
Subject: Re: [users@httpd] Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

On Fri, Jul 31, 2020 at 9:01 PM eika from Ru-Board <[hidden email]> wrote:

>
> Hi folks!
>
> I come across with issue I can't fix. I have Apache/2.4.43
> OpenSSL/1.1.1g with mod_autoindex (showing directory listings instead
> of index.html) and with mod_deflate.
>
> I was able to get content gzipped, but only if they are files with
> extensions (e.g. .css, .html, etc.) But when I want to get gzipped
> page coming from mod_autoindex, I can't get it. E.g.
> https://domain.dom, https://domain.dom/somedir/,
> https://domain.dom/test/, etc. These URLs comes without Content-encoding: gzip header.
>
> I think that I am not far from the reason why, because I found that
> directory indexes sent by chunks (transfer-encoding: chunked). But
> files with extensions came with these headers:

I suspect that you have simply configured mod_gzip to be only active for URLs ending in .html etc, and the transfer-encoding just happens to correlate with that because it's not a local "file" but generated dynamically. You should check the outgoing content-type instead of the URL. We've been using the following for quite some time:

   FilterProvider buffer           BUFFER   "%{CONTENT_TYPE} =~ m|^text/|"
   FilterProvider buffer           BUFFER   "%{CONTENT_TYPE} =~ m|^model/|"
[... more content types ]
   FilterProvider gzip_compression DEFLATE  "%{CONTENT_TYPE} =~ m|^text/|"
   FilterProvider gzip_compression DEFLATE  "%{CONTENT_TYPE} =~ m|^model/|"
[...]
   FilterChain buffer gzip_compression

rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

Rainer Canavan-2
[...]
>         FilterProvider  COMPRESS        DEFLATE "%{Content_Type} = 'text\/html.*$'"

I don't think you can use regular expressions with just '=', you'll
have to use '=~'

search 'regex' in the documentation at
https://httpd.apache.org/docs/2.4/expr.html for the exact syntax
required.


rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

eika from Ru-Board
Hi Rainer!

Thanks you again.

This is what I tried:

FilterDeclare COMPRESS
FilterProvider COMPRESS DEFLATE "%{CONTENT_TYPE} =~ m#text/html#"
FilterProvider COMPRESS DEFLATE "%{CONTENT_TYPE} =~ m#text\/html.*$#"
FilterProvider COMPRESS DEFLATE "%{CONTENT_TYPE} =~ m#text\/html\.*$#"
FilterChain COMPRESS gzip_compression
FilterProtocol COMPRESS DEFLATE change=yes;byteranges=no

And still the same. I.e. chunked content :(

Is my syntax incorrect and doesn't cover [Content-Type is 'text/html;charset=UTF-8'] header?

-----Original Message-----
From: Rainer Canavan [mailto:[hidden email]]
Sent: 04 August 2020 13:39 PM
To: [hidden email]
Subject: Re: [users@httpd] Apache mod_autoindex and mod_deflate (gzip). Can't get pages compressed

[...]
>         FilterProvider  COMPRESS        DEFLATE "%{Content_Type} = 'text\/html.*$'"

I don't think you can use regular expressions with just '=', you'll have to use '=~'

search 'regex' in the documentation at
https://httpd.apache.org/docs/2.4/expr.html for the exact syntax required.


rainer

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]