Location/LocationMatch/If ordering - which one wins?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Location/LocationMatch/If ordering - which one wins?

Graham Leggett
Hi all,

I am having an odd case where my reading of the docs and httpd itself aren’t matching and I’m stumped as to why.

I have a config like this (unrelated directives chopped for clarity):

    SSLVerifyClient optional
    <Location /jira>
      <If "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS’">
        # cert + group member? you can come in
        require ldap-group xxx
      </If>
      <Else>
        # no cert, go away
        require all denied
      </Else>
    </Location>
    <LocationMatch ^\/jira\/servicedesk\/customer\/portal\/3\/(.+)\/unsubscribe(.*)>
      # cert or no cert, let them in
      require all granted
    </LocationMatch>

When I try and use the following URL that should be matched by LocationMatch, the “require all denied” wins, which has me stumped.

https://[server]/jira/servicedesk/customer/portal/3/ENQUIRY-5/unsubscribe

What am I missing?

Does the use of If affect this in some way?

Regards,
Graham



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Eric Covener
On Thu, Jul 11, 2019 at 6:55 PM Graham Leggett <[hidden email]> wrote:

>
> Hi all,
>
> I am having an odd case where my reading of the docs and httpd itself aren’t matching and I’m stumped as to why.
>
> I have a config like this (unrelated directives chopped for clarity):
>
>     SSLVerifyClient optional
>     <Location /jira>
>       <If "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS’">
>         # cert + group member? you can come in
>         require ldap-group xxx
>       </If>
>       <Else>
>         # no cert, go away
>         require all denied
>       </Else>
>     </Location>
>     <LocationMatch ^\/jira\/servicedesk\/customer\/portal\/3\/(.+)\/unsubscribe(.*)>
>       # cert or no cert, let them in
>       require all granted
>     </LocationMatch>
>
> When I try and use the following URL that should be matched by LocationMatch, the “require all denied” wins, which has me stumped.
>
> https://[server]/jira/servicedesk/customer/portal/3/ENQUIRY-5/unsubscribe
>
> What am I missing?
>
> Does the use of If affect this in some way?

Yes, definitely.

- the location* are processed in config order.
- if/else directives are added to the dirconf
- `require all granted `is merged into empty authz_core (or whatever)
module config
- ifwalk walks <if> sections
- either of the if/else `require` directives have to be merged into
the current authz_core dirconf

Maybe some always-true <if> would get the `require all granted` to be
merged in last when the locationmatch has it active.

--
Eric Covener
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Graham Leggett
On 12 Jul 2019, at 01:14, Eric Covener <[hidden email]> wrote:

Yes, definitely.

- the location* are processed in config order.

My understanding is that we walk first to last, and the last matching configuration wins, and in theory that means the LocationMatch should win.

The trouble is Location is winning, and I can’t see why.

- if/else directives are added to the dirconf
- `require all granted `is merged into empty authz_core (or whatever)
module config
- ifwalk walks <if> sections
- either of the if/else `require` directives have to be merged into
the current authz_core dirconf

Maybe some always-true <if> would get the `require all granted` to be
merged in last when the locationmatch has it active.

So do the If sections in Location come back to life and re-override the LocationMatch?

My head is hurting.

Regards,
Graham


smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Eric Covener
> My understanding is that we walk first to last, and the last matching configuration wins, and in theory that means the LocationMatch should win.
The last match doesn't win, it's merged on top of whatever the current
per-dir*per-module config is.

Just like a normal mod, whoever evaluates <if> (core) can decide to
replace/merge/append then act on that config later.
The merge for If sections encountered is to accumulate them in the
per-dir config. After all the other sections have been parsed, the
accumulated if sections are evaluated shortly after the traditional
location/dirwalk stuff.

sections.html does show <if> as evaluated last:
http://httpd.apache.org/docs/current/sections.html#merging
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Graham Leggett
On 12 Jul 2019, at 01:32, Eric Covener <[hidden email]> wrote:

>> My understanding is that we walk first to last, and the last matching configuration wins, and in theory that means the LocationMatch should win.
> The last match doesn't win, it's merged on top of whatever the current
> per-dir*per-module config is.

That’s what I mean by “win” - it takes precedence and is merged on top of earlier config.

> Just like a normal mod, whoever evaluates <if> (core) can decide to
> replace/merge/append then act on that config later.
> The merge for If sections encountered is to accumulate them in the
> per-dir config. After all the other sections have been parsed, the
> accumulated if sections are evaluated shortly after the traditional
> location/dirwalk stuff.
>
> sections.html does show <if> as evaluated last:
> http://httpd.apache.org/docs/current/sections.html#merging

I know that Ifs are evaluated last, but what concrete effect does this have?

Or to put it another way, what changes do I need to make to the earlier config to “punch a hole” in the URL space such that “…/unsubscribe” is always granted access?

I am having the exact same problem with Directory and DirectoryMatch. When there are Ifs in a Directory, the Directory overrides the DirectoryMatch, even though the DirectoryMatch is more specific and should “win” (win meaning be merged on top of all that has gone before it).

Regards,
Graham



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Graham Leggett
On 12 Jul 2019, at 01:46, Graham Leggett <[hidden email]> wrote:

> I am having the exact same problem with Directory and DirectoryMatch. When there are Ifs in a Directory, the Directory overrides the DirectoryMatch, even though the DirectoryMatch is more specific and should “win” (win meaning be merged on top of all that has gone before it).

Here is a simpler example:

    <Directory /home/${HOST}/storage>
      Dav on
      SSLVerifyClient optional
      <If "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS'">
        require valid-user
      </If>
      <Else>
        require valid-user
      </Else>
    </Directory>
    <Directory /home/${HOST}/storage/home>
      require all denied  <—— has no effect
    </Directory>

Why, when a valid user is logged in (via cert or not cert), does httpd grant access to the file /home/${HOST}/storage/home/foo?

Most specifically, why does “require all denied” have no effect when a file matches that directory section?

Regards,
Graham



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Eric Covener
On Thu, Jul 11, 2019 at 8:21 PM Graham Leggett <[hidden email]> wrote:

>
> On 12 Jul 2019, at 01:46, Graham Leggett <[hidden email]> wrote:
>
> > I am having the exact same problem with Directory and DirectoryMatch. When there are Ifs in a Directory, the Directory overrides the DirectoryMatch, even though the DirectoryMatch is more specific and should “win” (win meaning be merged on top of all that has gone before it).
>
> Here is a simpler example:
>
>     <Directory /home/${HOST}/storage>
>       Dav on
>       SSLVerifyClient optional
>       <If "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS'">
>         require valid-user
>       </If>
>       <Else>
>         require valid-user
>       </Else>
>     </Directory>
>     <Directory /home/${HOST}/storage/home>
>       require all denied  <—— has no effect
>     </Directory>
>
> Why, when a valid user is logged in (via cert or not cert), does httpd grant access to the file /home/${HOST}/storage/home/foo?

Because the last thing merged into authz_cores r->per_dir_config is a
dirconf w/ `require valid-user` directives from the if/else.

What else could it mean for <If> to be merged last?

> Most specifically, why does “require all denied” have no effect when a file matches that directory section?

Because the core added an <If> section and is going to evaluate it
later. Even if you used some core directives inside the 2nd
<Directory> section to cause a merge to happen, there is no directive
that removes/resets existing <If> sections from cores per-dir-config.
They are accumulated in the early configuration section walking then
evaluated after.
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Ruediger Pluem
In reply to this post by Graham Leggett


On 07/12/2019 12:55 AM, Graham Leggett wrote:

> Hi all,
>
> I am having an odd case where my reading of the docs and httpd itself aren’t matching and I’m stumped as to why.
>
> I have a config like this (unrelated directives chopped for clarity):
>
>     SSLVerifyClient optional
>     <Location /jira>
>       <If "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS’">
>         # cert + group member? you can come in
>         require ldap-group xxx
>       </If>
>       <Else>
>         # no cert, go away
>         require all denied
>       </Else>
>     </Location>
>     <LocationMatch ^\/jira\/servicedesk\/customer\/portal\/3\/(.+)\/unsubscribe(.*)>
>       # cert or no cert, let them in
>       require all granted
>     </LocationMatch>
>

Given Erics comments, what about:

     SSLVerifyClient optional
     <Location /jira>
       <If "%{REQUEST_URI} =~'^\/jira\/servicedesk\/customer\/portal\/3\/(.+)\/unsubscribe(.*)'>
         require all granted
       </If
       <ElseIf "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS’">
         # cert + group member? you can come in
         require ldap-group xxx
       </ElseIf>
       <Else>
         # no cert, go away
         require all denied
       </Else>
     </Location>


Regards

Rüdiger
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Graham Leggett
In reply to this post by Eric Covener
On 12 Jul 2019, at 02:51, Eric Covener <[hidden email]> wrote:

Because the last thing merged into authz_cores r->per_dir_config is a
dirconf w/ `require valid-user` directives from the if/else.

What else could it mean for <If> to be merged last?

For a start it means that this part of our manual is invalidated the moment you add an If statement:


Most specifically, why does “require all denied” have no effect when a file matches that directory section?

Because the core added an <If> section and is going to evaluate it
later. Even if you used some core directives inside the 2nd
<Directory> section to cause a merge to happen, there is no directive
that removes/resets existing <If> sections from cores per-dir-config.
They are accumulated in the early configuration section walking then
evaluated after.

I am trying to get my head around how we can make If work like the docs above. Is If broken, or is require broken?

Regards,
Graham


smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Eric Covener
> For a start it means that this part of our manual is invalidated the moment you add an If statement:
> http://httpd.apache.org/docs/current/sections.html

I agree it's not intuitive (either way), but which part of the above
invalidated?
It already says <if> is merged last.

When it's not nested, the <if> is part of the configuration and wins.
When it's nested, it's conditionally part of the configuration, and if
it's part of the configuration it wins.

> I am trying to get my head around how we can make If work like the docs above. Is If broken, or is require broken?

I don't agree that either is broken.  Neither <if> nesting semantic is
very intuitive but what we have seems pretty consistent to me -- it
acts the similar to how it would act if the enclosing scope is part of
your expression.

I don't really like the idea of an directive to reset the list of <if>
sections that have been accumulated by walking the config so far.

The most I can personally see here is providing more examples in
sections.html and the directive info for <if> to clarify that nesting
is complicated.
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Graham Leggett
On 12 Jul 2019, at 13:05, Eric Covener <[hidden email]> wrote:

>> For a start it means that this part of our manual is invalidated the moment you add an If statement:
>> http://httpd.apache.org/docs/current/sections.html
>
> I agree it's not intuitive (either way), but which part of the above
> invalidated?
> It already says <if> is merged last.
>
> When it's not nested, the <if> is part of the configuration and wins.
> When it's nested, it's conditionally part of the configuration, and if
> it's part of the configuration it wins.
Saying “if is merged last” is meaningless as a statement. Sure, if you’re describing the internals of the config walk process, that statement makes sense, but to an end user who is trying to write an httpd config to achieve a specific objective, the statement means nothing at all.

Most specifically, in the document above, "Some useful examples” shows this:

"the directives in this example will be applied in the order A > B > C > D > E”

<Location "/">
    E
</Location>
<Files "f.html">
    D
</Files>
<VirtualHost *>    
<Directory "/a/b">
        B
</Directory>
</VirtualHost>
<DirectoryMatch "^.*b$">
    C
</DirectoryMatch>
<Directory "/a/b">
    A
</Directory>

Add an If to the above what changes happen to A > B > C > D > E, and do these make any coherent sense to an end user?

>> I am trying to get my head around how we can make If work like the docs above. Is If broken, or is require broken?
>
> I don't agree that either is broken.  Neither <if> nesting semantic is
> very intuitive but what we have seems pretty consistent to me -- it
> acts the similar to how it would act if the enclosing scope is part of
> your expression.
>
> I don't really like the idea of an directive to reset the list of <if>
> sections that have been accumulated by walking the config so far.
>
> The most I can personally see here is providing more examples in
> sections.html and the directive info for <if> to clarify that nesting
> is complicated.
If we cannot sensibly and accurately describe the behaviour in our docs that an ordinary end user can understand in a reasonable length of time, then it’s definitely broken and we need to fix it.

Regards,
Graham



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Eric Covener
> If we cannot sensibly and accurately describe the behaviour in our docs that an ordinary end user can understand in a reasonable length of time, then it’s definitely broken and we need to fix it.

There's literally an ordered list in the manual and <If> is last.
Please don't change how it works.
Reply | Threaded
Open this post in threaded view
|

Re: Location/LocationMatch/If ordering - which one wins?

Graham Leggett
In reply to this post by Ruediger Pluem
On 12 Jul 2019, at 09:10, Ruediger Pluem <[hidden email]> wrote:

Given Erics comments, what about:

    SSLVerifyClient optional
    <Location /jira>
      <If "%{REQUEST_URI} =~'^\/jira\/servicedesk\/customer\/portal\/3\/(.+)\/unsubscribe(.*)'>
        require all granted
      </If
      <ElseIf "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS’">
        # cert + group member? you can come in
        require ldap-group xxx
      </ElseIf>
      <Else>
        # no cert, go away
        require all denied
      </Else>
    </Location>

The expression syntax for regexes isn’t that clear in the docs (the examples are too trivial to be useful), what eventually worked was this:

      <If "%{REQUEST_URI} =~ m#^\/jira\/servicedesk\/customer\/portal\/3\/(.+)\/unsubscribe(.*)#>
        require all granted
      </If>

The next step is to attack the Directory/DirectoryMatch problem, and this one I’m also struggling to make work.

    Alias /storage /home/${HOST}/storage
    <Directory /home/${HOST}/storage>
      Dav on
      Options +Indexes
      SSLVerifyClient optional

      # first, handle cert auth or basic auth...
      <If "%{SSL_CLIENT_VERIFY} == 'SUCCESS' || %{SSL_CLIENT_VERIFY} == 'GENEROUS'">
        SSLUserName SSL_CLIENT_CERT_RFC4523_CEA
        AuthLDAPBindDN xxx
        AuthLDAPURL xxx
        AuthLDAPRemoteUserAttribute inetSubscriberAccountId
      </If>
      <Else>
        AuthBasicProvider ldap
        AuthType basic
        AuthName "Storage"
        AuthLDAPBindDN xxx
        AuthLDAPURL xxx
      </Else>

      # ...then apply authz
      <If "%{REQUEST_FILENAME} =~ m#^/home/${HOST}/storage/atlassian/jira-home#>
        require ldap-group xxx
      </If>
      <ElseIf "%{REQUEST_FILENAME} =~ m#^/home/${HOST}/storage/home/(?<USER>[^/]+)#>
        <RequireAll>
          require valid-user
          require expr %{env:MATCH_USER} == %{REMOTE_USER}
        </RequireAll>
      </ElseIf>
      <Else>
        require valid-user
      </Else>
    </Directory>

Getting rid of DirectoryMatch above and consolidating everything into one directory and a series of If/Else sections, we have the config above.

The first If/Else declares that users must be grabbed from the cert if present, or basic auth if absent.

The second set of If/ElseIf/Else says that if you’re part of the ldap-group you can see the jira stuff (this works), if your username you logged in as matches the directory path you can come in (this doesn't work), and the third part says otherwise if you are a valid-user you can see everything else (this works).

Most specifically, when you try and access /home/${HOST}/storage/home/, which should not match the regex, for some reason it does match the regex and the config applies (and fails). Then, if you try access /[hidden email]/ which definitely matches the regex, the request fails as follows:

[Fri Jul 12 15:17:27.304567 2019] [authz_core:debug] [pid 172949:tid 140637026383616] mod_authz_core.c(820): [client x] AH01626: authorization result of Require valid-user : granted
[Fri Jul 12 15:17:27.304573 2019] [authz_core:debug] [pid 172949:tid 140637026383616] mod_authz_core.c(820): [client x] AH01626: authorization result of Require expr %{env:MATCH_USER} == %{REMOTE_USER}: denied (no authenticated user yet)

What’s confusing me is that “require valid-user” works, but then directly afterwards the expression fails saying the user that one line before has been authenticated now isn’t authenticated, and that’s not making sense.

Regards,
Graham


smime.p7s (4K) Download Attachment