Topic for discussion... 2.4.26

classic Classic list List threaded Threaded
83 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Topic for discussion... 2.4.26

Jim Jagielski
Would be nice, I think, to start discussion on a T&R of 2.4.26 and
to open the doors to who wants to RM. Note, that if *nobody*
offers to RM, I will... and no matter what, I offer to help
whoever wishes to RM.
Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

William A Rowe Jr
With the passing of OpenSSL 1.0.1, is OpenSSL 1.1.0 on our radar for the next release?

I'm not clear how that merge branch is intended to be used, I'm don't understand whether we propose to adopt every feature and API change commit to modules/ssl/* - and why it has been rebased, unless we intend to svn cp the resulting tree on top of modules/ssl/.

I'm set up to review it against 1.1.0, if I understood how that branch would be applied.


On Feb 16, 2017 11:25 AM, "Jim Jagielski" <[hidden email]> wrote:
Would be nice, I think, to start discussion on a T&R of 2.4.26 and
to open the doors to who wants to RM. Note, that if *nobody*
offers to RM, I will... and no matter what, I offer to help
whoever wishes to RM.
Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Stefan Eissing
Also interested in the state of the openssl 1.1.0 support. Having it in the next release would be great. OpenSSL has promised TLS 1.3 beginning of April as a drop in against the 1.1.0 ABI - which remains to be seem if that works, but would be nice to be ready for it.

> Am 17.02.2017 um 00:46 schrieb William A Rowe Jr <[hidden email]>:
>
> With the passing of OpenSSL 1.0.1, is OpenSSL 1.1.0 on our radar for the next release?
>
> I'm not clear how that merge branch is intended to be used, I'm don't understand whether we propose to adopt every feature and API change commit to modules/ssl/* - and why it has been rebased, unless we intend to svn cp the resulting tree on top of modules/ssl/.
>
> I'm set up to review it against 1.1.0, if I understood how that branch would be applied.
>
>
> On Feb 16, 2017 11:25 AM, "Jim Jagielski" <[hidden email]> wrote:
> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
> to open the doors to who wants to RM. Note, that if *nobody*
> offers to RM, I will... and no matter what, I offer to help
> whoever wishes to RM.

Stefan Eissing

<green/>bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de

Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Luca Toscano
My personal wishlist:

1) Openssl 1.1.x support, a lot of people are asking for it in various support channels and it seems important to catch up with others project that already support it :)

2) Yann's work on mpm-event to remove the unnecessary 100ms of polling even when idling. I am really looking forward to see this patch on 2.4.x, but it might need more testing on platform different from Linux.

3) New mod_http2 features from Stefan to fix recent issues reported in dev@ 

4) mod_proxy_fcgi rework from Jim, Jacob and Eric.


Luca

2017-02-17 10:12 GMT+01:00 Stefan Eissing <[hidden email]>:
Also interested in the state of the openssl 1.1.0 support. Having it in the next release would be great. OpenSSL has promised TLS 1.3 beginning of April as a drop in against the 1.1.0 ABI - which remains to be seem if that works, but would be nice to be ready for it.

> Am 17.02.2017 um 00:46 schrieb William A Rowe Jr <[hidden email]>:
>
> With the passing of OpenSSL 1.0.1, is OpenSSL 1.1.0 on our radar for the next release?
>
> I'm not clear how that merge branch is intended to be used, I'm don't understand whether we propose to adopt every feature and API change commit to modules/ssl/* - and why it has been rebased, unless we intend to svn cp the resulting tree on top of modules/ssl/.
>
> I'm set up to review it against 1.1.0, if I understood how that branch would be applied.
>
>
> On Feb 16, 2017 11:25 AM, "Jim Jagielski" <[hidden email]> wrote:
> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
> to open the doors to who wants to RM. Note, that if *nobody*
> offers to RM, I will... and no matter what, I offer to help
> whoever wishes to RM.

Stefan Eissing

<green/>bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de


Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Jim Jagielski
In reply to this post by William A Rowe Jr
All IMHO:

> On Feb 16, 2017, at 6:46 PM, William A Rowe Jr <[hidden email]> wrote:
>
> With the passing of OpenSSL 1.0.1, is OpenSSL 1.1.0 on our radar for the next release?

Depends on the status of the patch support...

>
> I'm not clear how that merge branch is intended to be used, I'm don't understand whether we propose to adopt every feature and API change commit to modules/ssl/* - and why it has been rebased, unless we intend to svn cp the resulting tree on top of modules/ssl/.
>
> I'm set up to review it against 1.1.0, if I understood how that branch would be applied.
>
>
> On Feb 16, 2017 11:25 AM, "Jim Jagielski" <[hidden email]> wrote:
> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
> to open the doors to who wants to RM. Note, that if *nobody*
> offers to RM, I will... and no matter what, I offer to help
> whoever wishes to RM.

Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Jim Jagielski
In reply to this post by Jim Jagielski
Should we start thinking about having a release this month?

> On Feb 16, 2017, at 12:25 PM, Jim Jagielski <[hidden email]> wrote:
>
> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
> to open the doors to who wants to RM. Note, that if *nobody*
> offers to RM, I will... and no matter what, I offer to help
> whoever wishes to RM.

Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Eric Covener
On Thu, Mar 2, 2017 at 9:27 AM, Jim Jagielski <[hidden email]> wrote:
> Should we start thinking about having a release this month?
>
>> On Feb 16, 2017, at 12:25 PM, Jim Jagielski <[hidden email]> wrote:
>>
>> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
>> to open the doors to who wants to RM. Note, that if *nobody*
>> offers to RM, I will... and no matter what, I offer to help
>> whoever wishes to RM.
>

+1 but keep in mind pwn2own has that mid-month (3-15/3-17) event w/
the big  Apache-on-Ubuntu bounty.
Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Jim Jagielski
Right... I was thinking the latter half of the month

> On Mar 2, 2017, at 9:31 AM, Eric Covener <[hidden email]> wrote:
>
> On Thu, Mar 2, 2017 at 9:27 AM, Jim Jagielski <[hidden email]> wrote:
>> Should we start thinking about having a release this month?
>>
>>> On Feb 16, 2017, at 12:25 PM, Jim Jagielski <[hidden email]> wrote:
>>>
>>> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
>>> to open the doors to who wants to RM. Note, that if *nobody*
>>> offers to RM, I will... and no matter what, I offer to help
>>> whoever wishes to RM.
>>
>
> +1 but keep in mind pwn2own has that mid-month (3-15/3-17) event w/
> the big  Apache-on-Ubuntu bounty.

Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Stefan Eissing
+1

> Am 02.03.2017 um 16:48 schrieb Jim Jagielski <[hidden email]>:
>
> Right... I was thinking the latter half of the month
>
>> On Mar 2, 2017, at 9:31 AM, Eric Covener <[hidden email]> wrote:
>>
>> On Thu, Mar 2, 2017 at 9:27 AM, Jim Jagielski <[hidden email]> wrote:
>>> Should we start thinking about having a release this month?
>>>
>>>> On Feb 16, 2017, at 12:25 PM, Jim Jagielski <[hidden email]> wrote:
>>>>
>>>> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
>>>> to open the doors to who wants to RM. Note, that if *nobody*
>>>> offers to RM, I will... and no matter what, I offer to help
>>>> whoever wishes to RM.
>>>
>>
>> +1 but keep in mind pwn2own has that mid-month (3-15/3-17) event w/
>> the big  Apache-on-Ubuntu bounty.
>

Stefan Eissing

<green/>bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de

Reply | Threaded
Open this post in threaded view
|

Re: Topic for discussion... 2.4.26

Eric Covener
In reply to this post by Eric Covener
Apparently unscathed / unattempted.

https://twitter.com/zh4ck/status/843036999569346560

On Thu, Mar 2, 2017 at 9:31 AM, Eric Covener <[hidden email]> wrote:

> On Thu, Mar 2, 2017 at 9:27 AM, Jim Jagielski <[hidden email]> wrote:
>> Should we start thinking about having a release this month?
>>
>>> On Feb 16, 2017, at 12:25 PM, Jim Jagielski <[hidden email]> wrote:
>>>
>>> Would be nice, I think, to start discussion on a T&R of 2.4.26 and
>>> to open the doors to who wants to RM. Note, that if *nobody*
>>> offers to RM, I will... and no matter what, I offer to help
>>> whoever wishes to RM.
>>
>
> +1 but keep in mind pwn2own has that mid-month (3-15/3-17) event w/
> the big  Apache-on-Ubuntu bounty.



--
Eric Covener
[hidden email]
Reply | Threaded
Open this post in threaded view
|

The drive for 2.4.26

Jim Jagielski
Let's shoot for a 2.4.26 within the next handful of
weeks. There are some entries in STATUS that could
use some love and attention, and I'm hoping/pushing
for a Brotli[1] release so we can fold *that* capability
in as well.

1. https://github.com/google/brotli
   https://github.com/google/brotli/issues/483
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

William A Rowe Jr
On Wed, Mar 29, 2017 at 2:37 PM, Jim Jagielski <[hidden email]> wrote:
> Let's shoot for a 2.4.26 within the next handful of
> weeks.

++1 - my only question is whether we can get an apr[-util] release in the
next week or two ahead of our release, to encourage users to update their
entire stack?

> There are some entries in STATUS that could
> use some love and attention, and I'm hoping/pushing
> for a Brotli[1] release so we can fold *that* capability
> in as well.
>
> 1. https://github.com/google/brotli
>    https://github.com/google/brotli/issues/483

+1. The docs are still lacking, I'm hoping to add some caveats about never
using the module dynamically unless fronted with a content cache. It seems
like we might need an entire docs page dedicated to content encoding to
keep folks from doing things like enabling this module (except for significantly
large generated content), but dynamically routing requests to corresponding
.br or .gz content files as the client advertises. The current
copy-paste silliness
for static content misses interop between brotli and gzip, disabling compression
to half of all clients. Given this limited issue and ~1 week - this
seems fixable.

With luck we can fix remoteip PROXY support to do what the spec says, as
well.
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Evgeny Kotkov
In reply to this post by Jim Jagielski
Jim Jagielski <[hidden email]> writes:

> Let's shoot for a 2.4.26 within the next handful of
> weeks. There are some entries in STATUS that could
> use some love and attention, and I'm hoping/pushing
> for a Brotli[1] release so we can fold *that* capability
> in as well.
>
> 1. https://github.com/google/brotli
>    https://github.com/google/brotli/issues/483

I noticed that the current mod_brotli backport proposal lacks a couple of
changes that allow building against the official repository.  I think that
the core change (excluding the docs) should be:

    https://svn.apache.org/r1761714
    https://svn.apache.org/r1761824
    https://svn.apache.org/r1762512
    https://svn.apache.org/r1762515
    https://svn.apache.org/r1771789
    https://svn.apache.org/r1771791
    https://svn.apache.org/r1771827
    https://svn.apache.org/r1779077
    https://svn.apache.org/r1779111
    https://svn.apache.org/r1779700
    https://svn.apache.org/r1790852
    https://svn.apache.org/r1790853
    https://svn.apache.org/r1790860

For the convenience, here is a shortlog for these changes:

    r1761714: Add initial implementation.

    r1761824: Unbreak building other filter modules without libbrotlienc.

    r1762512: Allow compression ratio logging with new BrotliFilterNote
    directive.

    r1762515: Handle new 'no-brotli' internal environment variable that
    disables Brotli compression for a particular request.

    r1771789: Rewrite the autoconf script in a, hopefully, less convoluted way.
    This lays the groundwork to simplify the switch to the official Brotli
    library.

    r1771791: Explicitly cast 'const uint8_t *' to 'const char *' when using
    the data received from Brotli to create a bucket.

    r1771827: Update makefiles to use the library layout of the official
    Brotli repository.

    r1779077: Unused variable error could mistakenly note that brotli isn't
    available.

    r1779111: Update makefile to cope with the pkg-config layout change
    in https://github.com/google/brotli/commit/fe9f9a9

    r1779700: Save a few bytes and a few cycles.

    r1790852: Update makefile to allow using Brotli library >= 0.6.0.

    r1790853: Fix a minor typo in the description of BrotliAlterETag
    that has been referring to httpd 2.2.x.

    r1790860: Comment on the default choice (0) for BROTLI_PARAM_LGBLOCK.


Regards,
Evgeny Kotkov
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Stefan Eissing
Nice work, looking forward to seeing this backported!

> Am 10.04.2017 um 18:24 schrieb Evgeny Kotkov <[hidden email]>:
>
> Jim Jagielski <[hidden email]> writes:
>
>> Let's shoot for a 2.4.26 within the next handful of
>> weeks. There are some entries in STATUS that could
>> use some love and attention, and I'm hoping/pushing
>> for a Brotli[1] release so we can fold *that* capability
>> in as well.
>>
>> 1. https://github.com/google/brotli
>>   https://github.com/google/brotli/issues/483
>
> I noticed that the current mod_brotli backport proposal lacks a couple of
> changes that allow building against the official repository.  I think that
> the core change (excluding the docs) should be:
>
>    https://svn.apache.org/r1761714
>    https://svn.apache.org/r1761824
>    https://svn.apache.org/r1762512
>    https://svn.apache.org/r1762515
>    https://svn.apache.org/r1771789
>    https://svn.apache.org/r1771791
>    https://svn.apache.org/r1771827
>    https://svn.apache.org/r1779077
>    https://svn.apache.org/r1779111
>    https://svn.apache.org/r1779700
>    https://svn.apache.org/r1790852
>    https://svn.apache.org/r1790853
>    https://svn.apache.org/r1790860
>
> For the convenience, here is a shortlog for these changes:
>
>    r1761714: Add initial implementation.
>
>    r1761824: Unbreak building other filter modules without libbrotlienc.
>
>    r1762512: Allow compression ratio logging with new BrotliFilterNote
>    directive.
>
>    r1762515: Handle new 'no-brotli' internal environment variable that
>    disables Brotli compression for a particular request.
>
>    r1771789: Rewrite the autoconf script in a, hopefully, less convoluted way.
>    This lays the groundwork to simplify the switch to the official Brotli
>    library.
>
>    r1771791: Explicitly cast 'const uint8_t *' to 'const char *' when using
>    the data received from Brotli to create a bucket.
>
>    r1771827: Update makefiles to use the library layout of the official
>    Brotli repository.
>
>    r1779077: Unused variable error could mistakenly note that brotli isn't
>    available.
>
>    r1779111: Update makefile to cope with the pkg-config layout change
>    in https://github.com/google/brotli/commit/fe9f9a9
>
>    r1779700: Save a few bytes and a few cycles.
>
>    r1790852: Update makefile to allow using Brotli library >= 0.6.0.
>
>    r1790853: Fix a minor typo in the description of BrotliAlterETag
>    that has been referring to httpd 2.2.x.
>
>    r1790860: Comment on the default choice (0) for BROTLI_PARAM_LGBLOCK.
>
>
> Regards,
> Evgeny Kotkov

Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Jim Jagielski
We are very, very close to being in a releasable state... looks
like just 1 show-stopper. If we also want to wait until APR 1.6
is released, we can also look at having redis/memcached parity in
socache as well, which would be good for 2.4.26...

Thoughts?

PS: it would be great to have this out by ApacheCon next month.
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Gregg Smith (gsmith)
ABS doesn't work with openssl 1.1.0, on windows anyway. It builds
without warning yet doesn't work.

abs https://www.domain.com
just sits there forever and never completes or shows anything.

I cannot imagine this being a windows only problem.

On 4/20/2017 3:24 AM, Jim Jagielski wrote:
> We are very, very close to being in a releasable state... looks
> like just 1 show-stopper. If we also want to wait until APR 1.6
> is released, we can also look at having redis/memcached parity in
> socache as well, which would be good for 2.4.26...
>
> Thoughts?
>
> PS: it would be great to have this out by ApacheCon next month.
>
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Rainer Jung-3
Am 20.04.2017 um 16:31 schrieb Gregg Smith:
> ABS doesn't work with openssl 1.1.0, on windows anyway. It builds
> without warning yet doesn't work.
>
> abs https://www.domain.com
> just sits there forever and never completes or shows anything.
>
> I cannot imagine this being a windows only problem.

Any chance you can try the trunk version:

https://svn.apache.org/repos/asf/httpd/httpd/trunk/support/ab.c

The chances are good, that you can simply replace the ab.c source file
and recompile. No need to compile a complete httpd trunk. If the trunk
version works, we could focus on the delta which is not huge but noticeable.

Thanks!

Rainer

> On 4/20/2017 3:24 AM, Jim Jagielski wrote:
>> We are very, very close to being in a releasable state... looks
>> like just 1 show-stopper. If we also want to wait until APR 1.6
>> is released, we can also look at having redis/memcached parity in
>> socache as well, which would be good for 2.4.26...
>>
>> Thoughts?
>>
>> PS: it would be great to have this out by ApacheCon next month.
>>
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Jacob Champion-2
In reply to this post by Gregg Smith (gsmith)
On 04/20/2017 07:31 AM, Gregg Smith wrote:
> ABS doesn't work with openssl 1.1.0, on windows anyway. It builds
> without warning yet doesn't work.
>
> abs https://www.domain.com
> just sits there forever and never completes or shows anything.
>
> I cannot imagine this being a windows only problem.

I haven't tested Windows yet, but in Ubuntu, ab built with OpenSSL 1.1.0
hangs whether you're using HTTP or HTTPS.

We call OPENSSL_malloc_init(), which in 1.1.0 is documented to be
unnecessary except "in certain shared-library situations." (I haven't
found documented examples of these "situations" yet.) This is a macro
that just sets a bunch of malloc callbacks to their defaults.

Unfortunately on my machine, the "default" functions are actually
translated into PLT stubs for the linker -- it's a macro call, so it
uses the executable's addresses for the functions rather than the
library's. So CRYPTO_malloc calls the PLT stub which calls CRYPTO_malloc
which calls the PLT stub which recurses into madness. Configuring
OpenSSL with --debug turns the hang into a stack overflow like we'd expect.

On the one hand, it's arguably an API bug in OpenSSL, but I get the
feeling that we're not supposed to call most of these initialization
functions anymore as of 1.1.0.

--Jacob
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Gregg Smith (gsmith)
In reply to this post by Rainer Jung-3
This is ApacheBench, Version 2.3 <$Revision: 1750960 $>
Same result with trunk, it just hangs.

Glad it's not just Windows!

On 4/20/2017 9:48 AM, Rainer Jung wrote:

> Am 20.04.2017 um 16:31 schrieb Gregg Smith:
>> ABS doesn't work with openssl 1.1.0, on windows anyway. It builds
>> without warning yet doesn't work.
>>
>> abs https://www.domain.com
>> just sits there forever and never completes or shows anything.
>>
>> I cannot imagine this being a windows only problem.
>
> Any chance you can try the trunk version:
>
> https://svn.apache.org/repos/asf/httpd/httpd/trunk/support/ab.c
>
> The chances are good, that you can simply replace the ab.c source file
> and recompile. No need to compile a complete httpd trunk. If the trunk
> version works, we could focus on the delta which is not huge but
> noticeable.
>
> Thanks!
>
> Rainer
>
>> On 4/20/2017 3:24 AM, Jim Jagielski wrote:
>>> We are very, very close to being in a releasable state... looks
>>> like just 1 show-stopper. If we also want to wait until APR 1.6
>>> is released, we can also look at having redis/memcached parity in
>>> socache as well, which would be good for 2.4.26...
>>>
>>> Thoughts?
>>>
>>> PS: it would be great to have this out by ApacheCon next month.
>>>
Reply | Threaded
Open this post in threaded view
|

Re: The drive for 2.4.26

Rainer Jung-3
In reply to this post by Jacob Champion-2
Am 20.04.2017 um 21:23 schrieb Jacob Champion:

> On 04/20/2017 07:31 AM, Gregg Smith wrote:
>> ABS doesn't work with openssl 1.1.0, on windows anyway. It builds
>> without warning yet doesn't work.
>>
>> abs https://www.domain.com
>> just sits there forever and never completes or shows anything.
>>
>> I cannot imagine this being a windows only problem.
>
> I haven't tested Windows yet, but in Ubuntu, ab built with OpenSSL 1.1.0
> hangs whether you're using HTTP or HTTPS.
>
> We call OPENSSL_malloc_init(), which in 1.1.0 is documented to be
> unnecessary except "in certain shared-library situations." (I haven't
> found documented examples of these "situations" yet.) This is a macro
> that just sets a bunch of malloc callbacks to their defaults.
>
> Unfortunately on my machine, the "default" functions are actually
> translated into PLT stubs for the linker -- it's a macro call, so it
> uses the executable's addresses for the functions rather than the
> library's. So CRYPTO_malloc calls the PLT stub which calls CRYPTO_malloc
> which calls the PLT stub which recurses into madness. Configuring
> OpenSSL with --debug turns the hang into a stack overflow like we'd expect.
>
> On the one hand, it's arguably an API bug in OpenSSL, but I get the
> feeling that we're not supposed to call most of these initialization
> functions anymore as of 1.1.0.

Thanks for the analysis. So the following patch on trunk works for me
when using OpenSSL 1.0.1e (on Solaris 10):

Index: support/ab.c
===================================================================
--- support/ab.c        (revision 1792155)
+++ support/ab.c        (working copy)
@@ -2576,8 +2576,6 @@
  #else
  #if OPENSSL_VERSION_NUMBER < 0x10100000L
      CRYPTO_malloc_init();
-#else
-    OPENSSL_malloc_init();
  #endif
  #endif
      SSL_load_error_strings();


The same fix should apply for 2.4.x.

In addition I noticed the following glitch:

Index: support/ab.c
===================================================================
--- support/ab.c        (revision 1792155)
+++ support/ab.c        (working copy)
@@ -2465,14 +2465,14 @@
              case 'B':
                  myhost = apr_pstrdup(cntxt, opt_arg);
                  break;
+            case 'm':
+                method = CUSTOM_METHOD;
+                method_str[CUSTOM_METHOD] = strdup(opt_arg);
+                break;
  #ifdef USE_SSL
              case 'Z':
                  ssl_cipher = strdup(opt_arg);
                  break;
-            case 'm':
-                method = CUSTOM_METHOD;
-                method_str[CUSTOM_METHOD] = strdup(opt_arg);
-                break;
              case 'f':
  #if OPENSSL_VERSION_NUMBER < 0x10100000L
                  if (strncasecmp(opt_arg, "ALL", 3) == 0) {


The "-m" option is independent of SSL use and should be handled outside
of "#ifdef USE_SSL".

Will apply some time over the weekend if noone does it before.

Regards,

Rainer
12345