[Bug 64709] New: mod_cgi not returning 504 properly after TimeOut

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Bug 64709] New: mod_cgi not returning 504 properly after TimeOut

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64709

            Bug ID: 64709
           Summary: mod_cgi not returning 504 properly after TimeOut
           Product: Apache httpd-2
           Version: 2.4.43
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mod_cgi
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ---

My TimeOut directive is set to 300 seconds, but I get a 504 error in my browser
when using mod_cgi after 600 seconds.

Interestingly, Apache logs the following after 300 seconds, but it doesn't
return the response to my browser, and it does not terminate or kill the CGI
script. The CGI script keeps running and my browser keeps waiting for content
from Apache.

[Fri Sep 04 03:16:13.955573 2020] [cgi:warn] [pid 7709] [client
172.20.0.1:42238] AH01220: Timeout waiting for output from CGI script
/kohadevbox/koha/tools/import_borrowers.pl, referer:
http://localhost:8081/cgi-bin/koha/tools/import_bo
rrowers.pl
[Fri Sep 04 03:16:13.955703 2020] [cgi:error] [pid 7709] [client
172.20.0.1:42238] Script timed out before returning headers:
import_borrowers.pl, referer:
http://localhost:8081/cgi-bin/koha/tools/import_borrowers.pl

Then 300 seconds later, Apache logs another timeout, and that's when I receive
a 504 error in my browser and the CGI script is killed.

[Fri Sep 04 03:21:14.027398 2020] [cgi:warn] [pid 7709] [client
172.20.0.1:42238] AH01220: Timeout waiting for output from CGI script
/kohadevbox/koha/tools/import_borrowers.pl, referer:
http://localhost:8081/cgi-bin/koha/tools/import_bo
rrowers.pl

I've poked around the httpd and apr source code, but I can't find the answer.

I've also used the ScriptLog directive, and it records a 504 in the log after
the first timeout, which is consistent with the "server/util_script.c" code
that shows returns HTTP_GATEWAY_TIME_OUT.  
%% [Fri Sep 04 03:16:13 2020] POST /cgi-bin/koha/tools/import_borrowers.pl
HTTP/1.1
%% 504 /kohadevbox/koha/tools/import_borrowers.pl

It seems to me like the CGI script cleanup isn't happening in a timely fashion.
I'm guessing that Apache must try to read from the CGI process 1 last time
before terminating it, and that is where the 2nd timeout happens and the actual
HTTP response is returned to the browser.

But that's just a guess.

I'm hoping that you folks will be able to find the answer.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64709] mod_cgi not returning 504 properly after TimeOut

Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64709

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|2.4.43                      |2.4.25

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64709] mod_cgi not returning 504 properly after TimeOut

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64709

--- Comment #1 from Joe Orton <[hidden email]> ---
Created attachment 37422
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=37422&action=edit
mod_cgi

Thanks for the report, can you try this patch?

I've committed the equivalent to trunk in r1881459 - possibly there are some
other cases where there is a double-timeout effect, but this seems to catch the
common 504 error path.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64709] mod_cgi not returning 504 properly after TimeOut

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64709

Joe Orton <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64709] mod_cgi not returning 504 properly after TimeOut

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64709

--- Comment #2 from [hidden email] ---
Thanks for the quick response, Joe!

I had never built Apache httpd from source before, so I took the Dockerfile
from the official Docker image
(https://github.com/docker-library/httpd/blob/5bc2d71c4c1949d98a2e8671eadc5043b6c0b7f4/2.4/Dockerfile)
and modified it to include your patch.

I've tested my modified image against the official image using a simple Perl
CGI script, and I can confirm that your patch works.

With your patch, I get the 504 in my browser at TimeOut time and the CGI script
stops executing.

Without your patch, I see the timeout in the logs, but the CGI script keeps
running. I don't get the 504 and the CGI script doesn't terminate until a 2nd
timeout occurs.

So that's perfect. Thanks for that fix. Please let me know if there is anything
else you need from me.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64709] mod_cgi not returning 504 properly after TimeOut

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64709

--- Comment #3 from [hidden email] ---
Should we update the version from 2.4.25 to 2.4.46 since I've now reproduced
the bug in that version as well?

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[Bug 64709] mod_cgi not returning 504 properly after TimeOut

Bugzilla from bugzilla@apache.org
In reply to this post by Bugzilla from bugzilla@apache.org
https://bz.apache.org/bugzilla/show_bug.cgi?id=64709

Joe Orton <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|2.4.25                      |2.4.46
             Status|NEEDINFO                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #4 from Joe Orton <[hidden email]> ---
Great, thanks a lot for confirming the fix.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]