Apache memory usage

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Apache memory usage

Danny Mallory
Anyone here know a good way to tell what Apache may be chewing up memory on? 

I have Apache running purely as a reverse proxy (not hosting anything), timeouts configured everywhere and good cleanup on threads, scoreboard staying clean,  but after a clean restart it will slowly climb and climb throughout the day using more and more memory then when it hits around 85% we start seeing core notices, then warnings and finally errors and death. 

child segmentation fault (notice) 
still did not exit (SIGTERM) warning
still did not exit (SIGTERM) error

Every process is dead from above logs entries except one remaining that will not stop and no responses anymore from apache without kill -9 and restarting. 

thx in advance,
Danny
Reply | Threaded
Open this post in threaded view
|

Re: Apache memory usage

Yann Ylavic
On Thu, Jun 18, 2020 at 8:03 PM Danny Mallory <[hidden email]> wrote:
>
> Anyone here know a good way to tell what Apache may be chewing up memory on?

Do you have MaxMemFree configured already
(https://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxmemfree)?
If not, does "MaxMemFree 2048" (for instance) help?

Regards;
Yann.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Apache memory usage

Yann Ylavic
On Fri, Jun 19, 2020 at 4:20 PM Yann Ylavic <[hidden email]> wrote:
>
> On Thu, Jun 18, 2020 at 8:03 PM Danny Mallory <[hidden email]> wrote:
> >
> > Anyone here know a good way to tell what Apache may be chewing up memory on?
>
> Do you have MaxMemFree configured already
> (https://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxmemfree)?
> If not, does "MaxMemFree 2048" (for instance) help?

Nevermind, MaxMemFree 2048 is already the default so it's likely not the issue.

Which modules do you load?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Apache memory usage

Danny Mallory
In reply to this post by Yann Ylavic
I do not have that installed but will definitely look into it as it may have a similar effect on what I found.  The problem I found (well kind of) is that there is some unknown memory leak. Was looking at another thread where it stated if there is a memory leak in one of the modules and MaxConnectionsPerChild (mpm_event) is set to 0 then it will slowly climb and eat all your memory.  Sure enough I changed that from 0 to 4000 (just a guess) and memory has been flat lined even since. In addition Apache CPU usage on my xymon graphs was just all over the place throughout the day and its flatlined now too. It could be the nature of the app behind this reverse proxy as it performs horribly and there are LOTs of items that just timeout and dont get responded to so we had to be aggressive on our timeouts in the first place. 



On Fri, Jun 19, 2020 at 9:20 AM Yann Ylavic <[hidden email]> wrote:
On Thu, Jun 18, 2020 at 8:03 PM Danny Mallory <[hidden email]> wrote:
>
> Anyone here know a good way to tell what Apache may be chewing up memory on?

Do you have MaxMemFree configured already
(https://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxmemfree)?
If not, does "MaxMemFree 2048" (for instance) help?

Regards;
Yann.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Apache memory usage

Danny Mallory
In reply to this post by Yann Ylavic
mods enabled: 
access_compat.load  authz_groupfile.load  dir.load      mpm_event.conf      proxy.load          ssl.conf
alias.conf          authz_host.load       env.load      mpm_event.load      remoteip.load       ssl.load
alias.load          authz_user.load       filter.load   negotiation.conf    reqtimeout.conf     status.conf
auth_basic.load     autoindex.conf        headers.load  negotiation.load    reqtimeout.load     status.load
authn_core.load     autoindex.load        ldap.conf     proxy_ajp.load      rewrite.load        vhost_alias.load
authn_file.load     deflate.conf          ldap.load     proxy.conf          setenvif.conf
authnz_ldap.load    deflate.load          mime.conf     proxy_connect.load  setenvif.load
authz_core.load     dir.conf              mime.load     proxy_http.load     socache_shmcb.load


We did find a LOT of ldap connections not being closed also (when Apache was hung we saw over 300 established LDAP connections).  While it didnt fix the overall memory leak we fixed those number of connections hung by forcing the ldap connection and pool to timeout after 60s.  LDAP seems suspect for the memory leak but not sure. 

On Fri, Jun 19, 2020 at 9:28 AM Yann Ylavic <[hidden email]> wrote:
On Fri, Jun 19, 2020 at 4:20 PM Yann Ylavic <[hidden email]> wrote:
>
> On Thu, Jun 18, 2020 at 8:03 PM Danny Mallory <[hidden email]> wrote:
> >
> > Anyone here know a good way to tell what Apache may be chewing up memory on?
>
> Do you have MaxMemFree configured already
> (https://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxmemfree)?
> If not, does "MaxMemFree 2048" (for instance) help?

Nevermind, MaxMemFree 2048 is already the default so it's likely not the issue.

Which modules do you load?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]