Fixed some missing includes. Thanks.

/ Martin Stjernholm, Roxen IS

Previous text:

2004-09-27 00:07: Subject: memory leak in oracle module

---

thanx. compiled pike 7.4 from cvs, but failed with 7.6 from cvs (see below).

when now running running roxen with valgrind starting of roxen up hangs while starting mysql. but i have to include child processes with valgrind, otherwise it would trace only the start script.

there were also "Broken environment var" messages in the debug log.

/m

compiler error when i tried to compile todays pike 7.6 with-valgrind:

Making Gmp make[3]: Entering directory `/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/modules/Gmp' Compiling /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `mpzmod_reduce': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: error: `gc_external_refs_zapped' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: error: (Each undeclared identifier is reported only once /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: error: for each function it appears in.) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: warning: implicit declaration of function `gc_check_zapped' /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `debug_get_mpz': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:447: error: `gc_external_refs_zapped' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `mpzmod_invert': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:1377: error: `gc_external_refs_zapped' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `pike_module_exit': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:1919: error: `gc_external_refs_zapped' undeclared (first use in this function) WARNING: Compiler failure! Trying without optimization! cc1: warning: -Wuninitialized is not supported without -O /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `mpzmod_reduce': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: error: `gc_external_refs_zapped' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: error: (Each undeclared identifier is reported only once /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: error: for each function it appears in.) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:104: warning: implicit declaration of function `gc_check_zapped' /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `debug_get_mpz': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:447: error: `gc_external_refs_zapped' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `mpzmod_invert': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:1377: error: `gc_external_refs_zapped' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c: In function `pike_module_exit': /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c:1919: error: `gc_external_refs_zapped' undeclared (first use in this function) Compilation command was: /tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/smartlink gcc -I. -I/tmp/Pike-v7.6-snapshot/src/modules/Gmp -I/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686 -I/tmp/Pike-v7.6-snapshot/src -I/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/bundles/include -I/usr/local/include -I/opt/gnome/include -I/usr/X11/include -DHAVE_CONFIG_H -g -Wa,--execstack -mcpu=i686 -I/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/bundles/include -I/usr/local/include -I/opt/gnome/include -I/usr/X11/include -W -Wall -Wno-unused -Wcomment -Wformat -Wimplicit-function-declaration -Wmultichar -Wswitch -Wuninitialized -Wpointer-arith -Wchar-subscripts -Wno-long-long -c /tmp/Pike-v7.6-snapshot/src/modules/Gmp/mpz_glue.c -o mpz_glue.o make[3]: *** [mpz_glue.o] Error 1 make[3]: Leaving directory `/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/modules/Gmp' make[2]: *** [Gmp] Error 1 make[2]: Leaving directory `/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/modules' make[1]: *** [module_objects] Error 1 make[1]: Leaving directory `/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686' make: *** [compile] Error 2

/ Michael J. Stenitzer

sorry, i still get an compilation error when compiling --with-valgrind (see below).

another irritating problem: i do not get an error stopping the compilation when valgrind is not found (in the path). it was just a message being lost in the output of make.

error when compiling --with-valgrind

Compiling /tmp/Pike-v7.6-snapshot/src/gc.c /tmp/Pike-v7.6-snapshot/src/gc.c: In function `cleanup_markers': /tmp/Pike-v7.6-snapshot/src/gc.c:1539: error: `GC_CLEANUP_FREED' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/gc.c:1539: error: (Each undeclared identifier is reported only once /tmp/Pike-v7.6-snapshot/src/gc.c:1539: error: for each function it appears in.) /tmp/Pike-v7.6-snapshot/src/gc.c:1540: error: structure has no member named `xrefs' /tmp/Pike-v7.6-snapshot/src/gc.c:1541: error: structure has no member named `saved_refs' WARNING: Compiler failure! Trying without optimization! cc1: warning: -Wuninitialized is not supported without -O /tmp/Pike-v7.6-snapshot/src/gc.c: In function `cleanup_markers': /tmp/Pike-v7.6-snapshot/src/gc.c:1539: error: `GC_CLEANUP_FREED' undeclared (first use in this function) /tmp/Pike-v7.6-snapshot/src/gc.c:1539: error: (Each undeclared identifier is reported only once /tmp/Pike-v7.6-snapshot/src/gc.c:1539: error: for each function it appears in.) /tmp/Pike-v7.6-snapshot/src/gc.c:1540: error: structure has no member named `xrefs' /tmp/Pike-v7.6-snapshot/src/gc.c:1541: error: structure has no member named `saved_refs' Compilation command was: /tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/smartlink gcc -DPIKE_SRC_ROOT=/tmp/Pike-v7.6-snapshot -I. -I/tmp/Pike-v7.6-snapshot/src -I/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/bundles/include -I/usr/local/include -I/opt/gnome/include -I/usr/X11/include -g -Wa,--execstack -mcpu=i686 -I/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686/bundles/include -I/usr/local/include -I/opt/gnome/include -I/usr/X11/include -W -Wall -Wno-unused -Wcomment -Wformat -Wimplicit-function-declaration -Wmultichar -Wswitch -Wuninitialized -Wpointer-arith -Wchar-subscripts -Wno-long-long -c /tmp/Pike-v7.6-snapshot/src/gc.c -o gc.o make[2]: *** [gc.o] Error 1 make[2]: Leaving directory `/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686' make[1]: *** [all] Error 2 make[1]: Leaving directory `/tmp/Pike-v7.6-snapshot/build/linux-2.4.21-243-default-i686' make: *** [compile] Error 2

/ Michael J. Stenitzer

Previous text:

2004-09-27 00:33: Subject: memory leak in oracle module

---

Sorry, that was a bit hasty. Now it got a chance of working.

/ Martin Stjernholm, Roxen IS

It doesn't look like its working. Arithmetic exception on all xenofarm systems.

/ Martin Nilsson (DivX Networks)

Previous text:

2004-09-28 00:07: Subject: memory leak in oracle module

---

...

sorry, i still get an compilation error when compiling --with-valgrind

Now it should work. Note that --with-valgrind implies --with-cleanup-at-exit to avoid noise in the leak report from valgrind. It was the cleanup-at-exit code that didn't work without rtldebug.

...

another irritating problem: i do not get an error stopping the compilation when valgrind is not found (in the path). it was just a message being lost in the output of make.

It's collected and shown last in the summary at the end of configure. I think that's a good way to report it.

/ Martin Stjernholm, Roxen IS

Yes. Thank you.

--with-rtldebug still doesn't work. gc_check_zapped inside of #ifdef PIKE_DEBUG uses GC_CLEANUP_FREED, which needs DO_PIKE_CLEANUP.

/ Martin Nilsson (DivX Networks)

Previous text:

2004-09-28 19:08: Subject: memory leak in oracle module

---

Now, perhaps?

/ Martin Stjernholm, Roxen IS

Right. Missed that in my scrollback buffer.

/ Marcus Agehall (PacketFront)

Previous text:

2004-09-27 17:53: Subject: memory leak in oracle module

---

I cannot provide a test case, but I can provide another leak-confirmation:

My roxen installation seems to leak a few Mb per hour, even though I restart it every night at 04:00, it's currently 1.9Gb large (which is getting close to the memory limit)

According to the pike-memory-usage action, 18Mb is used.

I do not use Oracle. I do make heavy use of MySQL, though.

/ Per Hedbor ()

I now made a simple test case with a select query from an rxml page. Still doesn't leak, but if I push many concurrent requests it dumps core on me in OCIHandleFree called from oracle.c:497 (testing on Solaris 8 sparc). That's not good either, but it seems to be unrelated to the issue at hand. Still, what are your experiences with Oracle and concurrent queries?

As for test case, I'd naturally prefer one that is as minimized as possible. What sort of operation causes the leak (connecting, selecting, insert, updates, transactions, etc)? Are concurrent threads involved? Any sort of closer characterstics are useful.

Maybe you can make a replica of your server and use ab to find out which specific page causes the leak and thereby the queries involved?

/ Martin Stjernholm, Roxen IS

Previous text:

2004-09-27 18:05: Subject: memory leak in oracle module

---

well, what could i exactly provide? website: no problem, server-config also no problem. but without the oracle db not helpful. if its really helpful, i could ask our db-provider for an read-only access. he has also some interest in fixing the problem ...

/ Michael J. Stenitzer

ab is ApacheBench, a simple test client that sends many requests for the same page.

/.../ as far as i see, all our pages using queries are leaking.

Is a page just containing something like the following enough to trig the leak for you?

<html> <sqltable host="..." query="select * from test_table"/> </html>

That's essentially what I've tested so far and it works fine for me.

/ Martin Stjernholm, Roxen IS

Previous text:

2004-09-27 21:42: Subject: memory leak in oracle module

---

...

I now made a simple test case with a select query from an rxml page. Still doesn't leak, but if I push many concurrent requests it dumps core on me in OCIHandleFree called from oracle.c:497 (testing on Solaris 8 sparc). That's not good either, but it seems to be unrelated to the issue at hand. Still, what are your experiences with Oracle and concurrent queries?

we actually did not have problems so far. but i forward this question to peter holzer (who is probably not listening here) as i'm really no expert at this topic.

...

As for test case, I'd naturally prefer one that is as minimized as possible. What sort of operation causes the leak (connecting, selecting, insert, updates, transactions, etc)? Are concurrent threads involved? Any sort of closer characterstics are useful.

we are using only selects and alter session (i'v logged all queries and selected a few typical examples below). as far as i see, all our pages using queries are leaking.

...

Maybe you can make a replica of your server and use ab to find out which specific page causes the leak and thereby the queries involved?

sorry, i don't understand, what exactly mean. what is ab?

/michael

selection of typical queries:

select COMP_DTL_ID from comp_details where modul_comp_modul_comp_id='3639' and infcont_infcont_id='26' select DBMS_LOB.GETLENGTH(wert_clob) as LENGTH from comp_details where COMP_DTL_ID='20818' alter session set nls_date_format='YYYY-MM-DD' SELECT * FROM languages WHERE lang_code='de' SELECT * FROM mod_typen,infomodule WHERE INSTR ('iprojekt,publ', modul_code)>0 AND lang_lang_id=1 AND mod_typ_id=mod_typ_mod_typ_id SELECT TECH_VALUE, TECH_TYP_TECH_TYP_ID FROM technologien WHERE LANG_LANG_ID='1' AND tech_typ_tech_typ_id='19' SELECT kwb.KEYW_VALUE, count(kwb.KEYW_VALUE) KEYCOUNT, kwb.KEYW_TYP_KEYW_TYP_ID FROM keywords kwa, keywords kwb, keywords_modul_components, modul_components, tech_module, technologien, infomodule WHERE('1'=0 or kwb.LANG_LANG_ID='1') AND ('19'=0 or technologien.TECH_TYP_TECH_TYP_ID='19') AND ('19'=0 or kwa.TECH_TYP_TECH_TYP_ID='19') AND (0 IN (2,7) OR infomodule.mod_typ_mod_typ_id IN (2,7)) AND tech_module.modul_comp_modul_comp_id=modul_components.modul_comp_id AND tech_module.tech_tech_id=technologien.tech_id AND keywords_modul_components.keyw_keyw_id=kwa.keyw_id AND keywords_modul_components.modul_comp_modul_comp_id=modul_components.modul_comp_id AND infomodule.infmod_id=modul_components.infmod_infmod_id AND kwa.KEYW_TYP_KEYW_TYP_ID=kwb.KEYW_TYP_KEYW_TYP_ID GROUP BY kwb.KEYW_VALUE, kwb.KEYW_TYP_KEYW_TYP_ID SELECT DISTINCT COMP_DTL_ID, DBMS_LOB.SUBSTR(wert_clob, 2000) AS TITLE, MODUL_COMP_ID, COMPONENT_CODE, exp_date, mod_comp_cr_date FROM comp_details, infcont, v_keywords_module_components WHERE (0 IN (19) OR tech_typ_id IN (19)) AND lang_id IN (1) AND (0 IN (2,7) OR mod_typ_id IN (2,7)) AND (0 IN (340) OR keyw_typ_id IN (340)) AND (exp_date>='1900-01-01' or exp_date is null) AND _comp_modul_comp_id=modul_comp_id AND infcont_infcont_id=infcont_id AND sortby='1' ORDER BY mod_comp_cr_date DESC SELECT COMP_DTL_ID, DBMS_LOB.GETLENGTH(wert_clob) as LENGTH, DBMS_LOB.SUBSTR(wert_clob, 2000, 1) as STR FROM comp_details, infcont WHERE modul_comp_modul_comp_id='3250' AND infcont_infcont_id=infcont_id AND sortby='2'

/ Michael J. Stenitzer

sqltable host="energy" query="select * from technologien" --> leaking 133 kB for each request (44 rows)

This looks very much like my test query which doesn't leak. I wonder what the relevant difference might be. Maybe I need to get it working under linux.

/ Martin Stjernholm, Roxen IS

Previous text:

2004-09-27 22:59: Subject: memory leak in oracle module

---

...

ab is ApacheBench, a simple test client that sends many requests for the same page.

I c.

...

Is a page just containing something like the following enough to trig the leak for you?

<html> <sqltable host="..." query="select * from test_table"/> </html>

...

That's essentially what I've tested so far and it works fine for me.

i testing 50 request with ab from a remote server on the demo module containing:

sqlqeury query="alter session set nls_date_format='YYYY-MM-DD'" --> no leaking discovered

emit#sql query: select * from technologien" scope="sql" source="sql"

<insert scope="_" variables="full"/> </emit> --> leaking 133 kB for each request (44 rows)

sqltable host="energy" query="select * from technologien" --> leaking 133 kB for each request (44 rows)

emit#sql query: select COMP_DTL_ID from comp_details where modul_comp_modul_comp_id='3639' and infcont_infcont_id='26' --> no leaking discovered (after 50 requests, returning 1 row each, little data)

emit#sql query: SELECT * FROM mod_typen,infomodule WHERE INSTR ('iprojekt,publ', modul_code)>0 AND lang_lang_id=1 AND mod_typ_id=mod_typ_mod_typ_id -> leaking 4 kB for each request (2rows)

emit#sql query: SELECT kwb.KEYW_VALUE, count(kwb.KEYW_VALUE) KEYCOUNT, kwb.KEYW_TYP_KEYW_TYP_ID FROM keywords kwa, keywords kwb, keywords_modul_components, modul_components, tech_module, technologien, infomodule WHERE('1'=0 or kwb.LANG_LANG_ID='1') AND ('19'=0 or technologien.TECH_TYP_TECH_TYP_ID='19') AND ('19'=0 or kwa.TECH_TYP_TECH_TYP_ID='19') AND (0 IN (2,7) OR infomodule.mod_typ_mod_typ_id IN (2,7)) AND tech_module.modul_comp_modul_comp_id=modul_components.modul_comp_id AND tech_module.tech_tech_id=technologien.tech_id AND keywords_modul_components.keyw_keyw_id=kwa.keyw_id AND keywords_modul_components.modul_comp_modul_comp_id=modul_components.modul_comp_id AND infomodule.infmod_id=modul_components.infmod_infmod_id AND kwa.KEYW_TYP_KEYW_TYP_ID=kwb.KEYW_TYP_KEYW_TYP_ID GROUP BY kwb.KEYW_VALUE, kwb.KEYW_TYP_KEYW_TYP_ID -> no leaking (each request returned 12 rows)

/ Michael J. Stenitzer

Per said before (text 12361162) that he's using MySQL.

/ Jonas Walldén

Previous text:

2004-09-28 12:56: Subject: memory leak in oracle module

---

Per, can you disclose your hardware config to us? Also, you are not using Oracle, are you?

/ Marcus Agehall (PacketFront)

...

sqltable host="energy" query="select * from technologien" --> leaking 133 kB for each request (44 rows)

This looks very much like my test query which doesn't leak. I wonder what the relevant difference might be. Maybe I need to get it working under linux.

if of any help: we use linux 2.4.21 (suse 9.0) on our test server. but it leaked also on redhat 7.3 (linux 2.4.20)

the oracle client libraries are version Oracle8i Release 2 (8.1.6) (they worked before on other pike/roxen versions)

/ Michael J. Stenitzer

Previous text:

2004-09-28 12:51: Subject: memory leak in oracle module

---

...

sqltable host="energy" query="select * from technologien" --> leaking 133 kB for each request (44 rows)

This looks very much like my test query which doesn't leak. I wonder what the relevant difference might be. Maybe I need to get it working under linux.

/ Martin Stjernholm, Roxen IS

My experience with the Oracle libraries is that they SUCK. Mostly they suck if you try to use them with threads. You might want to try different version of the oracle client libraries, because I had different problems with different versions when I originally implemented the oracle module. (8.0.5.something worked really well for me.. :)

Apparenly Pike is the only application in existance which uses the dynamic callbacks in the oracle library, and that particular API doesn't seem to be tested before Oracle releases their software... :(

/ Fredrik (Naranek) Hubinette (pappa)

Previous text:

2004-09-27 18:25: Subject: memory leak in oracle module

---

I now made a simple test case with a select query from an rxml page. Still doesn't leak, but if I push many concurrent requests it dumps core on me in OCIHandleFree called from oracle.c:497 (testing on Solaris 8 sparc). That's not good either, but it seems to be unrelated to the issue at hand. Still, what are your experiences with Oracle and concurrent queries?

As for test case, I'd naturally prefer one that is as minimized as possible. What sort of operation causes the leak (connecting, selecting, insert, updates, transactions, etc)? Are concurrent threads involved? Any sort of closer characterstics are useful.

Maybe you can make a replica of your server and use ab to find out which specific page causes the leak and thereby the queries involved?

/ Martin Stjernholm, Roxen IS

Nope, sorry, I was confused. It seems to be the big_query compat wrapper that causes the leak somehow.

/ Martin Stjernholm, Roxen IS

Previous text:

2004-10-12 17:28: Subject: memory leak in oracle module

---

The direct cause of the leak is that OCIInitialize is called with OCI_THREADED in 7.4 and later. Are things likely to go bonkers if that flag is removed? Do you still remember why you added it? (In 7.2 that flag is used only if ORACLE_USE_THREADS isn't defined.)

/ Martin Stjernholm, Roxen IS

That does not really explain the leak I am seeing, though, since I am not using Oracle.

/ Per Hedbor ()

Previous text:

2004-10-12 18:59: Subject: memory leak in oracle module

---

The leak has been identified in get_global_error_handle().

/ Henrik Grubbström (Lysator)

great news. many thanx to martin and grubba. i'll have a test run this night.

/m

/ Michael J. Stenitzer

Previous text:

2004-10-12 19:15: Subject: memory leak in oracle module

---

Yes. It was trigged when date objects in the results are cast to strings. It's fixed now.

Thanks to Michael Stenitzer for providing a test case.

/ Martin Stjernholm, Roxen IS

I think I added it to avoid crashes.

/ Fredrik (Naranek) Hubinette (pappa)

Previous text:

2004-10-12 17:28: Subject: memory leak in oracle module

---

The direct cause of the leak is that OCIInitialize is called with OCI_THREADED in 7.4 and later. Are things likely to go bonkers if that flag is removed? Do you still remember why you added it? (In 7.2 that flag is used only if ORACLE_USE_THREADS isn't defined.)

/ Martin Stjernholm, Roxen IS