Description
Submitted by: Carlos Clicio (clicio)
Attachments:
trace.txt
tst_inc
I´m a long time firebird user with serveral production environments runnig Firebird 2.5.X . I´m extremally glad with Firebird..
When FB 3 was reeleased , i´ve got so exited with the great news available, and contiuned with the tests i´ve been doing since RC1.
It all was fine, until i´ve got some tests, with long time runnig batch proccess in realy great databeses / tables , with thousands of insert/updates per proccess.
Thats when my problem starts, after runnig some steps of the batch, the program abort abnormaly. I´ve started investigation and notices that the databese consumed ALL MEMORY AND SWAP , and run out of resorces , aboring as follows:
Fri May 13 19:29:46 2016
unable to allocate memory from operating system
After some time researching i have find a simple situation that reproduce the problem as follows.
I´ve noticed that my simple program that makes 2001 insertes, mixed with selects, makes that memory used by firebird process keeps growin , even after the program exits.
So I put it in a linux for loop to reproduce a long batch with several interactions,with different programs images
Even when i run a sweep in the process memory is not free.
Script to repeat the usage of the program :
for i in $(seq 100); do ./tst_inc localhost sysdba masterkey /home/firebird/database.gdb ; done
Command to monitor memory usage after the script above: ps auxw | grep firebird | grep -v grep
Result : firebird 4034 13.6 10.6 [[542176]] 431140 ? Sl 10:37 7:35 /opt/firebird/bin/firebird # final memory usage 540Mb after the loop
PS : the initial usage of the process is 90 Mb afer na isql in database.gdb;
PS2: I did the same with FB2.5 and there is NO PROBLEM
Program code:
// cat tst_inc.pas
uses sysutils,IBConnection,sqldb;
var
FSqlConnection : TIBConnection;
t : tsqltransaction;
Q,Qry : TSqlQuery;
id,i: integer;
function legenerator:integer;
var
Query : TSqlQuery;
begin
try
Query := TSqlQuery.create(nil);
Query.DataBase := FSqlConnection;
Query.Sql.text := 'SELECT GEN_ID(GEN_LOG_EVENTOS,1) FROM RDB$DATABASE';
Query.Open;
if not query.isEmpty then result := Query.Fields.Fields[0].AsInteger
else raise exception.create(' Nao conesguiu alocar generator');
Query.Close;
finally
Query.Free;
end;
end;
begin
FSqlConnection := TIBConnection.Create(nil);
FSqlConnection.HostName := paramstr(1);
FSqlConnection.UserName := paramstr(2);
FSqlConnection.Password := paramstr(3);
FSqlConnection.DatabaseName := paramstr(4);
t := tsqltransaction.create(nil);
FSqlConnection.transaction := t;
FSqlConnection.Connected := True;
t.Active := True;
Q := TSqlQuery.create(nil);
Q.DataBase := FSqlConnection;
q.sql.add('select * from x);
q.open;
for i := 1 to 2001 do
begin
Qry := TSqlQuery.create(nil);
Qry.DataBase := FSqlConnection;
Qry.sql.add('INSERT INTO T (C1,c2,c3,c4,c5,c6,c7,c8,c9,c10) values');
if paramstr(5) = '' then
Qry.sql.add('( ''cc'','+inttostr(legenerator)+',''...............................'',''dd'',''2016-01-01'',''.....................................'',''......................'',''.........................................'',''.................................'',''............................'' ) ')
else
Qry.sql.add('( ''cc'',0, ''...............................'',''dd'',''2016-01-01'',''.....................................'',''......................'',''.........................................'',''.................................'',''............................'' ) ');
Qry.ExecSql;
t.commit;
Qry.Close;
Qry.Free;
end;
q.close;
q.free;
t.active := false;
FSqlConnection.close;
end.
// to compile fpc tst_inc
PS2: if i remov either one or the other select the firebird process doesn´t keep growing.
Table structure :
CREATE GENERATOR GEN_LOG_EVENTOS;
create table x ( a varchar(10 ) );
create table t (
C1 VARCHAR(1000) Not Null,
C2 INTEGER Not Null,
C3 BLOB,
C4 VARCHAR(2000) Not Null,
C5 DATE Not Null,
C6 VARCHAR(1000) ,
C7 VARCHAR(1000) ,
C8 VARCHAR(1000) ,
C9 VARCHAR(1000) ,
C10 VARCHAR(1000) ,
C11 VARCHAR(10000)
);
delete from t;
select count (*) from t;
I´ll attach a fbmonitor output with 5 interactions