Hi Tim

i can't use parallel programming of XE7 with EDB 2.
if i use in sub function ( isPrime ) edb component class, parallalel don't work more.


how i can manage it edb for tipical database procedure ?





function IsPrime (N: Integer): Boolean;
var
Test: Integer;
begin


Ho i can create EDB Table / Session here ?


IsPrime := True;
for Test := 2 to N - 1 do
  if (N mod Test) = 0 then
  begin
    IsPrime := False;
    break; {jump out of the for loop}
  end;
end;


 Tot := 0;
 Ticks := GetTickCount;

 TParallel.For(1, Max, procedure (I: Int64)
   begin
     if IsPrime (I) then
       InterlockedIncrement (Tot);
   end); // ,Pool);

 Ticks := GetTickCount - Ticks;

 Memo1.Lines.Add (Format (  'Parallel for: %d - %d', [Ticks, Tot]));

Mauro Botta,

When using a multi-threaded application (I assume that's what TParallel is doing), each thread needs to instantiate its own set of EDB components  (TEDBSession, TEDBDatabase, TEDBEngine, TEDBTable, TEDBQuery etc.) usually by instantiating a TDatabase for each thread and dispose of it when the thread is terminated. You cannot share any of these components between threads.

Barry

Mauro


I go along with what Barry says. I did a bit of googling and it looks as though the parallel processing is some work to make threading easier (one comment was that its based on the OmniThread library or is very similar to it).

If you hope that because Embarcadero's introduced this suddenly all the third party controls will become parallel then you're going to be disappointed

In database engine terms making it parallel would involve things like splitting a query across multiple processors and then integrating the results back together to a coherent and sensible whole. This is something that would need addressing on a component by component basis.

So to answer your question "how i can manage it edb for tipical database procedure ?"

Exactly as you do now. Create a thread, instantiate a TEDBSession and other objects needed and use them.

Roy Lambert

Check my source.

TParallel is slow, don't work fine.

if i remove the EDB component in IsRecordFound function, this function is 10x more fast of standard plain code.

EDB component create a bottleneck ? or TParallel is only for math / string base code ?




Result at Runtime, 64 bit, release:
------------------------------------------------------------
Plain for: 14344 - 17 User active: 0
Parallel for: 14250 - 17 User active: 0
------------------------------------------------------------


source:

unit ParallelFor_MainForm;

interface

uses
 SysUtils, Types, Classes, Variants, Graphics, Controls,
 Forms, Dialogs, StdCtrls, ExtCtrls, ComCtrls, Windows, Data.DB,

 System.Threading,
 edbcomps, edbxcomponents;

type
 TFormParallelFor = class(TForm)
   btnPlainForLoop: TButton;
   Memo1: TMemo;
   btnParallelForLoop: TButton;
   Label1: TLabel;
   EDBEngine: TEDBEngine;
   EDBDatabase: TEDBDatabase;
   procedure btnPlainForLoopClick(Sender: TObject);
   procedure btnParallelForLoopClick(Sender: TObject);
 private
   { Private declarations }
   function IsRecordFound (N: Integer): Boolean;
 public
   { Public declarations }
 end;

var
 FormParallelFor: TFormParallelFor;
 Pool: TThreadPool;
 iUser:Integer;

implementation

{$R *.dfm}


uses
 SyncObjs, System.Diagnostics;


{function local to the unit}
function TFormParallelFor.IsRecordFound(N: Integer): Boolean;
var
 tblTest:TedbxTable;
 EDBSession:TEDBSession;
 sWork:String;
begin

Inc(iUser);

EDBSession := TEDBSession.Create(Nil);
EDBSession.AutoSessionName := True;
EDBSession.LoginUser := 'aaaaaa';
EDBSession.LoginPassword := 'ppppp';
EDBSession.CharacterSet := csAnsi;
EDBSession.LocalConfigPath := 'D:\DelphiXE7\EDB';
EDBSession.LocalTempTablesPath := 'C:\';

EDBSession.Connected := True;

tblTest := TedbxTable.Create(Nil);
tblTest.DatabaseName := 'Planet';
tblTest.SessionName := EDBSession.SessionName;
tblTest.TableName := 'CUSTOMERS';
tblTest.IndexName := 'CODE';

tblTest.Open;

Result := tblTest.FindKey([N]);

sWork := '';
while tblTest.Eof = False do
     begin
     // Work...
     sWork := sWork+tblTest.FieldByName('DESCRIZ1').AsString;

     tblTest.next;
     end;

//
//  IsPrime := True;
//  for Test := 2 to N - 1 do
//    if (N mod Test) = 0 then
//    begin
//      IsPrime := False;
//      break; {jump out of the for loop}
//    end;

Dec(iUser);

tblTest.Close;
tblTest.Free;

EDBSession.Close;
EDBSession.Free;
end;

const
 Max = 30;
//  Max = 100000;

procedure TFormParallelFor.btnPlainForLoopClick(Sender: TObject);
var
 I, Tot: Integer;
 Ticks: Cardinal;
begin
iUser := 0;

Tot := 0;
Ticks := GetTickCount;
for I := 1 to Max do
   begin

   if IsRecordFound (I) then
      Inc (Tot);

   //Application.ProcessMessages;
   end;
Ticks := GetTickCount - Ticks;
Memo1.Lines.Add (Format ( 'Plain for: %d - %d', [Ticks, Tot])+' User active: '+IntToStr(iUser));
end;

procedure TFormParallelFor.btnParallelForLoopClick(Sender: TObject);
var
 Tot: Integer;
 Ticks: Cardinal;
begin
iUser := 0;

//if Pool = nil then
//   begin
//   Pool := TThreadPool.Create;
//   Pool.SetMaxWorkerThreads(10);
//   end;

Tot := 0;
Ticks := GetTickCount;
TParallel.For(1, Max, procedure (I: Int64)
 begin

 if IsRecordFound (I) then
    InterlockedIncrement (Tot);

 end); // ,Pool);
Ticks := GetTickCount - Ticks;
Memo1.Lines.Add (Format (  'Parallel for: %d - %d', [Ticks, Tot])+' User active: '+IntToStr(iUser));
end;

end.

Mauro Botta wrote:

> Check my source.
>
> TParallel is slow, don't work fine.
>
> if i remove the EDB component in IsRecordFound function, this
> function is 10x more fast of standard plain code.
>
> EDB component create a bottleneck ? or TParallel is only for math /
> string base code ?

The "processor cost" of creating a new session, opening files, and all
that is involved in setting up to look at a single row of data is very
high. It is plainly not worth doing all that to find the data.

You could use threads to do the task faster, using "worker threads" and
a "task pool", but that isn't something the new Delphi facility will
help with. It isn't good for this type of task where the setup and
closedown is much higher than the single transaction.

--

Matthew Jones

Mauro


>Check my source.

I can look at it but I can't compile or run it - I do not have, and will probably never have, XE7

>TParallel is slow, don't work fine.

This wouldn't surprise me. There is always an overhead to using parallel code, sometimes that overhead is greater than the gains.

>if i remove the EDB component in IsRecordFound function, this function is 10x more fast of standard plain code.

If you look at what you're doing for every loop through you're creating a session and logging in - its always going to be slower

>EDB component create a bottleneck ?

Yes ElevateDB is causing a bottleneck, but that's because of the way you're using it.

> or TParallel is only for math / string base code ?

No idea, but anywhere you have to ensure isolation is unlikely to work out of the box. You'll have to roll your own threads and manage them and the result. If you're hoping that by using TParallel somehow ElevateDB will be managed in a thread safe way, and without any problems I'm sorry to say you have no chance.

Roy Lambert

Roy Lambert wrote:

> Yes ElevateDB is causing a bottleneck, but that's because of the way
> you're using it.

We both agree, but I just want to emphasise that while ElevateDB is the
bottleneck here, that's not because it is ElevateDB. Any database
system would do the same, some a little better perhaps, some probably
very worse.

--

Matthew Jones

Having many tasks the creation of them can be thought of.

I reviewed the project with only 4 TASK, instead of 30.

The function performs work longer, but with less task contemporaries.

Benchmark:

Plain for: 13,750 - 4 User active: 0

Parallel TASK for: 10,390 - 4 User active: 0

Parallel FOR: 11,406 - 4 User active: 0


Parallel work better. ( task manager show more cpu work )



new source:



unit ParallelFor_MainForm;

interface

uses
 SysUtils, Types, Classes, Variants, Graphics, Controls,
 Forms, Dialogs, StdCtrls, ExtCtrls, ComCtrls, Windows, Data.DB,

 System.Threading,
 edbcomps, edbxcomponents;

type
 TFormParallelFor = class(TForm)
   btnPlainForLoop: TButton;
   Memo1: TMemo;
   btnParallelForLoop: TButton;
   Label1: TLabel;
   EDBEngine: TEDBEngine;
   EDBDatabase: TEDBDatabase;
   btnTASK: TButton;
   procedure btnPlainForLoopClick(Sender: TObject);
   procedure btnParallelForLoopClick(Sender: TObject);
   procedure btnTASKClick(Sender: TObject);
 private
   { Private declarations }
   function IsRecordFound (N: Integer): Boolean;
 public
   { Public declarations }
 end;

var
 FormParallelFor: TFormParallelFor;
 Pool: TThreadPool;
 iUser:Integer;

implementation

{$R *.dfm}


uses
 SyncObjs, System.Diagnostics;


{function local to the unit}
function TFormParallelFor.IsRecordFound(N: Integer): Boolean;
var
 tblTest:TedbxTable;
 EDBSession:TEDBSession;
 sWork:String;
 QQ:Integer;
begin
Inc(iUser);

EDBSession := TEDBSession.Create(Nil);
EDBSession.AutoSessionName := True;
EDBSession.LoginUser := 'aaaaaaa';
EDBSession.LoginPassword := 'ppppppppppppppp';
EDBSession.CharacterSet := csAnsi;
EDBSession.LocalConfigPath := 'D:\DelphiXE7\EDB';
EDBSession.LocalTempTablesPath := 'C:\';

EDBSession.Connected := True;

tblTest := TedbxTable.Create(Nil);
tblTest.DatabaseName := 'World';
tblTest.SessionName := EDBSession.SessionName;
tblTest.TableName := 'CUSTOMERS';
tblTest.IndexName := 'CODE';

tblTest.Open;

for qq := 1 to 30000 do                      //                          <<---
   begin
   Result := tblTest.FindKey([N]);

   sWork := '';
   while tblTest.Eof = False do
         begin
         // Work...
         sWork := sWork+tblTest.FieldByName('DESCRIZ1').AsString;

         tblTest.next;
         end;

   //
   //  IsPrime := True;
   //  for Test := 2 to N - 1 do
   //    if (N mod Test) = 0 then
   //    begin
   //      IsPrime := False;
   //      break; {jump out of the for loop}
   //    end;
   end;

Dec(iUser);

tblTest.Close;
tblTest.Free;

EDBSession.Close;
EDBSession.Free;
end;

const
Max = 4;

//  Max = 30;
//  Max = 100000;

procedure TFormParallelFor.btnPlainForLoopClick(Sender: TObject);
var
 I, Tot: Integer;
 Ticks: Cardinal;
begin
iUser := 0;

Tot := 0;
Ticks := GetTickCount;
for I := 1 to Max do
   begin

   if IsRecordFound (I) then
      Inc (Tot);

   //Application.ProcessMessages;
   end;
Ticks := GetTickCount - Ticks;
Memo1.Lines.Add (Format ( 'Plain for: %d - %d', [Ticks, Tot])+' User active: '+IntToStr(iUser));
end;

procedure TFormParallelFor.btnTASKClick(Sender: TObject);
var
 xx,Tot: Integer;
 Ticks: Cardinal;
 tasks: array of ITask;
begin
iUser := 0;

Setlength (tasks ,Max);

Tot := 0;
Ticks := GetTickCount;

//if Pool = nil then
//   begin
//   Pool := TThreadPool.Create;
//   Pool.SetMaxWorkerThreads(10);
//   end;


sDeb := '';
for xx := 1 to Max do
   begin
   tasks[xx-1] := TTask.Create (procedure ()
      begin
      if IsRecordFound(XX) then
         begin
         TInterlocked.Increment(Tot);
         end;
      end);
    tasks[xx-1].Start;
   end;



TTask.WaitForAll(tasks);

Ticks := GetTickCount - Ticks;

Memo1.Lines.Add (Format (  'Parallel TASK for: %d - %d', [Ticks, Tot])+' User active: '+IntToStr(iUser));
end;

procedure TFormParallelFor.btnParallelForLoopClick(Sender: TObject);
var
 Tot: Integer;
 Ticks: Cardinal;
begin
iUser := 0;

//if Pool = nil then
//   begin
//   Pool := TThreadPool.Create;
//   Pool.SetMaxWorkerThreads(10);
//   end;

Tot := 0;
Ticks := GetTickCount;

TParallel.For(1, Max, procedure (I: Int64)
 begin
 if IsRecordFound(I) then
    begin
    TInterlocked.Increment (Tot);
    end;

 end); // ,Pool);
Ticks := GetTickCount - Ticks;

Memo1.Lines.Add (Format (  'Parallel for: %d - %d', [Ticks, Tot])+' User active: '+IntToStr(iUser));
end;

end.

I'm using Delphi XE7 Update 1.

Matthew


+1 about 98 times

Roy Lambert