E SYSTEMES

Intro - Speed Comparison - Download - History - Contact

FastBatch Lasso Tag

When you want to send large amount of emails with Lasso using the tag Email_Queue, you will see that the queueing process appears like a bottleneck.

SQLite is synchronous, so every times that you put an email in the queue with Email_Queue, SQLite will do a fsync (or system equivalent) to guarantee the integrity of the database. (see the SQLite doc)

The idea is to not do only one INSERT statement at the time, but put multiple INSERTs surrounded with BEGIN/COMMIT to reduce number of fsync. It will considerably increase the speed of the email queueing process.

fastBatch is a custom type to speed up emails batch sending - It requires Lasso 8.5.5 and greater.

BEGIN;
	INSERT ...;
	INSERT ...;
	INSERT ...;
COMMIT;

Here is how it works :

[

include: 'fastBatch.lasso' ;

var: 'Queue' = fastBatch() ;

// Assuming that $recipients is an array of recipient objects
iterate: $recipients, var: recipient ;

	local: 'msg_to_send' = (Email_Compose: 	
		-To=$recipient->email,
		-From='you@domain.com',
		-HTML='HTML Content',
		-Subject='Awesome !',
		-Body='txt body');

	// Adding the INSERT statement
	$Queue->(addStatement:#msg_to_send, 'returnpath@domain.com');
	
	local_remove: 'msg_to_send' ;

/iterate;

Email_Batch;
	var: 'error' = $queue->proceed; // Submit the INSERT statements
/Email_Batch;

if: ($error);
	// ohOh, Something is going wrong
else;
	// Ok
/if;

]

Email_Send VS fastBatch, ready .... fight !

Let's try a quick and dirty speed comparaison :

[

var: 'start' = Date_Msec ;
var: 'Queue' = fastBatch() ;

// that's a queueing test, we don't want to send those emails :
Global: '__SMTP_Queue_Status__' = 'Pause';
	
local: 'msg_to_send' = (Email_Compose: 	
	-To='me@domain.com',
	-From='you@domain.com',
	-HTML='HTML Content',
	-Subject='Awesome !',
	-Body='txt body');

	// Adding the INSERT statement
	
	Loop: 1000 ;
		$Queue->(addStatement:#msg_to_send, 'returnpath@domain.com');
	/Loop;
	
	// Queueing, returns queueing errors array, empty array if no error
	var: 'error' = $queue->proceed; 
	
'Time elapsed : '((Date_Msec)-$start);

]

Returns : Time elapsed : 4429 ms (yeah not bad :-] )

[

var: 'start' = Date_Msec ;
var: 'Queue' = fastBatch() ;

// that's a queueing test we don't want to send those emails :
Global: '__SMTP_Queue_Status__' = 'Pause';
	
local: 'msg_to_send' = (Email_Compose: 	
	-To='me@domain.com',
	-From='you@domain.com',
	-HTML='HTML Content',
	-Subject='Awesome !',
	-Body='txt body');
	
Loop: 1000 ;
	Email_Queue: 
	-Recipients=#msg_to_send->Recipients, 
	-Data=#msg_to_send->Data, 
	-From='bounce@domain.fr' ;
/Loop;
		
'Time elapsed : '((Date_Msec)-$start);

]

Returns : Time elapsed : 142985 ms

fastBatch is about 30 Times faster here!
Note: If you want to try this test don't forget to reset the Queue Status :

[ 
Global: '__SMTP_Queue_Status__' = 'Active'; 
]

Download

You can download the source of fastBatch here

Feedback and Bug Report

Feel Free to download the source and give us your feedback and/or bug report by using the form below or by email : . If you want report a bug please give us as much details as you can (Lasso version, OS, etc...)

Recent Changes :

2008 december 18 - V1.10b Adding retries handling if SQLite database is Locked during INSERTs submiting
2008 december 09 - V1.00b is online

Intro - Speed Comparison - Download - History - Contact


A propos de fastBatch en français dans le blog E SYSTEMES
Home Page - fastBatch Tag - Site E SYSTEMES website - Info éditeur / legal notices