Scalable Smalltalk Short Message Center (SMSC)
The work on this software has been sponsored by On-Waves ehf. Commercial support for deployment, maintenance and extensions is available through moiji-mobile. Please contact [email protected].
Design Goals
-
Receive to be delivered SMS by SMPP from multiple connections.
-
Store DeliverSM, SubmitSM or MAP T-PDU (Submit or Deliver)
-
Be able to have multiple delivery processes.
Design Ideas
Overview
Storage
The central storage is planned to be a MongoDB. The reason is that it supports tailable cursors on capped collections, replication, atomic update of a document and the write concern for replication.
SMS/SMPP Record
One record will be a single message with meta information. Most of the information will be extracted for some queries. Most notable the following fields need to be there:
-
Arrival time at the SMSC
-
Destination number
-
Priority
-
Sender Report needed
-
Expiration time
-
Retry counter
-
Scheduled re-transmit time
-
Locking information (state + time)
-
Payload being either SMPP DeliverSM, SubmitSM or MAP T-PDU
Event structure
Create small capped collection for notification entries. This allows one to use a tailable mongo cursor that blocks until an entry has been inserted into the collection.
SMPP Inserter
Receive SMPP SubmitSM/DeliverSM and store in the database for store and forward. Multiple inserters must be able to run in parallel.
-
Use existing Smalltalk SMPP framework to open or listen to connections
-
After the content has been inserted add an event (which will lead to the delivery agents waking up)
Delivery
The task is to wake up for the next delivery time or when a new SMS has been inserted into the table. By using a capped collection for the notifcation collection the cursor will become readable and the system can wake up and select the message to be delivered. With a system of multiple delivery nodes running, all of them will wake up but only one will be able to lock the SMPP record with a CAS and process with the delivery process.
Once a SMS has been picked it will be checked if there are more SMS for this subscriber and up to four for messages will be blocked and scheduled in the same delivery run. If the delivery of a message fails the next delivery time will be set and the delivery worker goes back to waiting for work.
In general we want to avoid two delivery nodes to deliver to the same subscriber. This might result in a more complicated CAS locking scheme. The first lock would be to declare interest to deliver to the user and second lock would increase the lock by checking that no other delivery node has reached that level. Once that lock level has been reached it can try to find up to four different messages for delivery on the same TCAP session.
We might decide to put the delivery history (attempts, timestamps, info) into the record as well to help with debugging.
On delivery the encoded data might need to be converted. This can lead to conversion from any format to any other format. This might resolve in SMPP to GSM and Deliver to Submit.
Clean-up
As the locking process might leave some locked SMS behind and we need to remove SMS that were never delivered there should be a clean-up task with the following responsibilities:
-
Remove old SMS that could not be delivered
-
Reset SMS that are in the pending state for too long
Database locking protocol
There is one collection for "sms" and one for the "locks". Each SMS is one entry in the "sms" collection and for each destination MSISDN there is either one or no entry in the "locks" collection.
A worker will use Compare-And-Swap to lock one entry that is to be delivered. It is possible that two workers lock two SMS to the same user. This means as the next step both workers will try to get a destination lock. This is by having a unique index per destination MSISDN in the "locks" collection and then using CAS to try to lock the destination.
The system that could not lock the destination will need to release the lock and it is done using a CAS as well.
The system that got the lock and will now continue. It will try to lock more SMS using the CAS approach to lock matching SMS and ignore the current lock setting. The SMS will be either removed or unlocked depending on the delivery result. The lock from the other system will either be used after it has been unlocked or it will be stolen.
A third system might try to do delivery and needs to avoid always locking a SMS that is currently been delivered to. This is done by maintaining a list of excluded destMSISDN in the current round.