Nfs or ftp server

Server 1Server 2

ClinCapture

web appSAS script

SAS REST notification service[SAS RNS]

(can be run as standalone app)

upload / download operations

CC Database

1. SAS RNS is simply a jar that can be run as standalone app.

2. If we use DB between CC & SAS RNS then we will not lose file operations. Probably… we need one new table

HTTP Notification:

<< converted and ready to download

>> start converting from folder

How does it work

1. CC web app will have new export format "SAS dataset" When user starts the export to that format then CC generates ODM files, makes zip and uploads it to the shared disk/folder. On the CC side it works as typical dataset (CC stores info about latest started dataset job in the current session and maybe it will be better to change it to store info about all started dataset jobs...)

2. We will introduce new table sas_dataset in the CC DB in that process. These fields should be present:

sas_dataset_id dataset_id odm_folder -------------------- (folder name format: odm_[user_id]_[timestamp]) CC_status -------------------- (uploading, uploaded, downloading, downloaded) uploaded_datetime downloaded_datetime SAS_status -------------------- (converting, failed, ready) convert_start_datetime convert_end_datetime SAS_dataset_file_name When CC has generated a ODM dataset then it will change the CC_status to 'uploading’. When it's uploaded then CC changes the CC_status to ‘uploaded’.

3. On the SAS server we will run RNS (it can be standalone app that will be implemented as jar. In the same time I thought that we can use something like Mule ESB but it seems that it's not free already. Any ideas?)

RNS has direct connection with CC DB (both servers could be in the same internal network and Postgresql should accept the outside connections) and asks via scheduler every N seconds the fixed amount of records with CC_status == ‘uploaded’ and convert_start_datetime == null (period N and amount of records could be configurable)

We can use the convert_start_datetime field to exclude records if the current_time - convert_start_datetime < N minutes If the current_time - convert_start_datetime > N minutes then RNS sets the SAS_status to ‘failed’.

4. Then RNS calls for each record the OpenExport script (it gets as parameters the odm_folder, sas_dataset_id etc...) and then RNS changes the SAS_status to 'converting'

5. When the converting is done then the OpenExport should send the HTTP request to the RNS with parameters: SAS_dataset_file_name, sas_dataset_id, etc…

6. Then RNS sets: - SAS_status to 'ready' - converted_datetime - SAS_dataset_file_name

7. Further we can choose two ways:

THE WAY NUMBER 1:

a. On the CC side we will introduce new scheduler that asks every N seconds the fixed amount of records with CC_status == ready (period N and amount of records could be configurable)

Then CC starts downloading (probably as new 'SAS thread') and If we mount our share on the CC server as a drive then we can simply use a file copy util.

b. When downloading is done then SAS thread sends the email and sets the CC_status to 'downloaded'

c. + When user updates a CC page then he will see the message like: "your SAS dataset was genereated and it's available @link[here]".

THE WAY NUMBER 2:

a. RNS sends email to the user

b. + when user updates a CC page then he will see the message like (b/c SAS_status == 'ready'): "your SAS dataset was genereated and it's available @link[here]".

c. When user goes to the datasets page and clicks the 'SAS datset' link (before this link was not available) then CC starts downloading form the shared disk / folder (If we mount our share on the CC server as a drive then we can use the same downloading process as we have it already in the CC)

I like the way NUMBER 2.