Pipeline 1 PipeOnline Architecture High Level Overview

Pipeline 1 PipeOnline Architecture High Level Overview

Menu

The system is implemented as a distributed Java web application, following a multi-tiered design composed mainly of a data tier, a logic tier and a presentation tier, based on technologies that are part of ​Java EE (​JPA, ​Servlets, ​JSP). The 3 tiers are described below:

Data Tier

Most of the data manipulated by the system (e.g. user accounts, job queues) will be persisted using a database management system. The current prototype uses a ​relational database (​MySQL), but it could be decided to use an ​XML database (e.g. ​eXist).

The data stored in the database is manipulated using the ​Java Persistence API (JPA). The current prototype uses ​TopLink Essentials as the JPA implementation but other options should be investigated such as the ​EclipseLink project (open source port of Oracle TopLink and the JPA 2.0 reference implementation).

For some system settings, simple configuration files (Java Properties, XML) can be preferred over database for convenience. Standard Java APIs such as ​Java Properties API, ​JAXP (SAX/DOM/StAX) are used to access these files.

Logic Tier

This tier is basically responsible for manipulating the data tier based on user inputs from the presentation tier. This includes the following actions:

  • User management (creation, settings, login)
  • Creation of new jobs (added to the queue or for instant execution)
  • Execution of jobs (by polling the queue or instantly)
  • Extraction and storage of statistics

The logic tier is implemented in plain Java objects, using standard Java APIs and the DAISY Pipeline APIs.

The current prototype launches an external daemon to regularly sniff the queues in the database. When a job is identified and is ready to be executed, the daemon will execute the job with the DAISY Pipeline in a new JVM instance.

Presentation Tier

The presentation tier defines a web-oriented user interface. The initial implementation features a set of web pages for the end user UI, one or several pages will be added afterwards for the admin UI. Additionally, a future version may feature a rich desktop client.

The web UI is mostly based on the ​Java Servlets and ​Java Sever Pages (JSP) technologies. The application is deployed as a WAR (Java Web Archive) on a servlet container such as ​Tomcat or ​Jetty.

Tags: Pipeline 1