EPICSv4 Overview and Architectures

EPICS V4 Overview and Architectures, Editors' Draft, 22-Jan-2013

Editors:
James Rowland, Diamond

Abstract

EPICS V4 is a software platform for high performance control systems and data services. This document summarises the components of EPICS V4, and outlines some software architectures that may be employed using it. It includes such things as interoperation with EPICS V3, services for aggregating the data of many Input Output Controllers, connection to online data sources such as databases, and facilities for building Service Oriented Architectures.

Table of Contents


Introduction

EPICSv4 extends the scope of EPICSv3 with structured data types and request/response messaging patterns.

EPICSv3 systems consist of I/O controllers (IOCs) connected to temperatures, magnet currents, beam positions, and other scalars and vectors of timestamped primitive types. Above this layer are objects such as response matrices, machine lattice descriptions and backup and restore groups that are not easily modelled in the limited range of data types available in EPICSv3. Facilities use tools such as Matlab and SDDS to model high level concepts. Similarly many high-level services are request/response, returning the result of some processing or retrieval in response to a query. Tools such as Corba or Web Services fill this role, as EPICSv3 only has 'put with callback', a response that indicates that processing has completed but carries no payload.

EPICSv4 supports Java, C++ and Python clients and servers on Windows, Linux and OSX. vxWorks portability is being tested.

The Getting Started page describes the build system.

Overview of EPICSv4

Type System and Serialization (pvData)

pvData is a run-time type system with serialization and introspection. A similar product is Google Protocol Buffers (PB), an important difference is that pvData always has the full type information (including field names) present when messages are encoded or decoded, while PB depends on precompiled stubs on each side.

pvData C++ Reference Documentation

pvData Java Reference Documentation

Network Procotol (pvAccess)

See also pvAccess Protocol Specification (Technical Document)

pvAccess C++ Reference Documentation

pvAccess Java Reference Documentation

The EPICSv4 network protocol is pvAccess. pvAccess is a TCP protocol, a connection to a named endpoint is a Channel. Channel endpoints have unique names within a facility, and the default name resolution method is UDP broadcast prior to connection. Channels are typed with the type determined by the server.

pvAccess supports the following messaging patterns:

I/O Controller (pvIOC)

The IOC combines pvData structures with processing behaviour to form records. A domain-specific configuration language allows these records to be connected in a dataflow style to form a database, allowing customization of behaviour without systems programming.

pvAccess C++ Reference Documentation

pvAccess Java Reference Documentation

Normative Types

To allow generic clients and prevent the unbounded spread of application-specific types, the EPICSv4 Working Group specifies a set of (about 10) Normative Types (Document) for common use cases. Examples are timestamped control data, relational database tables and multi-dimensional images. Generic tools will operate on Normative Types and it is expected that facilities will use Normative Types for the majority of applications.

Standard Components and Nominal Architectures

This section demonstrates the use of EPICSv4 with components that are found in many facilities. It is typical for each facility, and often each service within a facility, to use their own network protocol and data model.

Hello World

The most basic architecture is single client, single server, using RPC. The RPC server does not contain an EPICSv4 database, the developer implements the processing in C++, Java or Python code.

The helloWorld example is located in the exampleJava project, currently it is not possible to write an RPC server in Java without the IOC database but this will be added.

hello world diagram

Process Control / SCADA

EPICSv4 supports all EPICSv3 messaging patterns including publish/subscribe via Monitors.

ioc diagram

EPICSv3 Integration (pvaSrv)

Currently EPICSv4 focusses on high-level applications. It is expected that facilities will continue to use EPICSv3 IOCs for I/O until drivers are available for EPICSv4 IOCs. pvaSrv is a pvAccess server that runs on an EPICSv3 IOC, allowing EPICSv4 clients to communicate with EPICSv3 process variables.

The pvaSrv example is bundled with pvaSrv.

Startup Script for pvaSrv Example

pvaSrv Example Makefile

The last working version of the pvaSrv example is on the default branch of pvaSrv. It compiles against the 2.0-BETA releases of pvDataCPP, pvAccessCPP, and pvIOCCPP. These are the combination of changesets you need:

git clone -b 2.0-BETA https://github.com/epics-base/pvIOCCPP.git
git clone -b 2.0-BETA https://github.com/epics-base/pvDataCPP.git
git clone -b 2.0-BETA https://github.com/epics-base/pvAccessCPP.git

The pvaSrv documentation explains the 3 simple steps needed to install the pvAccess server in your V3 IOC.

v3 integration diagram

Client Stream Query (pvManager)

pvManager is a client utility library for EPICSv3 and EPICSv4 that performs rate reduction and averaging on data streams. This abstration layer makes writing robuts clients easier, especially GUIs. pvmanager diagram

Operator Interfaces (OPI)

The proposed client tool for OPI is Control System Studio (CSS) BOY. The BOY widgets will be updated to use pvManager as their data source. opi diagram

A limited functionality proof of concept using the org.csstudio.data API directly to make a CSS EPICSv4 connector is here.

Service Aggregation (Gather)

Many facilites need to concatenate and summarize information from many sensors into a single location, for example the electron beam position in a synchrotron is presented to the user as a vector but is read from many devices throughout the facility (see Diamond BPM Concentrator). The Gather service is a standard implementation of this pattern.

The Gather service combines values from multiple sources into a single point. The Gather service can also support EPICSv3 Channel Access. This is so that Gather can be used without upgrading existing EPICSv3 servers with v3Channel. Gather can be used as a software component or a service.

Gather used as a software component: gather diagram Gather used as a service component (details omitted): gather service diagram

Gather Implementation

Gather is ready for testing. The server is implemented in Java. The client is implemented in Java, C++, and Python. It is expected that the way gather currently presents data to the client will change but until it has been used by real applications it is not clear what to do.

The current sources are avaliable via the following mercurial repositories:

Java
github.com/epics-base/pvServiceJava
git web
C++ and Python
github.com/epics-base/pvServiceJava
git web

The latest documentation is:

Java
gather Java
C++
gather C++
Python
gather Python

The MASR service contains its own gather implementation, see the code here that will soon be merged with the other gather libraries.

Model Service

The Model Service returns information from the machine model, such as the linear optics matrices of a particle accelerator. It may return pre-calculated values or connect to a simulation code. model service diagram

Relational Database Service

Most facilites need to retrieve information from relational database tables. The RDB Service provides a database-independent way to perform canned queries over EPICSv4. The mapping from a query name to an SQL query string can be stored in the Directory Service (discussed later) or in the RDB itself.

rdbService is located in the exampleJava project.

rdb service diagram

Name Resolution Server

EPICSv3 includes a little-used name server that can respond to UDP name requests with the target IP address and TCP port of the server, replacing the UDP broadcast name resolution. It would be possible to have a Name Resolution Server of this type in EPICSv4.

Machine Physics and Scripting

Matlab Middlelayer is an interactive scripting environment used for machine commissioning. The EPICSv4 Java client connects Matlab to high-level services and process data. The Normative Types provide data in a form suitable for manipulation without further processing, such as timestamps in Matlab date format, and tables as structures of arrays. mml diagram

Save and Restore

Process data setpoints must be saved to permanent storage and restored to reset the facility to a known working state. MASAR is the EPICSv4 Save and Restore tool. It combines RPC, Service Aggregation (Gather), and RDB access. An RPC client initiates the save or restore request, the MASAR server Gathers the process variables and stores them in the RDB.

MASR Project Reposotory

masr diagram

Time-Series Database

EPICS archivers are a familiy of time-series databases. The EPICS Channel Archiver (RTree) is a chunked array store with an RTree index and a C++ API, libStorage. The EPICSv4 archive service provides a pvAccess RPC interface to these files, supporting range queries with on-the-fly decimation including statistics (mean/min/max) into a table structure. This replaces the XMLRPC interface distributed with the EPICS Channel Archiver. archiver diagram

ChannelArchiverService Project Repository (decimation is not supported yet)

Data Acquisition

X-Ray detectors produce image frames annotated with experimental metadata. 10 Gigabit Ethernet is increasingly used to transport these images from the acquisition machine to a storage server. Protocols for this include network or distributed file systems such as NFS or Lustre, or custom TCP protocols. Each custom TCP protocol must deal with connection handing, error handing and serialization. Data is stored in HDF5 files. pvAcces provides an efficient TCP transport for structured image types in conjunction with areaDetector.

Contact Ulrich Pedersen at Diamond for more information about acqusition to HDF5 over 10gbE

detector diagram

Service Oriented Architectures (SOA) and Directory Services

EPICSv4 services are self-describing through introspection of channel endpoints. EPICSv4 services are also discoverable via broadcast name resolution of globally unique channel endpoint names. Further discoverability can be provided by the Directory Service. The EPICSv4 directory service is ChannelFinder.

ChannelFinder returns lists of Channel names and properties associated with a particular key. This can be used to answer queries such as "return a list of channels endpoint names that are connected to this magnet". Other high-level services such as Save and Restore may depend on the information in the Directory Service. The Directory Service may store information in a relational database, but the interface exposes the associations between items in a typical facility in a standard way.

ChannelFinder is a REST web service, and was designed to be used in EPICSv3 and EPICSv4 installations. The REST interface provides additional authentication functionality, but a pvAccess interface to ChannelFinder is under development to facilitate integration with EPICSv4.

The ability to populate the Directory Service from an IRMIS database is also under development but using IRMIS is not mandated. directory services diagram