Charter of the EPICS V4 Working Group

2nd Charter of the EPICS V4 Working Group, for Oct 2012 to Oct 2013

This version:
charter_20130428.html
Latest version:
charter.html
Previous version:
charter_20120217.html
Editors:
Greg White, SLAC, PSI
Bob Dalesio, Brookhaven Lab

Abstract

EPICS is a set of Open Source software tools, libraries and applications developed collaboratively and used worldwide to create distributed soft real-time control systems for scientific instruments such as a particle accelerators, telescopes and other large scientific experiments.

EPICS Version 4 (V4), is intended to be the next major version of EPICS. The EPICS V4 Working Group is a collaborative effort of members invited by Brookhaven Lab, to bring EPICS V4 to its full potential.

This Charter is a statement of the basis for the work of the EPICS V4 working group, the intended outcomes for that work, its deliverables and success criteria. It also outlines some administrative matters of the organization of the group, and its working practices.

This version of the Charter is specifically for the second year of operations of the Working Group, covering approximately the year October 2012 to October 2013.

For more information about the EPICS, please refer to the home page of the Experimental Physics and Industrial Control System.

Status of this Document

This is the 15-Jan-2013 revision of the Charter, that covers the period 2012-2013. This version has not yet been approved by the working group.

The terms MUST, MUST NOT, SHOULD, SHOULD NOT, REQUIRED, and MAY when highlighted (through style sheets, and in uppercase in the source) are used in accordance with RFC 2119 [RFC2119]. The term NOT REQUIRED (not defined in RFC 2119) indicates exemption.

Table of Contents

Introduction

EPICS version 3 and its predecessors have been enormously successful as a framework for distributed device control, but have not been optimal for all purposes. Firstly, the underlying network protocol has not been documented and standardized, and so is not suitable for independent implementation or extension. Secondly, the network codec and message passing system has not supported passing semi-structured data, nor arguments, and so has not been suitable for use as a transport for modern needs of experimental physics detectors or host level scientific applications.

The EPICS V4 Working Group was chartered last year to define and publish an open control protocol for control system endpoints, which functionally extended channel access to suitability for both device control and high level services. It included data types suitable for high level services, such as tables and complete image data such as from gated photon detectors.

In the past year, EPICS V4 successfully brought a communications protocol for controls, "pvAccess", to advanced public review status, and a software framework suitable for high performance distributed control, message passing, and high level software services, to beta release status.

Mission

This year the Mission of the EPICS V4 working group is to integrate with the broader EPICS toolkit. Further, it is to extend EPICS into the application level of scientific instrument controls and optimization, and into the high performance computing data pipeline of backend data processing. It is intended that high performance scientific applications, particualrly those that use or processes instrument data, should be made easier to develop and to effectively deploy.

Basis for Work

The EPICS V4 working group is supported by Brookhaven Lab (BNL), Paul Scherrer Institute (PSI) and Diamond Light Source, and collaborating organizations, subject to their following mission, goals, and expected deliverables, of this Charter, and following the EPICS V4 Process as the work method. Conformance to this Charter is required for participation in the working group.

Basic Scope and Goals

Presently, in an EPICS installation it is expected that control and module support would be done with EPICS V3 IOCs, while middle layer and "Service Oriented Architecture" based operations would be done with EPICS V4. V4 includes some support for protocol interoperability with V3. In short, EPICS V4 builds on EPICS V3.

This year the initial goal of the EPICS V4 Working group is expected to bring fully integrate EPICS V3 and V4 technologies, for seamless interoperation.

Scope for this year's charter is specifically extended to the IOC level, where we will complete integration with vxWorks and interoperation with the classic EPICS IOC.

We intend to improve the EPICS IOC's suitability and performance as a scientific instrument data processing pipeline, specifically for fast detector image data, and to greatly improve facilities for clients to deal with these data.

Wheres last year, Java, C or C++, reference bindings must have been supplied for all implementations of each normative standard document; this year (only) the reference bindings of the client side of EPICS V4 tools must be supplied in both C++ and Java. IOC work may be C++ only.

Note that, as last year, our primary focus is on definition of standard behavior as normative documents of the Working Group, informed by the development of software which implements the behavior. As last year, we shall supply reference implementations for these standards. It is to be expected that in some respects, reference implementations may trail the standard.

Goals

The goals of the Working Group are the following

  1. Begin development of the EPICS IOC, as a high performance instrument data processing pipeline suitable for detector data processing and communications.
  2. Development of a pvAccess server for the IOC which enables IOC channel data to be collectively requested and returned as coherent data package to clients. For instance, so a client can make one request for the pertinent data of a beam position monitor (BPM), and get all and only the x, y, current, and status at that BPM.
  3. Further development of pvAccess, concentrating on high performance and communication mechanisms other than TCP/IP (multicast in particular).
  4. Further development of pvData, concentrating on high performance, particularly by zero-copy and block data transfer enhancements. pvData is the open, documented, data type encoder/decoder capable of exchanging both atomic and semi-structured data, suitable as the data exchange format for high end diagnostics and services.
  5. Interoperability with EPICS v3. The pvAccess message protocol must be interoperable with EPICS IOCs, and add the Channel Access provider to pvAccess client side. See Interoperability with EPICS v3 below
  6. Development of data aggregation services; completion of the EPICS V4 core services Directory Service and extend the Gather service for connection time domain rebasing (a server side pvManager). These are to enable scientific application programs to deal simply with many channels and channels whose update rates are different to their client consumption rates.
  7. High Performance. The above protocols and controller framework must be of high performance. See Performance goals below
  8. Much easier for developers to get started. The above protocols and framework must be transparent, documented, and delivered so that it is very easy for a developer to create EPICS V4 IOC applications and services and interop pretty easily with "V3". Note that we do not yet intend to support easy adoption by EPICS users such as database writers and screen developers
  9. Implementability. The above protocols, and the processing of message I/O, must be documented in such as way that outside software developers can easily create an interoperable implementation
  10. Error handling. All normative software implementations must handle and log all errors in such a way that distributed users can see both the causes and outcomes of all system errors (scientific and control issues, as well as detected program exceptions). In practice, this may mean the creation and adoption of a single error messaging standard, perhaps an EPICS V4 service whose job it is to receive and log errors from all other endpoints.

It is anticipated that one further goal, which may be stated as production of a software framework for scientific services, may be undertaken either as part of the EPICS V4 working group, or by an associated working group operating within the same EPICS V4 Process as the EPICS V4 Working Group.

Interoperability with EPICS V3

EPICS V4 clients must be interoperable with EPICS v3 IOCs. This interoperability must be both interlayer (IOC ⇔ client) and intralayer (V3 IOC ⇔ "V4 IOC"). PvAccess must run in version 3 IOCs and serve all v3 type structures. dbLinks will be adapted to interoperate with pvaccess accessible data.

Add ability to get or set V3 channels (pvaSrv) synchronously (so long as they are all in the same lockset) as a result of 1 V4 channel operation.

This mechanism must satisfy the expected use case of device control including EPICS V4 in the medium term; in which device control will continue to be through EPICS v3, with control wire protocols and client sides in EPICS V4. It is expected that in the longer term, past the duration of this charter, the EPICS V4 IOC will be properly extended into device control.

Interoperability within EPICS V4

Each normative standard must be demonstrated to be interoperable with adjacent layers in the EPICS v4 protocol stack by production of, and testing, at least one implementation, in each of C++ and Java, from device to client (end to end).

Specific requirements for Pipelining

  1. loss-less, high performance transport of detector and camera images

See others from David

Performance Goals

The following minimum performance goals are unchanged from last years charter. This year we intend to concentrate on certifying their achievement particularly for streams over network connections:

Out of Scope

The following are out of scope for the group specifically:

  1. The production of the new Input/Output Controller (IOC) framework, suitable for both controls and services, known as "pvIOC", will not be pursued as a software deliverable this year. Rather, requirements and design references for such a new IOC will be pursued, with a view to development next year
  2. Both and only Java and C/C++ bindings are required for reference implementations of the core protocols (pvAccess and pvData). One or the other language binding is required for service implementations
  3. The Windows server side is out of scope for now. A Windows client side may be developed. Windows server side may be added in a charter revision subject to resources and specifically funded requirement (lookin at you PSI).
  4. Development of specific production quality services are out of scope. A few reference services may be produced, but they are only for illustration and offered as a basis for extensions.

This statement of Out-of-Scope has been changed to last year's to omit the ban on development of a specific application; this year we will concentrate on the specific application of high performance image pipelining.

Success Criteria

Derived from the goals above, success criteria are at least:

  1. Adoption or planned adoption of EPICS V4 in the EPICS community
  2. Demonstrated interoperability of EPICS V4 clients and EPICS V4 services
  3. Demonstrated interoperability with EPICS v3
  4. Demonstrated high performance
  5. Demonstrated easy to implement an EPICS V4 service, taking arguments, and returning semi-structured data
  6. Demonstrated easy to implement an EPICS V4 client, taking data from EPICS v3 records, packaged by an EPICS V4 service (pvaSrv)
  7. Demonstrated easy for an end user to get a list of all PVs and named entities in an EPICS network, through an EPICS client, given a name pattern
  8. What else?

Deliverables and Duration

The outputs of the working group will be delivered as a system of documented standards, plus reference implementation source code where relevant. These documents and source codes will all be available from the EPICS V4 web site, http://epics-pvdata.sourceforge.net/.

Deliverables

The group is expected to produce the following normative deliverables WG to agree:

  1. Performance measurement and micro-benchmark test framework for pvAccess, including documentation, made publicly available. This must run on both host level and embedded computers. This should be used to measure, and optimize performance per the performance goal
  2. An image processing pipeline outline design framework, particularly oriented towards encapsulating the semantics of asynRecord and asynDriver. At the end of the year this must also be documented as implemented.
  3. Create proven vxWorks cross-compile ports of pvAccess, pvData and associated code sets (pvCommon, [where is pvaSrv?]). Track and publish the proven host compilation, cross-compilation, and runtime experience of at least the most recent public release tar, and the tips of the source code repos (when those are thought to be consistent and should work)
  4. Support for Channel Access client in pvAccessCPP (it already exists in pvAccessJava), together with a suitable mechanism for dealing with name collision
  5. Further develop the IOC module that acts as the V3 Ca to pvAccess clients. This module is now called pvaSrv, to include at least the following:
    1. pvAccess client get or put or monitor, a single ca channel
    2. pvAccess client get or put or monitor, a single pva channel which corresponds to a set of ca channels where the members of the set is defined at runtime prior to the get call
    3. pvAccess client get or put or monitor, a single pva channel which corresponds to a set of ca channels where the members of the set has been defined by a prior configuration
    Precise semantics in the case of locksets, status and alarms, particularly in the case of the I/O operations involving more than on ca channel, must be defined and clearly started in documentation
  6. Completion of the easy to use pvAccess API, now called easyPVA, for both Java and C++ bindings. Publish matlab examples
  7. Completion to at least 2PWD of the normative document of the EPICS V4 interoperable data types. These data types must be universally understood by every client and service which claims EPICS V4 compatibility. The requirement for this deliverable is distinct from the pvData document deliverable, since pvData can encode any type, this deliverable recommends the confined set of data objects that will be used by EPICS V4 interoperable services. In particular this document must clarify the definition of types with respect to images and areaDetector - decide on whether we use a general image metatype, an area detector type, or both
  8. A directory service accessible through the EPICS V4 API itself, from which can be found at least both CA and PVA channel and entity names, and associated service names
  9. Completion of support an use of NTURI
  10. A report of interoperability of the EPICS V4 IOC with EPICS V3 record processing
  11. Write and publish to EPICS community, a performance report, comparing EPICS v3 to EPICS V4 for some common EPICS v3 control and read tasks. Comparisons to other common high performance data interconnects should be made, eg ICE, ASN.1, EXI Web Service, though are to specific deliverables
  12. Extend the "Getting Started" document to how one would start programming the use of EPICS V4
  13. synoptic displays: pvManager. Complete support for all of stably defined normative types in pvManager. Begin normative type support in caQtDm.
  14. pvIOCCPP. Develop a discussion document for the requirements and design of the V4 IOC. Don't attempt an actual non-alpha release! Instead take time to define requirements for the pv processing, pvCopy, support, and the local pvAccess server.

Duration

The expiration date of this charter is 1-Oct-2013.

Expected Milestones

To be added.

Coordination with Other Bodies

The Working Group should coordinate and align objectives to satisfy the requirements of the following bodies:

BNL NSLS Controls
[TBD:]
PSI SwissFEL
PSI requires EPICS V4 services framework, for online accelerator modelling in particular, and interoperability of the EPICS V4 based services with EPICS v3 IOCs and their associated record support.
SLAC FEL accelerators (LCLS/LCLS-II)
SLAC requires EPICS V4 services framework in general, and for online accelerator modelling in particular.
Control System Studio (CSS)
Periodically joint meetings of the EPICS V4 WG and CSS developers.
"Database" group
Periodically joint meetings of the EPICS V4 WG and BNL database applications group [needs work - who are those guys, what are they called?], [need to say why]
Scientific Applications Working Group, if formed
It may be that in this year a working group for accelerator physics services and applications will be formed. If it is, it is expected that the EPICS V4 WG will work closely with that group.

Group Participation

Effective participation is expected to consume one to two working days per week for each group participant (see below); two days per week for editors of a specific normative document. Developers of implementations of a normative standard (such as pvAccess, pvData, services) are expected to consume three days per week. The Chair shall ensure that the criteria for Good Standing are understood and followed:

Participant Status
Prospective group participants should ask their primary EPICS contact or manager to send an email to one of the EPICS V4 chairs, showing that they understand the participant will be involved with EPICS V4 and the commitment level expected
Observer Status
Individuals who are not active at the full participation level, may join meetings of the group as observers. Email the chairs to get details of how to join the meetings.

The group is expected to have at least 4 active participants to be successful.

Good Standing

A participant who is absent from more than 3 meetings consecutively, or in the opinion of the chairs is not acting on Action Items or their responsibilities for group deliverables, will be deemed out of good standing, and will no longer participate.

Chair

The initial chairs of this Group are Greg White (PSI, SLAC) and Bob Dalesio (BNL).

Meetings

The Group will have distributed meetings, one to two hours every week, and face–to–face meetings, every three or four months.

Communication

The Working Group will utilize the mailing list, epics-pvdata-devel@lists.sourceforge.net. All except the most trivial messages should be cc'ed to this mailing list. The web archive of it used should be used as backward references in communications.

Confidentiality

The proceedings of this Working Group are open to all (not just members of the working group), subject to exceptions made by the Chairs, after consultation with the Working Group. The group's communications on the web archive will be world readable. Posting to epics-pvdata-devel@lists.sourceforge.net will be unrestricted for members, but moderated for non-members.

Patent Policy

This Working Group operates under EPICS Open License.

Glossary

Host Level
Host Level describes the category of computer systems including daemons, services, user level GUIs and other front-end programs, in the computer archiecture of a large distributed computer system, such as an accelerator control system. Host level is in distinction to field level. In the client-server paradigm, host level programs are largely clients of field level systems.
Field Level
Field Level describes the category of computer systems including front-end-controllers and network enabled diagnostic equipement in the computer archiecture of a large distributed computer system, such as an accelerator control system. In EPICS the field level corresponds to IOCs and instrument hardware. These systems are "in the field". In the client-server paradigm, field level systems are largely servers of host level systems.
V4 IOC
A "classic" IOC that is connected to pvaccess, such as one that includes pvaSrv. Note that a V4 IOC in this definition is very different from what may formerly have been understood as a V4 IOC, being an instance of an IOC running pvIOC. It is envisaged that an IOC running pvIOC may be called something else now that V4 is beign targetted to the classic EPICS IOC runtime.
Greg White, EPICS V4 group co-chair
Bob Dalesio, BNL Controls Group leader, and EPICS V4 co-chair
Last modified $ Date: 2011/09/15 17:21:49 $