Charter of the EPICS V4 Working Group

Abstract

EPICS V4 is a set of computer communications protocols, and a software framework, for high performance distributed control, message passing, and high level software services, as may be used in large scientific instruments and industrial plants. The EPICS V4 working group is a collaborative effort of members invited by Brookhaven Lab, to bring EPICS V4 to its full potential.

This Charter is a statement of the basis for the work of the of the EPICS V4 working group, the intended outcomes for that work, its deliverables and success criteria. It also outlines some administrative matters of the organization of the group, and its working practices.

For more information about the EPICS, please refer to the home page of the Experimental Physics and Industrial Control System.

Status of this Document

This is the 17th Feb 2012 version of the charter. This version adds criteria that distinguish Normative, Informative and Internal products of the Working Group. This version is based on previous versions approved by a meeting of the group on 14-Sept-2011 as the basis charter of the group from that date on.

The terms MUST, MUST NOT, SHOULD, SHOULD NOT, REQUIRED, and MAY when highlighted (through style sheets, and in uppercase in the source) are used in accordance with RFC 2119 [RFC2119]. The term NOT REQUIRED (not defined in RFC 2119) indicates exemption.

Table of Contents

1. Basis for Work
2. Mission
3. Scope and Goals
4. Out of Scope
5. Success Criteria
6. Deliverables and Duration
7. Expected Milestones
8. Coordination with Other Groups
9. Working Group Participation
10. Meetings
11. Communication
12. Confidentiality
13. Patent Policy
14. Glossary of Terms

Basis for Work

The EPICS V4 working group is supported by Brookhaven Lab (BNL) and collaborating organizations, subject to their following this Charter.

Mission

The main objective of the EPICS V4 Working Group (EPICS V4) is to specify, publish and implement, a core network messaging system, data format, and client and server APIs, that enable development of efficient distributed control and application services for large scientific instruments and industrial plants. A second objective is development of a core set of associated software tools and frameworks that use the data exchange system for productive development of host level services and interactive applications.

Scope and Goals

EPICS version 3 and its predecessors have been enormously successful as a framework for distributed device control, but have not been optimal for all purposes. Firstly, the underlying network protocol has not been documented and standardized, and so is not suitable for independent implementation or extension. Secondly, the network codec and message passing system has not supported passing semi-structured data, nor arguments, and so not been suitable for use as a services layer in support of host level scientific applications.

The EPICSV4 Working Group is chartered to define and publish an open control protocol for control system endpoints, which functionally extends channel access to suitability for both device control and high level services. This must include data types suitable for high level services, including at least a heterogeneous table (as might be used for returning results from a relational database query) and "multi-channel" array. It should also include a dynamically run-time constructed heterogeneous type analogous to CORBA's Any type or XMLHTTPRequest object (the basis of Web Services).

Specifically, the EPICSV4 group is chartered to procedure a standard for the interface of user level clients, for updating displays, control, and application level data, to services and device control, and experimental data acquisition, implemented in an EPICS V4 IOC, or in an EPICS v3 IOC (where a client may itself be an EPICS V4 IOC).

Java, C or C++, bindings must be supplied for all implementations of each normative standard.

Goals

The goals of the Working Group are:

1. Production of an open, documented, control system message protocol suitable for device control, experimental data acquisition, and services in a distributed control system; including support for semi-structured type transport (see below) and parametrized (attribute=value) PV value requests. This is known as pvAccess
2. Production of an open, documented, data type encoder/decoder capable of exchanging both atomic and semi-structured data with high performance, suitable as the data exchange format for services. This is known as pvData
3. Production of an Input/Output Controller (IOC) framework, suitable for both controls and services, that is on the back-end compatible with EPICS v3 record processing, and on the front-end with the message and data protocols described above. This is known as pvIOC.
4. Production of a software framework for scientific services, as will be suitable for online optimization (e.g. modelling) or coordinated device control (e.g. energy management of a whole linac, or beam synchronized BPM orbit acquisition)
5. Interoperability with EPICS v3. The pvAccess message protocol must be interoperable with EPICS v3.14. See Interoperability with EPICS v3 below
6. Intra-EPICS V4 interoperability and Directory Service. The client side API must be able to acquire any suitably named item, control PV or otherwise, that an EPICS V4 IOC knows how to acquire without specifying itself, from which service to get the value of the named item. Additionally, the client side API must be able to report the EPICS (v3 and V4) acquirable names of all items in its network matching a pattern. See Interoperability within EPICS V4 below.
7. High Performance. The above protocols and controller framework must be of high performance. See Performance goals below
8. Exceptionally easy adoptability. The above protocols and framework must be transparent, documented, and delivered so that it is very easy for a developer to create EPICS V4 IOC applications and services
9. Implementability. The above protocols, and the processing of message I/O, must be documented in such as way that outside software developers can easily create an interoperable implementation
10. Error handling. All normative software implementations must handle and log all errors in such a way that distributed users can see both the causes and outcomes of all errors. In practice, this will mean the creation and adoption of a single error messaging standard, perhaps an EPICS V4 service whose job it is to receive and log errors from all other endpoints.

Interoperability with EPICS v3

EPICS V4 clients must be interoperable with EPICS v3 IOCs. Specifically, PvAccess must run in version 3 IOCs and serve all v3 type structures.

This mechanism must satisfy the expected use case of device control including EPICS V4 in the medium term; in which device control will continue to be through EPICS v3, with control wire protocols and client sides in EPICS V4. It is expected that in the longer term, past the duration of this charter, the EPICS V4 IOC will be properly extended into device control.

The extent and any limitations of compatibility with adjacent layers in the EPICS v3 protocol stack must be documented as a normative document of the working group.

Interoperability within EPICS V4

Each normative standard must be demonstrated to be interoperable with adjacent layers in the EPICS v4 protocol stack by production of, and testing, at least one implementation.

Performance Goals

The following minimum performance goals are expected:

• put / get: 100% of Channel Access in equivalent scenario
• monitors: 100% of Channel Access in an equivalent scenario
• multicast: when available, 100% of Channel Access, but more scalable
• streams: 90% of the raw TCP socket throughput, and 80% of the capacity of the media. For example, over 10GBit Ethernet, then a little less than 8GBits of data transfer on a dedicated VLAN. This should be optimized for sending images and multidimensional arrays, and by implication should minimize copying.

Out of Scope

The following are out of scope for the group specifically:

1. Both and only Java and C/C++ bindings are required for reference implementations of the core protocols (pvAccess and pvData). One or the other language binding is required for service implementations
2. The Windows server side is out of scope for now. A Windows client side will be developed. Windows server side may well be added in a charter revision depending on progress.
3. Development of application specific data protocols or IOC processes, are out of scope. Like channel access itself, the new data protocol and IOC must support a wide variety of applications, and the group should not spend time optimizing for any given service. For instance, inclusion of specific type designs for archiving, images, scanning, are out of scope. Rather, it's expected that the EPICS V4 Interoperable Types specification, will define one or two general purpose types that can be used for semi-structured data, and include a dynamic type that can be used to construct highly application specific types. A few reference application specific data types may be published, but they are only for illustration and offered as a basis for extensions.
4. Development of specific production quality services are out of scope. A few reference services may be produced, but they are only for illustration and offered as a basis for extensions.

With respect to development of support for specific services in the core protocols, or development of specific services, being out of scope: any user, including a member of EPICS V4, is free to use the products of the EPICS V4 group to develop controls, extend any protocol, or create production services, but the group itself should concentrate on the normative standards and their implementations. A separate group may choose to produce some production interoperable services and applications, but they are out of scope for EPICS V4.

Success Criteria

Derived from the goals above, success criteria are at least:

1. Adoption or planned adoption of EPICS V4 in the EPICS community
2. Demonstrated interoperability of EPICS V4 clients and EPICS V4 services
3. Demonstrated interoperability with EPICS v3
4. Demonstrated high performance
5. Demonstrated easy to implement an EPICS V4 service, taking arguments, and returning semi-structured data
6. Demonstrated easy to implement an EPICS V4 client, taking data from EPICS v3 records, packaged by an EPICS V4 service
7. Demonstrated easy for an end user to get a list of all PVs and named entities in an EPICS network, through an EPICS client, given a name pattern
8. Easy for a developer to acquire, understand, and build, a reference implementation of pvAccess and pvData (independently of their server side housing in an EPICS V4 IOC)
9. Easy for a developer to acquire, understand, build and deploy, a reference implementation of EPICS V4 Java IOC or C/C++ IOC.

Deliverables and Duration

The outputs of the working group will be delivered as a system of documented standards, plus reference implementation source code where relevant. These documents and source codes will all be available from the EPICS V4 web site, http://epics-pvdata.sourceforge.net/.

Deliverables

The group is expected to produce the following normative deliverables:

1. A normative document of the pvAccess protocol
2. A normative document of the pvData protocol. The document must include the user API - how a programmer creates data objects for the wire, and extracts them on the other side
3. A normative document of the EPICS V4 IOC processing pipeline
4. A reference implementation of pvAccess in each of C++ and Java language bindings
5. A reference implementation of pvData in each of C++ and Java language bindings
6. A reference implementation of the EPICS V4 IOC in each of C++ and Java language bindings. The Java version has high priority.
7. A normative document of the EPICS V4 interoperable data types. These data types must be universally understood by every client and service which claims EPICS V4 compatibility. The requirement for this deliverable is distinct from the pvData document deliverable, since pvData can encode any type, this deliverable recommends the confined set of data objects that will be used by EPICS V4 interoperable services
8. A directory service accessible through the EPICS V4 API itself, from which can be found at least PV and entity names, and associated service names
9. A normative document of the EPICS V4 services API. This defines the form for encoding parameters and status descriptions between clients and services and back
10. A report of interoperability of the EPICS V4 IOC with EPICS v3 record processing
11. A performance report, comparing EPICS v3 to EPICS V4 for some common EPICS v3 control and read tasks, plus report of the expected performance of EPICS V4 service support. For instance, round trip time for network encoding/deserialization of results of 4 or 5 common service queries such as archive data, orbit data, whole beamline model etc. Comparisons to at least 2 other common high performance data interconnects should be made, eg ICE, ASN.1, EXI Web Service.
12. A "Getting Started" document for EPICS V4 Service developers
13. A User Guide for EPICS V4 IOC control application developers
14. A command line tool similar to caget (call it say pvget), which understands all the interoperable data types above, and conforms to the EPICS V4 services API above.
15. A normative document of the EPICS V4 Directory Service function, API, and unix command line tool.
16. A reference implementation of the EPICS V4 Directory Service.

Additionally, the group is expected to produce the following "extensions" of EPICS V4, as non-normative deliverables (as decided in meeting of meeting of 7-Sep-2011):

17. pvManager, a framework and client side application software library for managing sets of EPICS V4 PV acquisitions.
18. An archive data retrieval service. Only the EPICS service endpoint is in scope, not the archiver architecture itself. This may be modified in a future charter amendment.

A future charter revision may put "gather service" in scope.

Duration

The expiration date of this charter is 1-Sep-2012. At that time, the group may seek to recharter with BNL and PSI, or determine that EPICSV4 should be discontinued.

Expected Milestones

The Working Group's goals and deliverables will be split into three phases:

January 2012, Beta 1 release

Beta 1.0 release. A high level scientific programmer must be able to discover EPICS V4 easily, and learn from sufficient documentation to ascertain its features, working principles, APIs, build procedures and deployment, to build a fast high level data service on top of EPICS V4. The software deployed must be stable and there must be full support for error handling. pvAccess must be at at least Second Public Working Draft status with all facets of the protocol described.

March 2012, Beta 2 release

Beta 2 should add the following:

1. Gather service framework to support server side managed data acquisition, in which a service can be used to monitor many channels and report the values of all of them via the value of a single composite value channel
2. Normative Types. A standard for comprehensive set of interoperable data definitions mus have been published to at least First Public Working Draft
3. An easy to use API for the client side of pvData and pvAccess must be available at useable beta maturity, and include support for the normative types
4. A "Nominal Architectures" document should have been published, giving guidance to EPICS controls architects on the use of EPICS V4 in various common scenarios
5. A naming and directory service framework, probably based on ChannelFinder must have been planned, proposed and in useable prototype form
6. The pvAccess specification should go into a Last Call phase.

July 2012, First Release

The first release of EPICS V4 should include all documentation and software referenced in the deliverables, with professional finish and stable use.

Coordination with Other Bodies

The Working Group should coordinate and align objectives to satisfy the requirements of the following bodies:

BNL NSLS Controls

[TBD: What specific requirements are there of EPICSV4 to satisfy BNL requirements. Are these solely with respect to services, and if so which? Is BNL expecting work on EPICS V4 database or record support?]

PSI SwissFEL

PSI requires EPICS V4 services framework, for online accelerator modelling in particular, and interoperability of the EPICS V4 based services with EPICS v3 IOCs and their associated record support.

SLAC FEL accelerators (LCLS/LCLS-II)

SLAC requires EPICS V4 services framework in general, and for online accelerator modelling in particular.

Group Participation

Effective participation is expected to consume one to two working days per week for each group participant (see below); two days per week for editors of a specific normative document. Developers of implementations of a normative standard (such as pvAccess, pvData, services) are expected to consume three days per week. The Chair shall ensure that the criteria for Good Standing are understood and followed:

Participant Status

Prospective group participants should ask their primary EPICS contact or manager to send an email to one of the EPICS V4 chairs, showing that they understand the participant will be involved with EPICS V4 and the commitment level expected

Observer Status

Individuals who are not active at the full participation level, may join meetings of the group as observers. Email the chairs to get details of how to join the meetings.

The group is expected to have at least 4 active participants to be successful.

Good Standing

A participant who is absent from more than 2 meetings consecutively, or in the opinion of the chairs is not acting on Action Items or their responsibilities for group deliverables, will be deemed out of good standing, and will no longer participate.

Chair

The initial chairs of this Group are Greg White (PSI, SLAC) and Bob Dalesio (BNL).

Meetings

The Group will have distributed meetings, one to two hours every week, and face–to–face meetings, every three or four months.

Communication

The Working Group will utilize the mailing list, epics-pvdata-devel@lists.sourceforge.net. All except the most trivial messages should be cc'ed to this mailing list. The web archive of it used should be used as backward references in communications.

Confidentiality

The proceedings of this Working Group are open to all (not just members of the working group), subject to exceptions made by the Chairs, after consultation with the Working Group. The group's communications on the web archive will be world readable. Posting to epics-pvdata-devel@lists.sourceforge.net will be unrestricted for members, but moderated for non-members.

Patent Policy

This Working Group operates under EPICS Open License.

Glossary of Terms

This section defines basic terms used to identify the provenance of documents and code of the EPICS V4 Working Group. Where appropriate, working group material SHOULD identify itself as one of these:

Normative

Designates documents or source code that should be considered formally specified. Normative documents must contain mostly deterministically specified descriptions of designs and processes (though Normative documents may contain non-normative material to help clarify or illustrate, as long it's clearly shown to be so). Normative software should conform to one or more Normative documents that describe its function. In general, Normative work of the group is intended to be developed collaboratively in the open with the EPICS community. Core products of the working group must be Normative. Normative code will be documented to an agreed standard level and conform to the Coding Standard Guidelines

Informative

Documents and code additionally made available by the group, for illustration, or as minor test code, or incidental, should not be considered normative. Call those simply "informative"

Internal

Documents and code which are neither normative nor informative in the senses above, should be considered simply internal documents of the group. Note that internal documents may appear on the public web site.