EPICS V4 Proposed Changes to release 3.0

EPICS N4 Working Group, Working Draft, 20-Mar-2014

Latest version:
proposedChanges_3_0.html>
This version:
proposedChanges_3_0_20140320.html>i
Previous version:
proposedChanges_3_0_20131023.html>i
Editors:
Marty Kraimer, BNL
Matej Sekoranja, cosyLAB
Michael Davidsaver, BNL
David Hicken, Diamond Light Source

Abstract

Proposed Changes to pvData and pvAccess after Release 3.0

Status of this Document

This document discusses proposed changes to the repositories that make up release 3.0.

Table of Contents

Introduction

This document discusses the status of pvData, pvAccess, pvaSrv, pvDatabaseCPP, and pvIOCJava. It describes changes that have been made since the release last October. It also discusses changes that will also be made.

Since removing methods from pvData and the pvAccess API changes will break other projects the changes will be made on a branch named changesAfter3_0_2.

The changes will affect the following projects:

pvDataCPP
pvAccessCPP
pvaSrv
pvDatabaseCPP

pvDataJava
pvAccessJava
pvIOCJava

Only after all these projects are working will each be merged back into the default branch.

Following the description of each change that is not done is a action item with the names of who is responsible for the change. This is a paragraph starting with:

AI ...

pvUnion and pvUnionArray

Both Java and C++ now support two new data types: union and unionArray

The definition of Type is now:

public enum Type {
    scalar,
    scalarArray,
    structure,
    structureArray,
    union,
    unionArray;    
}
The types union and unionArray are new.

FieldCreate, the facility for creating introspection interfaces has the following new methods:

public interface FieldCreate {
    ...
    UnionArray createUnionArray(Union elementUnion);
    UnionArray createVariantUnionArray();
    ...
    Union createVariantUnion();
    Union createUnion(String[] fieldNames, Field[] fields);
    Union createUnion(String id, String[] fieldNames, Field[] fields);

PVDataCreate has the new methods:

public interface PVDataCreate {
    ...
    PVUnionArray createPVUnionArray(UnionArray unionArray);
    ...
    PVUnion createPVUnion(Union union);
    ...
    PVUnion createPVVariantUnion();
    PVUnionArray createPVVariantUnionArray();
    ...
    PVUnion createPVUnion(PVUnion unionToClone);
    ...
    PVUnionArray createPVUnionArray(Union union);
    ...
}    

A varient is a union where the data can be any pvType. A C++ example is:

static void exampleVarientUnion()
{
    cout << endl;
    cout << "exampleVarientUnion" << endl;
    PVUnionPtr pvUnion = pvDataCreate->createPVVariantUnion();
    cout<< "pvUnion " << pvUnion->dumpValue(cout) << endl;
    PVScalarPtr pvFloating = pvDataCreate->createPVScalar(pvDouble);
    convert->fromDouble(pvFloating,10.3);
    pvUnion->set(pvFloating);
    cout<< "pvUnion " << pvUnion->dumpValue(cout) << endl;
    PVScalarPtr pvInteger = pvDataCreate->createPVScalar(pvInt);
    convert->fromInt(pvInteger,10);
    pvUnion->set(pvInteger);
    cout<< "pvUnion " << pvUnion->dumpValue(cout) << endl;
}

This produces:

exampleVarientUnion
any 
    (none)
pvUnion 0x604148
any 
    double  10.3
pvUnion 0x604148
any 
    int  10
pvUnion 0x604148

The following is an example of a union that can only be a pvInt or pvDouble.

static void exampleUnion()
{
    cout << "exampleUnion" << endl;
    StringArray names(2);
    names[0] = "float";
    names[1] = "integer";
    FieldConstPtrArray fields(2);
    fields[0] = fieldCreate->createScalar(pvDouble);
    fields[1] = fieldCreate->createScalar(pvInt);
    PVUnionPtr pvUnion =  pvDataCreate->createPVUnion(
         fieldCreate->createUnion(names,fields));
    cout<< "pvUnion " << pvUnion->dumpValue(cout) << endl;
    PVScalarPtr pvFloating = pvDataCreate->createPVScalar(pvDouble);
    convert->fromDouble(pvFloating,10.3);
    pvUnion->select("float");
    pvUnion->set(pvFloating);
    cout<< "pvUnion " << pvUnion->dumpValue(cout) << endl;
    PVScalarPtr pvInteger = pvDataCreate->createPVScalar(pvInt);
    convert->fromInt(pvInteger,10);
    pvUnion->select("integer");
    pvUnion->set(pvInteger);
    cout<< "pvUnion " << pvUnion->dumpValue(cout) << endl;
}

This produces:

exampleUnion
union 
    (none)
pvUnion 0x604148
union 
    double  10.3
pvUnion 0x604148
union 
    int  10
pvUnion 0x604148

NOTE: neither pvDataCPP.html or pvDataJava.html discusses union.

AI Matej has "cookbook" with examples. Cookbook will be put in documentation directory. Marty will update both pvDataCPP.html and pvDataJava.html.

pvDataCPP New Array Support

Description

These are changes that Michael Davidsaver made. There is a new class that inplements shared_vector. It is a template class and is used by all the array types provided by pvData. The various PVArray types all inforce COW (Copy On Write). The implementation implements COW by using an immutable shared_vector to hold the data.

pvSubArrayCopy

pvDataCPP has a new and much smaller version of Convert. It no longer has the methods for converting between raw array data and PVScalarArrays. pvSubArrayCopy provides methods required by the pvCopy faclity implemented by pvDatabaseCPP, They are described in file pvSubArrayCopy.h and implemented in pvSubArrayCopy.cpp

Deprecated Array Methods

Many of the old array methods were kept but deprecated. This was because of code in pvAccess, pvaSrv, and pvDatabaseCPP that used the deprecated methods. All three projects no longer use any of the deprecated methods. All deprecated methods will be removed.

pvDataCPP.html

The documentation describes the new array support and also removed descriptions of all the deprecated methods. As mentioned above it does not describe union. Michael needs to look at and possibly correct the description of arrays.

AI Michael will review the documentation about shared_vector and new implementation of PVArrays

Build Support for Other Operating Systems

I think that there is now support for building pvDataCPP on windows and RTEMS. I do not kmow about vxWorks. I will let Matej give the status.

AI Matej will track progress of builds on other operating systems.

bitSet

The semantics of Java and C++ are now the same. The changes were to the following C++ methods, which did not have the same semantics as Java.

operator&=
The only change is to first check for self assignment
operator|=
Check for self assignment. Next make wordsInUse max of wordsInUse and set.wordsInUse. After operation on wordsInCommon copy any remaining words.
operator^=
Check for self assignment. Next make wordsInUse max of wordsInUse and set.wordsInUse. After operation on wordsInCommon recalculateWordsInUse()
operator-=
This is not in the Java version was removed from the C++ implementation.

AI: None. This work is complete.

Monitor Queuesize

This discussion applies to pvAccessCPP, pvAccessJava, pvIOCJava, and pvDatabaseCPP.

Previously pvAccess, pvIOCJava, and pvDatabaseCPP all implemented monitir queues with different semantics. Now they all only implement the following:

queueSize=1
There is a single monitorElement but a private changeBitSet, overrunBitSet, and pvStructure. When the client calls poll the private data is copied to the monitorElement.
queueSize>=2
A finite queue of monitorElements is created. The existing semantics for monitorQueue is OK. But should a queue be created on both client and server? I think not. Having a queue on the server but not on the client sounds desirable. How can client specify this since currently both client and server get the queueSize from pvRequest?

Previously there were additional options like noQueue, etc. These had problems.

AI None since this work is complete.

`

Other changes to pvDataCPP

This section discusses proposed changes to the definitions in pvData.h. The changes are mostly to remove methods. This is a result of discussions on the epics-pvdata mailing list. The main contributers to this discussion were Benjamin Franksen, David Hicken, and Michael Davidsaver.

const

David has sent some emails about more rigouous use of const in pvData.h. Michael agrees.

AI Some combination of Michael, David, and Matej will make changes. Perhaps David can make them and have Michael review changes?

PVField

The following methods will be removed from PVField

class PVField
...
public:
...
    virtual void message(String message,MessageType messageType);
    virtual void setRequester(RequesterPtr const &prequester)
    PVAuxInfoPtr & getPVAuxInfo();
    void replacePVField(const PVFieldPtr&  newPVField);
    void renameField(String const & newName);
...
protected:
...
    void replaceField(FieldConstPtr &field);

AI Marty will remove the above methods to pvDataCPP. The changes will not be made to pvDataJava until xml parsers are rewritten.

Question: Should the following methods be replaced?

    virtual void toString(StringBuilder buf);
    virtual void toString(StringBuilder buf,int indentLevel);
    virtual std::ostream& dumpValue(std::ostream& o) const = 0;

Perhaps by:

    virtual String toString() const;
    virtual std::ostream& operator<<(std::ostream& o) const = 0;

AI: Matej will make changes so that this is compatible with streams. He will make changes in pvDataJava so that toString does not show the entire array is huge.

PVStructure

The following methods will be removed from PVStructure

class PVStructure
...
public
...
    void appendPVField(String const &fieldName,PVFieldPtr const & pvField);
    void appendPVFields(StringArray const & fieldNames,PVFieldPtrArray const & pvFields);
    void removePVField(String const &fieldName);
    String getExtendsStructureName() const;
    bool putExtendsStructureName(String const &extendsStructureName);
...

AI Marty will remove these methods from pvDataCPP. The changes will not be made to pvDataJava until xml parsers are rewritten.

Question Should the following methods be added?

    PVBooleanArrayPtr getBooleanArrayField(String const &fieldName) const;
    ...
    PVStringArrayPtr getStringArrayField(String const &fieldName) const;

Note: The methods getBooleanField, ... , getStringField no longer generate a message if they fail.

AI The methods above will not be provided. The methods getBoolean, etc will be replaced by a template(generic) version of getSubField. Matej will make the changes.

Change to pvDataJava

For now the only change to pvDataJava is the toString method for scalar arrays. Currently the entire array appears. This will be changed to limit the output to some number of inital elements and some number of ending elements. The number will be compatible with what C++ does.

In the future the methods removed from pvDataCPP will also be removed from pvDataJava. This requires changes to the xml parser in pvIOCJava.

AI Marty will make the changes.

base 3.15

Is this the time to start building against the base 3.15 releases?

AI Nothing will be done until later. Ralph will decide when it will be done.

pvAccess API changes

The changes to channelArray and channelPutGet change the pva network protocol. The changes to channelArray will be made. The changes to channelArray only change the C++ API. Since these changes do change the network protocol the changes to channelPutGet are also considered.

The changes to channelGet, channelPut, and channelPutGet are suggested because they help prevent clients making errors related to thread concurrency. The changes are described for the Java implementation but similar changes can be made to the C++ implementation.

channelArray

The current definition is:

class ChannelArray : public ChannelRequest{
...
    virtual void putArray(bool lastRequest, int offset, int count);
    virtual void getArray(bool lastRequest, int offset, int count;
    virtual void setLength(bool lastRequest, int length, int capacity)'
...
}

This will be changed to:

class ChannelArray : public ChannelRequest{
...
    virtual void putArray(bool lastRequest, size_t offset, size_t count);
    virtual void getArray(bool lastRequest, size_t offset, size_t count;
    virtual void setLength(bool lastRequest, size_t length, size_t capacity)'
...
}

This change also requires a change to the pvAccess network protocal. Cureently the network prorotol states that offset and count are send as an int. This will state that they are sent as a size.

AI Matej will make the changes to pvAccessJava and to serialize/deserialize in both Java and C++

channelGet

The current definition for Java is:

public interface ChannelGetRequester extends Requester {
    void channelGetConnect(Status status,ChannelGet channelGet,PVStructure pvStructure,BitSet bitSet);
    void getDone(Status status);
}

interface ChannelGet extends ChannelRequest {
    void get(boolean lastRequest);
}

Should this be changed to?

public interface ChannelGetRequester extends Requester {
    void channelGetConnect(Status status,ChannelGet channelGet,Structure structure);
    void getDone(Status status,ChannelGet channelGet,PVStructure pvStructure,BitSet bitSet);
}

interface ChannelGet extends ChannelRequest {
    void get(boolean lastRequest);
}

channelPut

The current definition for Java is:

interface ChannelPutRequester extends Requester {
    void channelPutConnect(Status status, ChannelPut channelPut, PVStructure pvStructure, BitSet bitSet);
    void putDone(Status status);
    void getDone(Status status);
}

interface ChannelPut extends ChannelRequest {
    void put(boolean lastRequest);
    void get();
}

Should this be changed to?

interface ChannelPutRequester extends Requester {
    void channelPutConnect(Status status,ChannelPut channelPut,Structure structure);
    void putDone(Status status,ChannelPut channelPut);
    void getDone(Status status,ChannelPut channelPut,PVStructure pvStructure, BitSet bitSet);
}

interface ChannelPut extends ChannelRequest {
    void put(boolean lastRequest, PVStructure pvStructure, BitSet bitSet);
    void get();
}

ChannelPutGet

interface ChannelPutGetRequester extends Requester {
    void channelPutGetConnect(Status status, ChannelPutGet channelPutGet,
            PVStructure pvPutStructure, PVStructure pvGetStructure);
    void putGetDone(Status status);
    void getPutDone(Status status);
    void getGetDone(Status status);
}
interface ChannelPutGet extends ChannelRequest {
    void putGet(boolean lastRequest);
    void getPut();
    void getGet();
}

Should this be changed to?

interface ChannelPutGetRequester extends Requester {
    void channelPutGetConnect(Status status, ChannelPutGet channelPutGet,
            Structure putStructure, Structure getStructure);
    void putGetDone(Status status,ChannelPutGet channelPutGet,PVStructure getPVStructure,BitSet getBitSet);
    void getPutDone(Status status,ChannelPutGet channelPutGet,PVStructure putPVStructure,BitSet putBitSet);
    void getGetDone(Status status,ChannelPutGet channelPutGet,PVStructure getPVStructure,BitSet getBitSet);
}
interface ChannelPutGet extends ChannelRequest {
    void putGet(boolean lastRequest,PVStructure pvPutStructure,BitSet bitSet);
    void getPut();
    void getGet();
}

NOTE: Adding the bitSets changes the pvAccess network protocol. If the bitSets are not added then the network protocol does not change.

AI Matej will make the above changes. channelPutGet will have bitSets. More definition required fir bitSets to handle think like access control. Matej will direct requirements definition an implementation in pvAccessCPP and pvAccessJava. Marty will make any required changes to pvDatabaseCPP and pvIOCJava.

Cancellation of channel requests

All channelXXX requests will a new method that cancels currently pending request. The method will be added to ChannelRequest interface:

class ChannelRequest {
    virtual void cancel();
}

AI: Matej will make these changes.

Access Security support

pvAccess API will get new methods to support pluggable security mechanism (a bit more generic and flexible Channel Access security API design).

AI: Matej will make these changes.

pvaSrv

The pvAccess API changes will require changes to pvaSrv.

AI Ralph will make these changes.

pvIOCJava

The pvAccess API changes will require changes to pvIOCJava.

AI Marty will make these changes.

pvDatabaseCPP

The following changes will be made:

pvCopy
Consider breaking pvCopy into two components. One will know only how to copy between a top level structure and another top level structure that is a subset of the fields in the master. The other component is uses the first component and handles the PVRecord methods.
local Channel provider
This will leave the semantics coming via pvRequest to a layer below local provider. This will allow plugins. The local provider will make the pvRequest and pvCopy methods available to layer below.

AI Marty will makes these changes