Proposal for implementation of unsigned integers 10-Feb-2012

Introduction

The following are the changes required in pvData to support unsigned integers.

For C++ unsigned will be fully supported.

Because Java does not implement unsigned integers a "best" effort is implemented.

As part of the changes for unsigned support the C++ implementation will use size_t instead of int for everything related to size or length. This means that C++ on 64 bit architectures will support arrays of size greater than 2 gigaElements.

pvAccess will also have to be changed to support unsigned. The protocol specification already allows lengths greater than 2 gigaElements.

Examples

For C++ clients the use of the unsigned classes should be obvious because unsigned is fully supported. A Java client, because Java does not support unsigned, must take care when using the unsigned interfaces. The following gives an example for an unsigned scalar, an unsigned array, and usage of the Convert facility.

Unsigned PVScalar

When a client uses one of the unsigned data interfaces, the client must be aware that get returns a signed not an unsigned type. For put the user must be aware that the implementation performs no checks for unsigned.

For example:

    void someMethod(PVUByte pvValue) {
        byte val = pvValue.get();
        if(byte<0) {
            // BE WARY IT IS AN UNSIGNED BYTE
            // COULD DO SOMETHING LIKE
            short uxxx = xxx;
            xxx &= 0x00FF;
        }
        // OR JUST
        short uxxx = pvValue.getWider();
    }

Unsigned PVArray

When a client uses one of the unsigned data interfaces, the client must be aware that get is returning a signed not an unsigned type. For put the user must be aware that thye implementation performs no checks on the input value. For example:

void someMethod (PVUByteArray pvValue) {
    ByteArrayData data;
    int len = pvValue.getLenth();
    int num = pvValue.get(0,len,data);
    byte[] value = data.data; // BE WARY MAY CONTAIN unsigned bytes
    // OR JUST
    short[] uvalue = mew Short[pvValue.getLength()];
    num = pvValue.get(0,len,uvalue,0);

New Convert Features

The Convert Facliity has methods to handle unsigned. Any methods that internally performs a widening operation implements the proper semantics. For example if the conversion is from a byte to a short and the byte is unsigned the upper bits are set to 0 after the byte is copied to the short.

In addition the Convert Facliity has methods that support unsigned. For example:

    fromUByte(PVField pv,byte from);

If the PVField is a wider type: pvShort, pvUShort,pvInt,pvUUnt,pvLong,pvULONG then the upper bits are set to 0

PVData Meta Language

The following:

scalar
A scalar field can be any of the following:
boolean
Has the valuetrue or false
byte
An 8 bit signed integer.
short
An 16 bit signed integer.
int
An 32 bit signed integer.
long
An 64 bit signed integer.
float
A IEEE float.
double
A IEEE double.
string
An immutable string.
scalarArray
A scalarArray field is an array of any of the scalar types.
boolean[]
byte[]
short[]
int[]
long[]
float[]
double[]
string[]

will be changed to:

scalar
A scalar field can be any of the following:
boolean
Has the value true or false
byte
An 8 bit signed integer.
short
An 16 bit signed integer.
int
An 32 bit signed integer.
long
An 64 bit signed integer.
ubyte
An 8 bit unsigned integer.
ushort
An 16 bit unsigned integer.
uint
An 32 bit unsigned integer.
ulong
An 64 bit unsigned integer.
float
A IEEE float.
double
A IEEE double.
string
An immutable string.
scalarArray
A scalarArray field is an array of any of the scalar types.
boolean[]
byte[]
short[]
int[]
long[]
ubyte[]
ushort[]
uint[]
ulong[]
float[]
double[]
string[]

Interface Definitions

Types

The following:

enum ScalarType {
    pvBoolean,
    pvByte,pvShort,pvInt,pvLong,
    pvFloat,pvDouble,
    pvString;
    // The following are convenience methods
    public boolean isInteger();
    public boolean isNumeric();
    public boolean isPrimitive();
    public static ScalarType getScalarType(String type);
    public String toString();
}

Will be changed to:

enum ScalarType {
    pvBoolean,
    pvByte,pvShort,pvInt,pvLong,
    pvUByte,pvUShort,pvUInt,pvULong,
    pvFloat,pvDouble,
    pvString;
    // The following are convenience methods
    public boolean isInteger();
    public boolean isUInteger();
    public boolean isNumeric();
    public boolean isPrimitive();
    public static ScalarType getScalarType(String type);
    public String toString();
}

where

boolean
true or false
byte
An 8 bit signed byte
short
16 bit signed integer
int
32 bit signed integer
long
64 bit signed integer
ubyte
An 8 bit unsigned byte
ushort
16 bit unsigned integer
uint
32 bit unsigned integer
ulong
64 bit unsigned integer
float
32 bit IEEE float
double
64 bit IEEE float
string
An immutable string. The Java implementation is String. For other implementations the network representation must be the same as for Java. Note that a string is treated like it is a scaler.

In C++ there is a set of typedefs for the primitive types. The following will be added for unsigned types:

typedef uint8_t  uint8;
typedef uint16_t uint16;
// definitions for uint32 and uint54 already exist

typedef uint8 *  UByteArray;
typedef uint16 * UShortArray;
typedef uint32 * UIntArray;
typedef uint64 * ULongArray;

Introspection Interfaces

No changes.

Data Interfaces

The following:

Each scalar type has an associated data interface: PVBoolean, PVByte, PVShort, PVInt, PVLong, PVFloat, PVDouble, and PVString.

Will be changed to:

Each scalar type has an associated data interface: PVBoolean, PVByte, PVShort, PVInt, PVLong, PVUByte, PVUShort, PVUInt, PNULong, PVFloat, PVDouble, and PVString.

Java

For Java the get and put methods will have the same definition for the unsigned type as for the signed type.

Some examples are:

interface PVUByte extends PVScalar {
    byte get();
    void put(byte value);
    short getWider();    // new method
}

interface PVULong extends PVScalar {
    long       get();
    void       put(long value);
    BigInteger getWider();    // new method
}

public interface PVUByteArray extends PVScalarArray{
     int get(int offset, int length, ByteArrayData data);
     int put(int offset,int length, byte[] from, int fromOffset);
     void shareData(byte[] from);
     // NEW METHOD
     int get(int offset, int length, short[] dest, int toOffset);
}

public interface PVULongArray extends PVScalarArray{
     int get(int offset, int length, LongArrayData data);
     int put(int offset,int length, long[] from, int fromOffset);
     void shareData(byte[] from);
     // NEW METHOD
     int get(int offset, int length, BigInteger[] dest, int toOffset);   
}

C++

PVByte is:

class PVByte: public PVScalar {
    virtual uint8 get() = 0;
   virtual void put(uint8 value) = 0;
}

Introspection and Data creation

No changes.

pvData Conversion

pvData provides a conversion utility that is defined in Java by interface Convert and in C++ by class Convert

In Java the following methods will be added.

    interface Convert {
        byte toUByte(PVField pv);
        short toUShort(PVField pv);
        int   toUInt(PVField pv);
        long  toULong(PVField pv);
        void fromUByte(PVField pv, byte from);
        void  fromUShort(PVField pv, short from);
        void  fromUInt(PVField pv, int from);
        void  fromULong(PVField pv, long from);
        int toUByteArray(PVScalarArray pv,
            int offset, int len, byte[]to, int toOffset);
        int toUShortArray(PVScalarArray pv,
            int offset, int len, short[]to, int toOffset);
        int toUIntArray(PVScalarArray pv,
            int offset, int len, int[]to, int toOffset);
        int toULongArray(PVScalarArray pv,
            int offset, int len, long[]to, int toOffset);
        int fromUByteArray(PVScalarArray pv,
            int offset, int len, byte[]from, fromOffset);
        int fromUShortArray(PVScalarArray pv,
            int offset, int len, short[]from, fromOffset);
        int fromUIntArray(PVScalarArray pv,
            int offset, int len, int[]from, fromOffset);
        int fromULongArray(PVScalarArray pv,
            int offset, int len, long[]from, fromOffset);
    }

NOTE that the argument types are just like the unsigned argument types.

In C++ the following methods will be added:

    class Convert {
        uint8 toUByte(PVField *pv);
        uint16 toUShort(PVField *pv);
        uint32   toUInt(PVField *pv);
        uint64  toULong(PVField *pv);
        void fromUByte(PVField *pv, uint8 from);
        void  fromUShort(PVField *pv, uint16 from);
        void  fromUInt(PVField *pv, uint32 from);
        void  fromULong(PVField *pv, uint64 from);
        int toUByteArray(PVScalarArray *pv, int offset, int len,
            UByteArray to, int toOffset);
        int toUShortArray(PVScalarArray *pv, int offset, int len,
            UShortArray to, int toOffset);
        int toUIntArray(PVScalarArray *pv, int offset, int len,
            UIntArray to, int toOffset);
        int toULongArray(PVScalarArray *pv, int offset, int len,
            ULongArray to, int toOffset);
        int fromUByteArray(PVScalarArray *pv, int offset, int len,
            UByteArray rom, fromOffset);
        int fromUShortArray(PVScalarArray *pv, int offset, int len,
            UShortArray from, fromOffset);
        int fromUIntArray(PVScalarArray *pv, int offset, int len,
            UIntArray from, fromOffset);
        int fromULongArray(PVScalarArray *pv, int offset, int len,
            ULongArray from, fromOffset);
    }

In addition the internal case statements that have choices for all the integer types will add choices for all the unsigned integer types. When a widening operation is required the code will do the "correct" conversion, i.e. if an unsigned integer is converted to a wider integer (signed or unsigned) type high order bits will be made 0.

size_t instead of int for length and size

Since it has already been decided that we give up on complete compatibility between Java an C++ there will be one additional change while implementing unsigned.

For C++ the type associated for anything associated with length will be size_t instead of int. This means that on a 64 bit architecture C++ will support arrays > 2 giga elements in size. If an application creates arrays > 2 giga elements than the array can not be shared with a Java client or with a 32 bit C++ client.

Note that the pvAccess network specification already supports this.