Agenda
======

9.00 am convene
 
Morning - Beta 2

 * Microbenchmark testbench design and initial performance breakdown report (*Matej*). 

 * Status breaking v3Channel out of pvIOCCPP (*Ralph*, *Marty*), plus Demo.

 * Impressions of getting started with vxWorks and v3Channel 
   compilation (*Dirk Zimoch*

 * APS' requirements and desires for the IOC (*Andrew*
   For Andrew and Tim Mooney.

 * Beamline instrumentation controls requirements on EPICS V4 (*Bob*)
   Broad matters of high throughput large datasets etc.

Afternoon - IOC image processing requirements and CSS 

 * 1:30 pm NDarray / image server (*David*
   Demo included.

 * Versioning and the EPICS V4 release procedure (*Ralph*)

 * **3:30 pm** [9:30 am NY local time] CSS and PvManager status (*Gabriele*). 
   [Ralph will manage connection to Gabriele.]

 * Plans for CSS/pvAccess integration.
   *Matej*, *Gabriele*, *Bob*.

 * Try to make a real plan for the compression support of NTImage. 
   [Who can talk to this, *Nikolay?*]     


------------------------ Minutes ----------------------------

Present: GW,DH,RL,BD,MS,AJ,TK, Pavel Chevtsov, Martin Gasche, Dirk Zimoch
Scribe: TK

************
NEW TOPIC: Microbenchmark testbench design and initial performance breakdown report 
************
 Slides: Microbenchmarks
(*Matej*). 

Matej's talk: microbenchmarking
create a set of profiling tools for the entire chain. Support multithreaded sw.
Uses macros embedded in the code to insert monitoring points.

GW/AJ: Table of results give the RMS of time of each stage, but it was presented as the mean. Check
whether RMS col describes mean or variance.

Effects at start (filling of cache) should be taken a look at.

GW/AJ/DZ: The benchmark results presented should distinguish between intial runthrough timings from subsequent ones, because of various process warmup effects (caching, branch-prediction and so on).

Plans: automate processing of results. Run the benchmark and get the results

GW: have we compared Java and C++ results?
MJ: no big difference visible

GW: prepare instructions for programmers how to use the microbenchmarks

AJ/DZ: There are cases (eg snl) where the client and server are operating on the same machine. In those
cases it would be worthwhile looking into some local transport such as shared memory. Suggestion for
the future.

Need benchmarks run on different platforms: RT Linux, vxWorks, RTEMS etc. 

MS: Good idea to have a machine to run the benchmarks before each release to check the performance


************
NEW TOPIC: Status breaking v3Channel out of pvIOCCPP plus Demo.
************
 (*Ralph*, *Marty*),
 [Talk file pvaSrvStatus.html, from Marty, please fwd to Greg]
 
Ralph's presentation: pvaSrv status
Methods to use pvAccess to access version 3 records (anything that CA can connect to.)

DZ: What if the V3 channels are not in the same IOC?
RL: I assume [pvaSrv] won't work unless all channels are in the same IOC. However,
it may work with ca links in that case.

GW: It returns data as scalarArray, why not pvData structure [so that each V3 channel's name are the
names of the fields of the structure]?
RL: Don't know, ask Marty

GW: how to configure which channels are addressed?
RL: answer is in the talk. Defined in the command. Current implementation is a proof of principle, not final.

RL: after the access is done information is deliverd if the access was atomic. Need a method that indicates (failure) if the put was not atomic

RL: currently works only for value (VAL) fields. Reason to be clarified, no obvious reason for that limitation.
Semantics of puts and processing needs to be defined.

AJ: Change way of naming channels in multivalue to a single argument containing a space-separated list of names. e.g:
    $ pvput 'double01 double02 double03' 3 1 2 3
    $ pvput 'pva:///double01+double02+double03' 3 1 2 3

DZ: what is the purpose of this? Channel access clients would break this behaviour
Discussion: what are our goals with the IOC.
Proposal (RL): leave multiValue to be implemented only in 3.15 (and above)
What to do with pvaSrv? Add into a EPICS release soon? 

********
DECISION: Provide support for single value pvaSrv on EPICS ver 3.14. Support for multivalue pvaSrv will compile against 3.15 only.
********
********
AI: on GW: Add links to pvaSrv (given by Ralph) to the mainpage of EPICS V4 webpages. 
********

RL still has to actually remove pvaSrv code from pvIOCCPP. 

RL: How to check that a NT type is correct?
DZ: do not trust client writers, rather server writers
Need way to validate the structures.

DIscussion: do we need an API for writing normative types
GW: writing NT is too simple. Having a validator and API would be nice but we do not have the manpower.

************
NEW TOPIC: Issues (annoyances) with building EPICS 4 (Dirk)
************

************
ISSUE: Couldn't find out how to do things from the web page. For instance, how to get started to get things.
************

************
AI on GW: Make "getting started" doc more visible and try to make index page link to useful data on getting started quicker. Include a text version of Getting Started in the tar distribution, possibly right in the README.
************

************
AI on AJ: Send comments to GW on the Getting Started document.
************

DZ: Gives a list of complaints and findings in his experience of first building EPICS V4 and using the "v3Channel" to get V3 data through V4:

DZ: readme doesnt explain how to get started.
DZ: The required order of compilation is not given in the README.
AJ: These have since been covered in the gettingStarted doc.
GW: Actually they have always been covered in Getting Started.

DZ: There are too many warnings in the c++ compilation.

DZ: Getting started should say from where you should get boost for vxWorks, and how to integrate it into your compile.

************
ISSUE: Some code mixes EPICS 4 strings (String) and standard strings (std::string). This potentially breaks the code.
************
Passing a std::string pointer to a String pointer doesn't work.
Solution proposed is to remove typedef String and typedef StringBuilder and simply use std::string and STL IO streams.
AI on MS asap: remove typedef String and typedef StringBuilder and simply use std::string and STL IO streams.

ISSUE: The montavista compiler generates error on the SHOW_EXCEPTION macro.
This probably a compiler error!
Mitigation is to assume that this compiler will not be used commonly.
TK: We don't so far have a list of supported target architectures, is it time to creat one?
AJ: There are many V3 supported architectures, maybe V4 should aim to support (but not necessarily tested) on all those?
AJ: Suggests email thread for assembling list of archs and compilers that are known to work and not work.

DZ: vxWorks versions of IO Streams don't support exceptions...

************
ISSUE: There are differences in how functions are called in iocShell and vxWorks shell - iocShell wrappers are doing real work instead of just wrapping the call (which is available in vxWorks).
************
************
AI on RL by any time soon: Unify configuration calls for vxWorks shell and iocShell [to mitigate issue above]
************

********
ISSUE: DZ: the configuration mechanism for multiValue (pre-configuring records sets that are published under a new name) is rudimentary
TK: Do we want to keep that mechanism?
*******
*******
AI on RL & AJ: Need to take a look at how to configure multivalue properly. (Maybe through info fields?). 
*******


************
NEW TOPIC: APS' comments on getting started,
************
Andrew

Notes: ANJ-getting-started.txt

************
NEW TOPIC:  requirements and desires for the IOC
************
Andrew for Andrew and Tim Mooney.
Slides: v4-requirements

Recommendations for V4 db:
* There is presently no value type with error bars:
   GW: Not quite true, NTAggregate includes error bars, that's purpose of "dispersion" field.
   
*  Data Analsysis use-case.
   It would be a good idea to collaborate on a basic beamline data analysis IOC.
   BD: This might change soon.
   DZ: But experimenters won't trust an IOC they did not invent.
   Consensus: Such beamline IOC may be created within communities.
   
 In general, Tim is very positive about V4, but doesn't want to reinvent the Synapps package for it.
 
 Additional Ideas
   * Subscription parameters. Would be good to add to pvaccess the same kind of server side filtering that is in V 3.15.
   Eg, client-specfic monitor dead-bands, rate limits, conditional monitors.
   
   * Reflection API, ie, one should be able to ask an IOC what PVs it holds.
   
   Discussion of with V4 servers (pvIOCJava get & rpc, pure pva, and pvaSrv) should get an RI that
   at least allows one to get record name list.
   AJ: A client shoudl be abel to request of server that it tells only the list of names of records, not names of
   all fields as well.

*********
RESOLUTION: We shall add the addition of a reflection [needs to get a better name] interface accessible  from pvaccess clients, to the goals for the year, so one can get a list of names of records and possibly fields (at least), possibly also other metadata of the server. With respect to pvaSrv, support for acquisition V3 IOC metadata (names of records and possibly list of fields for a record) and precise function, are to be decided.
******


************
NEW TOPIC: Beamline instrumentation controls requirements on EPICS V4 (*Bob*)
************
Slides: Beamline Requirements

Broad matters of high throughput large datasets etc.

Bob described work being done at BNL for prototyping a high throughput IOC.



AFTERNOON

************
NEW TOPIC: NDarray / image server (*David*
************
 Test server is publishing data as defined in NTImage, modelled on the areaDetector data definition
GW: What for?
DH: To allow separating or distributing the image processing from the image acquisition. 
DH: pvGet take a long time and give time outs.

MS will fix the slow printing in pvGet for performance for pvStructure
MS will look into the race condition for the PV test server.
DH will work with Matej to incorporate this example into the test server
DH will work with Nikolay and Michael on image compression going forward
DH will tweak the performance for testing throughput in PVAccess

DH NTImage should use NTVariantArray for image data as the encoder of the pixels can change the data type during operation. (It usually won't, but it can.)
DH The unsigned types need to be supported.
DH field in image part includes fourcc (image encoding) (fourcc = 4 character code)

Tightening up the description of fourcc field of NTImage:
"fourcc" stands for 4 character code. Identifies codec that has been used in
      the case of compressed data. "JPEG", for example. <a
      href="http://www.faqs.org/rfcs/rfc2361.html">List of codecs</a>. The empty
      string, or a value of four spaces, indicates no compression has been
      used. Additionally, the field may be valued "IRAW" in which case data should be interpretted
      in compliance with  <a
      href="http://www.faqs.org/rfcs/rfc2361.html">RFC2361</a>.

DH make fourcc optional
In his version of the NT document, DH has made changes to the NTVariantArray to include unsigned types

*********
AI on DH: Update the NT document with the agreed changes to definitions of NTImage and NTVariantArray for support of NTImage and areaDetector.
*********

If image data contains compressed 3D data (meaning a series of compressed 2D images), there would need to be an offset stored for the start index of the compressed data of each slice (since the compressed slices won't all be the same size and can't easily be calculated). Ask Mark Rivers about this issue and discusss adjusting the NT spec if necessary.Versioning and the EPICS V4 release procedure 

************
AI on DH: publish the test image server along with a readme.
************


************
NEW TOPIC: Versioning and the EPICS V4 release procedure 
************
Ralph
Jenkins build is now on cloudbees - and includes not only V4 but also CSS, logs, etc....
This creates the build and pushes the result onto sourceForge
Gabriele has secured a medium size business account for us. (I.e. 5000 minutes per month build time, good support.)

TK: What about vxWorks build
AJ: ~ will be setting up vxWorks build at APS.

4.major.minor.bugfix
--- major is not forward compatible.
--- all core modules get pulled up to the major number when that one changes
--- a new release is 4.n.0.0 (a new major release resets minor and bugfix to 0, all parts are always present)

************
AI on RL: Describe this, distribute and get confirmation - and then implement it.
************

Java source in JAR - binary in TAR
Java TAR should contain:
- binary, sources and javadoc as JARs
- dependency libraries as JARs (in a lib subdirectory)
- README
C++ TAR should contain:
- sources expanded as a tree
- documentation and doxygen expanded as a tree

************
AI on Ralph: will release the development branching plan.
************
He will take care of this in Mercurial, based on Vincent Driessen's
http://nvie.com/posts/a-successful-git-branching-model/


************
New Topic: CSS nd PvManager status (*Gabriele*):
************
Slides: pvManager

AI on MS: will provide a PVA plugin for CSS.scalars, matrices, array, and image. Ideally that connection will be done 
by also writing a wrapper package of normative types, which is then used to get and set n-type variable values from vTypes.

************
AI: BD resolve how the development and integration of PVAccess into CSS will be done, ie identify a person to do it.
************


************
NEW TOPIC: Other issues
************

************
ISSUE: points out that the license and copyright files in EPICS V4 repo are rubbish. 
************
GW: Acknowledged and we have to fix it.

************
ISSUE: Very large data set requests, such as large NTTable data from a database, can cause middle services to crash or otherwise choke.
************

RL: We have to resolve how to send tables that are way too big to be fully kept in a middle layer service's memory. Do we change tables to do this, adding an optional "more"/"done" field? Do we need a data iterator?
MS: It should be a type - so the client can decide if it wants to do this. It may want to cancel the operation. 
AJ: This could also be needed for matrices, large arrays and images even...
RL: If the database cannot be cancelled, we can handle cancel on the middle layer server side by just dropping what the database is sending back.