TROUBLESHOOTING EPICS V4

Troubleshooting EPICS V4 Data Interactions

Abstract

NOTE: This page has not been updated since 2012. It needs work.

This document helps programmers using EPICS V4, to debug issues in their software, by listing potential user issues and their resolutions. This is not a buglist for EPICS V4 itself, it's a guide to help programmers who are new to using EPICS V4 find and resolve their own bugs.

Status of this document

This document is intended to be informative only. It will be updated on a best effort basis by the EPICS V4 working group. At present it is heavily oriented towards service interactions, because of the primary author's interest. Users of EPICS V4 should feel free to add to this document.

This document pertains to BETA 1 of the EPICS V4 distribution. Matters and issues discussed may or may not have bearing on following releases.

Last modified: Fri Sep 21 15:00:35 CEST 2012

Issues and Potential Resolutions

You see the following at the server side console:
Nov 11, 2011 6:01:21 PM org.epics.pvaccess.server.impl.remote.tcp.BlockingTCPAcceptor initialise
INFO: Configured TCP port 5066 is unavailable, trying to assign it dynamically.
Nov 11, 2011 6:01:21 PM org.epics.pvaccess.server.impl.remote.tcp.BlockingTCPAcceptor initialise
INFO: Using dynamically assigned TCP port 51659.
CHECK: Whether you have an instance of the same server running elsewhere.
You see the following at the server side console:
IOCXMLReader.convert terminating with IOException: /Users/greg/Development/epicsV4/workspace_hg/exampleJava/helloWorld/helloService.xml (No such file or directory)
CHECK: Path to the xml file specifying the server may be wrong, check the -record argument in your IOC server startup.
The client can't connect to the server
Example:
Exception in thread "main" java.lang.RuntimeException: connect error channel destroyed
        at helloWorld.HelloClient$Client.connectResult(HelloClient.java:115)
        at org.epics.pvService.rpc.ServiceClientFactory$ServiceClientImpl.waitConnect(ServiceClientFactory.java:98)
        at helloWorld.HelloClient$Client.connect(HelloClient.java:79)
        at helloWorld.HelloClient.main(HelloClient.java:44)
Can be caused by a variety of factors:
  1. CHECK: that the server name used in the client is identical, including case, with the value of recordName in the xml file. Eg, they must both be valued exactly "helloService"
  2. CHECK: no firewall is blocking the comms. Eg, Mac OS X firewall (OS 10.7) if it running, will ask whether "Java" is allowed to accept incoming connections. You may have said no in the past. If this the case the firewall should be reporting into the System Log (open Console and look for messages like 11/11/2011 17:03:44.213 Firewall: Deny java data in from ::ffff:129.129.144.131:58978 to port 5067 proto=17 To fix this in OS X 10.7, go to System Preferences - Security and Privacy - Firewall tab. Even if the Firewall is in Stop state, it's not really off altogether. Go to "Advanced" and check there is no red dot next to Java in the incoming connections list. It must be green:

    Safety and Privacy Dialog

    Even if the Firewall is allowing connections nominally, you may be asked specifically if you want to to allow connections at the time that an EPICS V4 service is started. You must allow it, or your clients won't find your servers.

    Allow Java Connections Dialog
  3. Premature timeout. Transcontinental network connects or slow database backend, etc, will cause the client side to timeout the server. You see: "RuntimeException: Connection error: channel destroyed". Example: % ./getRdb LCLS:elementInfo.byZ Exception in thread "main" java.lang.RuntimeException: Connection error: channel destroyed Diagnosis: The default 5.0 second delay may not be long enough. Just increase the timeout in your client side call to waitRequest.
  4. If the connection is failing for a channelRPC record, CHECK that the "factoryRPC" field value in your services record database (the xml file) is precisely equal to the FULL classname of your service factory, including, if java, the package name. Eg, if your factory is implemented in a class org.mylab.myCoolEPICSService, then your xml should look like:
    <database>
       <record recordName = "myCoolEPICSService" extends = "org.epics.pvService.service">
       <scalar name =
           "factoryRPC">org.mylab.myCoolEPICSService.myCoolEPICSServiceFactory</scalar>
    …
    
    Note that our examples for Java tend to be in the default Java package, so the examples don't have the package name. Yours probably will.
ISSUE: You see client side messages like:
warning PVRecord.registerListener called but listener
  org.epics.ioc.pvCopy.PVCopyImpl$ThePVCopyImpl$CopyMonitor@d0e999 already
  registered
  ...

In bad cases followed by:

Feb 21, 2012 3:52:43 PM org.epics.pvaccess.impl.remote.tcp.BlockingTCPTransport processSendQueue
WARNING: Exception caught in thread processing send requests.
java.lang.IllegalStateException: not queueElement returned by last call to getUsed
at org.epics.pvData.misc.QueueCreate$QueueImpl.releaseUsed(QueueCreate.java:93)
...
... more stacktrace
CHECK: Is your client creating one channelAccess and one channelProvider per channel (wrong) or per channel provider (right). I.e.:
Wrong:

for(int m=0;m<20;m++)
{
    ChannelAccess channelAccess = ChannelAccessFactory.getChannelAccess();
    ChannelProvider channelProvider = channelAccess.getProvider(providerName);
    ConnectionListener cl = new ConnectionListener();
    Channel channel = channelProvider.createChannel("counter_"+m, cl, ChannelProvider.PRIORITY_DEFAULT);
       cl.setChannel(channel);
    ...
}

Right:

ChannelAccess channelAccess = ChannelAccessFactory.getChannelAccess();
ChannelProvider channelProvider = channelAccess.getProvider(providerName);
ConnectionListener cl = new ConnectionListener();
for(int m=0;m<20;m++)
{
    Channel channel = channelProvider.createChannel("counter_"+m, cl, ChannelProvider.PRIORITY_DEFAULT);
        cl.setChannel(channel);
    ...
}

The data object returned by an RPC request, is null

On the RPC server, check that all of the names used to access fields in the introspection interface, match the actual field names. If any one of the field names used in a PVStructure.get*Field("<name>") does not match the field name used in the FeildCreate.createStructure(), then the server side will appear to work, but won't successfully construct the Structure at all. That will leave the client side with a null structure. Example:

Server side:
private final static Structure resultStructure = fieldCreate.createStructure( "uri:ev4:nt/2012:NTScalar", 
					new String[] { "value", "descriptor" },
					new Field[] { fieldCreate.createScalar(ScalarType.pvDouble),
							      fieldCreate.createScalar(ScalarType.pvString)
} );
...
PVStructure result = PVDataFactory.getPVDataCreate().createPVStructure(resultStructure);
			result.getDoubleField("value").put(a + b);
			result.getStringField("description").put("The sum of arguments a and b");

descriptor != description, which will cause all server side processing to quietly fail.