CEMon Client APIs

They are a set of C++ API that allow anyone to build a CEMonitor interactive client or a CEMonitor information consumer.

In the following I'll describe the test tools that give the user the possibility to immediately (i.e. without writing any code)
test some CEMon functionalities; and later I'll explain quickly how to write a CEMon C++ client using these APIs.

Using CEMonitor test tools

In order to use the tools described below you need to install the RPM built by the cruise control
(go there and click on "Build Artifacts" to get the latest nightly built RPM) or build yourself.

When the build of cemonitor client API (org.glite.ce.monitor-client-api-c) you can
immediately start to test the CEMon service using several executables put in your stage/bin area:

• CEMonitorConsumer
• CEMonitorGetEvent
• CEMonitorGetInfo
• CEMonitorGetTopics
• CEMonitorPauser
• CEMonitorResumer
• CEMonitorSubscriber
• CEMonitorUnsubscriber
• CEMonitorSubscriberMgr
• CEMonitorSubscriptionUpdater

Query Tools

In order to experience an immediate interaction with your CEMon service you can use the three command line tools:

1. CEMonitorGetEvent
2. CEMonitorGetInfo
3. CEMonitorGetTopics

In the following I'll assume the reader is already familiar with CEMonitor-related stuff: Topics (or Sensors), Dialects, Actions
(see the Architectural documentation at https://edms.cern.ch/document/585040).

CEMonitorGetInfo prints to STDOUT general informations about the CEMon; for now it says which topics are installed
(and which Dialects are active for each topic) and the active actions;
CEMonitorGetTopics prints to STDOUT all the topics plugged into the CEMon service and related active dialects;
CEMonitorGetEvent prints to STDOUT the last output of the information source (topic) the user must specify on the command line.

Command line option syntax:

• CEMonitorGetInfo <PROXY_CERT> <CERT_PATH> <SERVICE_URL>
  • <PROXY_CERT> your proxy certificate generated using grid-proxy-init or voms-proxy-init
  • <CERT_PATH> even if ignored (at the moment) it must be an existing readable path
  • <SERVICE_URL> the http[s] URL of the remote CEMon service
• CEMonitorGetTopics <PROXY_CERT> <CERT_PATH> <SERVICE_URL>
  • Same meaning of CEMonitorGetInfo's options
• CEMonitorGetEvent <PROXY_CERT> <CERT_PATH> <SERVICE_URL> <TOPIC> <DIALECT> <times>
  • <TOPIC> The topic whose information you're interested in
  • <DIALECT> The information output formatter
  • <times> The number (integer >= 1) of times you want to iterate the GetEvent remote call; if not provided, will be assumed as 10

Examples and output

CEMonitorGetInfo /tmp/x509up_u202 / https://cream-ce-01.pd.infn.it:8443/ce-monitor/services/CEMonitor

Description=[CE-Monitor_web_application]
Version    =[1.7.0]
Topic [CE_MONITOR]
   Dialect [ISM_CLASSAD_GLUE_1.2]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [ISM_CLASSAD]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [ISM_LDIF_GLUE_1.2]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [ISM_LDIF]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
Topic [ICE]
Action [SendExpiredNotification:Action:3]
   Property [supportedDialectsList]=[LFID,ISM_LDIF,ISM_LDIF_GLUE_1.2,OLD_CLASSAD,NEW_CLASSAD,ISM_CLASSAD,ISM_CLASSAD_GLUE_1.2]
Action [DoNotSendNotification:Action:2]
   Property [supportedDialectsList]=[LFID,ISM_LDIF,ISM_LDIF_GLUE_1.2,OLD_CLASSAD,NEW_CLASSAD,ISM_CLASSAD,ISM_CLASSAD_GLUE_1.2]
Action [SendNotification:Action:1]
   Property [supportedDialectsList]=[LFID,ISM_LDIF,ISM_LDIF_GLUE_1.2,OLD_CLASSAD,NEW_CLASSAD,ISM_CLASSAD,ISM_CLASSAD_GLUE_1.2]
   Property [TTLCEinfo]=[300]

CEMonitorGetTopics /tmp/x509up_u202 / https://cream-ce-01.pd.infn.it:8443/ce-monitor/services/CEMonitor

Topic [CE_MONITOR]
   Dialect [ISM_CLASSAD_GLUE_1.2]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [ISM_CLASSAD]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [ISM_LDIF_GLUE_1.2]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [ISM_LDIF]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
Topic [OSG_CE]
   Dialect [OLD_CLASSAD]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [NEW_CLASSAD]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
   Dialect [LDIF]
      queryLanguage [RegEx]
      queryLanguage [ClassAd]
Topic [ICE]

CEMonitorGetEvent /tmp/x509up_u202 / https://grid005:8443/ce-monitor/services/CEMonitor CE_MONITOR ISM_LDIF

Message[0]=glueserviceaccesspointurl
glueserviceaccesscontrolrule
glueinformationserviceurl
gluechunkkey
glueforeignkey
glueceaccesscontrolbaserule
glueclusterservice
gluehostservice
gluehostapplicationsoftwareruntimeenvironment
gluehostlocalfilesystemclient
gluehostremotefilesystemserver
gluecesebindgroupseuniqueid
gluesehostingsl
glueseaccessprotocolsupportedsecurity
glueslservice
gluesllocalfilesystemclient
gluesaaccesscontrolbaserule

Message[1]=dn: GlueCEUniqueID=grid005.pd.infn.it:2119/blah-lsf-grid01, mds-vo-name=local,o=grid

objectClass: GlueCETop
objectClass: GlueCE
objectClass: GlueSchemaVersion
objectClass: GlueCEAccessControlBase
objectClass: GlueCEInfo
objectClass: GlueCEPolicy
objectClass: GlueCEState
objectClass: GlueInformationService
objectClass: GlueKey
GlueSchemaVersionMajor: 1
GlueSchemaVersionMinor: 1
GlueCEHostingCluster: grid005.pd.infn.it
GlueCEName: grid01
GlueCEUniqueID: grid005.pd.infn.it:2119/blah-lsf-grid01
GlueCEInfoGatekeeperPort: 2119
GlueCEInfoHostName: grid005.pd.infn.it
GlueCEInfoLRMSType: lsf
GlueCEInfoLRMSVersion: 4.1
GlueCEInfoTotalCPUs: 3
GlueCEStateEstimatedResponseTime: 0
GlueCEStateFreeCPUs: 3
GlueCEStateRunningJobs: 0
GlueCEStateStatus: Production
GlueCEStateTotalJobs: 0
GlueCEStateWaitingJobs: 0
GlueCEStateWorstResponseTime: 0
GlueCEPolicyMaxCPUTime: 180
GlueCEPolicyMaxRunningJobs: 0
GlueCEPolicyMaxTotalJobs: 999999
GlueCEPolicyMaxWallClockTime: 150
GlueCEPolicyPriority: 1
GlueCEAccessControlBaseRule: VO:EGEE
GlueCEAccessControlBaseRule: VO:PROTO
GlueCEAccessControlBaseRule: VO:MASSIMO
GlueForeignKey: GlueClusterUniqueID=grid005.pd.infn.it
GlueInformationServiceURL: undefined

dn: GlueClusterUniqueID=grid005.pd.infn.it, mds-vo-name=local,o=grid
[...]

Subscription-related tools

• CEMonitorPauser
• CEMonitorResumer
• CEMonitorSubscriber
• CEMonitorUnsubscriber
• CEMonitorSubscriberMgr
• CEMonitorSubscriptionUpdater

These tools are needed to set/unset/suspend/resume/check/update a subscription. A subscription is an internal state of the
CEMon that allows it to asynchronously send an event to an information consumer; an event is an information blob
generated by some specific Topic(Sensor).

CEMonitorSubscriber allows the user to subscribe a certain consumer running somewhere (i.e. <host>:<tcp_port>) into the CEMon.
CEMonitorUnsubscriber deletes a subscription previously set by CEMonitorSubscriber.
CEMonitorPauser and CEMonitorResumer respectively suspend (for a certain amount of time) and resume a subscription. During the suspension time the CEMon wont send any event to the subscribed information consumer. CEMonitorSubscriberMgr prints on STDOUT the list of subscriptions present on a CEMon CEMonitorSubscriptionUpdater updates the parameters of a subscription (like duration, topic, consumer url etc.)

Command line option syntax:

• CEMonitorSubscriber <cert_proxy> <cert_path> <service_URL_address> <consumer_URL_address> <topic> <dialect> <rate> <duration>
  Subscribes an information consumer into the CEMon; it will print on STDOUT the subscription ID, a string the user (or a wrapping script) must remember to do unsubscription, pause or resume of that subscription
  • <cert_proxy> <cert_path> <service_URL_address>: see above (CEMonitorGet*)
  • <consumer_URL_address> the address of an information consumer (in the format http://<host>:<port>) that will receives CEMon's events
  • <topic> the topic (sensor) whose event the consumer is interested about
  • <dialect> the information output formatter
  • <duration> duration of the subscription in seconds
  • <rate> delay in seconds between an event and the next one
• CEMonitorUnsubscriber <cert_proxy> <cert_path> <service_URL_address> <Subscription_ID> deletes a subscription previously set by CEMonitorSubscriber
  • <Subscription_ID> the subscription identifier as returned by CEMonitorSubscription
• CEMonitorPauser <cert_proxy> <cert_path> <service_URL_address> <Subscription_ID> suspends a subscription previously set by CEMonitorSubscriber
  • <Subscription_ID> the subscription identifier as returned by CEMonitorSubscription
• CEMonitorResumer <cert_proxy> <cert_path> <service_URL_address> <Subscription_ID> resumes a subscription previously set by CEMonitorSubscriber
  • <Subscription_ID> the subscription identifier as returned by CEMonitorSubscription
• CEMonitorSubscriberMgr <cert_proxy> <cert_path> <service_URL_address> prints on STDOUT the list of subscription present on the specified host
• CEMonitorSubscriptionUpdater <cert_proxy> <cert_path> <service_URL_address> <SubID_to_update> <newConsumer> <newTopic> <newDialect> <duration> <newRate>
  Updates the parameters of a subscription;
  • <SubID_to_update> the subscription identifier to update (as returned by CEMonitorSubscriber or CEMonitorSubscriberMgr)
  • <newConsumer> the address of the new information consumer
  • <newTopic> the new topic
  • <newDialect> the new information output formatter
  • <duration> duration of the subscription in seconds since 'now'
  • <newRate> new delay in seconds between an event and the next one

Information Consumer

CEMonitorConsumer is a little C++ stand-alone web service that accept TCP connections from a CEMon and parses their SOAP messages (events).

Command line option syntax:

• CEMonitorConsumer TCP_PORT [host_certfile host_keyfile] Starts an information consumer listening for incoming connection on <TCP_PORT>
  • the couple of params <host_certfile> and <host_keyfile> must be used if CEMon sends encrypted notifications; they turn ON the authentication of the internal web service

Making your own information consumer

The CEMon client API provides a class that completely incorporates a stand-alone web-server written in C++ and using SOAP communication protocol (exploting the gSOAP implementation). The communication details with the CEMon (written in Java) are totally hidden, so the user can easily and quickly write its information consumer focusing on the businnes logic, and ignoring any SOAP technicalities. The class we're talking about is CEConsumer and the synopsis is :

#include "soapH.h"
#include "CEConsumer.h"
#include "Topic.h"
#include "CEMonitorConsumerBinding.nsmap"

CEConsumer *consumer = new CEConsumer(port);
Last line of code creates a CEConsumer object that can be binded to tcp port port; once a CEConsumer is succesfully created, in order to turn the web service ON, you must bind it to the TCP port you specified in the constructor and put it in ACCEPT mode:

consumer->bind();
consumer->accept();

both methods CEConsumer::bind(void) and CEConsumer::accept(void) return true is succesfull, false otherwise; the methods CEConsumer::getErrorMessage(void) and CEConsumer::getErrorCode(void) return 2 two string describing the reason of failure. Tipically CEConsumer::bind() can fail if a previous server binded on the same TCP port has been "brutally" killed (using untrapped CTRL-C, i.e. SIGTERM, or a SIGSEGV) without the call of close() or shutdown() on the TCP socket, and the kernel didn't yet free the resource binded to the TCP port. Usually waiting for a while (~20 secs) will allow to rebind on the same port. So, CEConsumer::bind() can be put in a loop that tries for a while...
After a succesfull CEConsumer::accept(), you must call CEConsumer::serve() that will process the SOAP message sent by the CEMon. Once the event message has been processed you can get the string array of messages by calling repeatedly CEConsumer::getNextEventMessage() until it return a NULL char*. If the CEConsumer::serve() fails you can still use CEConsumer::getErrorMessage(void) and CEConsumer::getErrorCode(void) to get a human readable error message. More CEConsumer's methods can provide more usefull information like:

• The producer name of last event (the current Topic or sensor the consumer is subscribed to) by calling CEConsumer::getEventProducer() (return a std::string)
• A pointer to a Topic object that contains several usefull informations by using CEConsumer::getEventTopic() (returns a Topic*)

It follows a very simple example of the concepts described above (that is extracted from the code of the test tool CEMonitorConsumer):

#include "soapH.h"
#include "CEConsumer.h"
#include "Topic.h"
#include "Dialect.h"
#include "CEMonitorConsumerBinding.nsmap"
#include <iostream>
using namespace std;
int main(void) {

cout << "error message=" << consumer->getErrorMessage() << endl; cout << "error code =" << consumer->getErrorCode() << endl; cout << "Retrying in 5 seconds..." <<endl; sleep(5);

cout << "error message=" << consumer->getErrorMessage() << endl; cout << "error code =" << consumer->getErrorCode() << endl; break; }

if(ret == false) { cout << "ErrorCode=[" << consumer->getErrorCode() << "]"<<endl; cout << "ErrorMess=[" << consumer->getErrorMessage() << "]"<<endl; exit(1); } cout << "Event Producer="<<consumer->getEventProducer()<<endl; consumer->getEventTopic()->print(); char *c; while((c = consumer->getNextEventMessage())!=NULL) cout << "message=" << c<<endl; delete(consumer); }

A Topic object can be queried about it's Dialect that formats the messages produced by the topic itself, by using the following code:

N.B.: the Topic* returned by CEConsumer::getEventTopic() and the DialectW* returned by Topic::getDialectAt(int) are pointer to internal data members of CEConsumer and Topic classes respectively, and MUST NOT be deleted by the user; the class descructors will take care of freeing the dinamically allocated memory (calling delete(consumer) will trigger the delete of its internal topic member that then will delete its internal DialectW member). Same consideration for the char* string returned by CEConsumer::getNextEventMessage().

Interacting with CEMonitor using C++ API

Interacting with CEMon server means instantiating objects of specific classes that perform specific tasks. All these classes inherit from a unique parent, AbsRequest. The class hierarchy is

AbsRequest | | ----------------------------------------------------------- | | | | | CEEvent CEInfo CEPing CETopics CESubscription

• CEEvent is to query the server about last event the client is interested to
• CEInfo is to query about general server infos (version, installed sensors that generate events, etc.)
• CEPing is to simply ping the server
• CETopics is to retrieve all the topics (the names of the sensors) installed into the server
• CESubscription is to subscribe an information consumer, to unsubscribe it, to pause and resume the subscription

A subscription is a server's internal persistent information that modify the server's behaviour. A subscribed consumer periodically receives an event from the CEMon server; the received event concerns the topic the consumer is subscribed to.

The common interface is defined of course in AbsRequest:

• std::string getErrorMessage()
• std::string getErrorCode()
• void setServiceURL(const std::string&)
• std::string getServiceURL(void)
• void cleanup(void)

getErrorMessage() and getErrorCode() return the error message from the server when it raised a fault as response to a request.
setServiceURL(...) sets the endpoint address the server CEMon is running on. This method (common to all CE* classes) must be called before sending the request to the server. For CEMon the service URL format is: https://<host>:<tcpport>/ce-monitor/services/CEMonitor
getServiceURL() does what it says
cleanup() removes from the heap all C++ deserialized data structures created when a request/response is considered finished and the client got the data it needed from the server. After this call another request can be send to the server without leaking memory.

CEEvent: retrieving last sensor event

The interface CEEvent provides is:

• CEEvent(const std::string& certfile, const std::string& certpath) throw(AuthenticationException&, GeneralException&);
• void setRequestParam(Topic *topic);
• void getEvent(void) throw(GenericException&,

TopicNotSupportedException&, DialectNotSupportedException&, ServiceNotFoundException&, GeneralException&) ;

• int getEventID() const;
• time_t getEventTimestamp() const;
• const char* getEventDateTime() const;
• const char* getEventProducer() const;
• const char* getNextEventMessage();
• const char* getLastEventMessage() const ;
• const char* getEventMessageAt(int index) const;
• int getNumberOfMessages() const;

The interface of the various exceptions will be clear from the example code itself.

Let's start with an usage example: what the client can do is to select a specific topic to query (the name of a sensor) and get last bunch of information produced by that topic; the information can be formatted using different dialect. To see the topics and dialects (of each topic) supported by your CEMon server see later, the CEInfo section. The steps are:

1. Creating a Topic object using the topic name as constructor argument
2. Creating a Dialect object using the dialect name as constructor argument
3. Adding the dialect to the topic
4. Creating a CEEvent object using the certificate proxy file name as first argument and the path containing the certificate of local machine as second argument
5. Setting the endpoint using the method CEEvent::setServiceURL(...)
6. Setting the topic to query using the method CEEvent::setRequestParam(...)
7. Calling the remote server method using CEEvent::getEvent()
8. Calling repeatedly the method CEEvent::getNextEventMessage() until this function returns NULL
9. Calling the cleanup routine, CEEvent::cleanup() to free allocated memory (this must be called AFTER collecting all message with the looped CEEvent::getNextEventMessage()).
10. returning to point 7

the code:

When you call the CEEvent constructor, the AbsRequest's constructor is called too and it tries to initializes the SOAP runtime environment. If this operation (that implies a dynamic memory allocation) fails a GeneralException is raised. The AuthenticationException is raised by the CEEvent's constructor if for any reason the SOAP authentication setting fails (the certificate file or the certificate path are not there, the file or path are not readable, the certificate file is bad...)

[to be continued...]

For any question please contact me at alvise.dorigo@pd.infn.it