CEMON / BDII Server
Table of Contents
- Introduction
- Schematic of OSG BDII Information Service
- CEMon-BDII Aggregator Components
- Interface to BDII
Introduction
The GOC hosted CEMon-BDII server acquires LDIF formatted information from several OSG resources (via CEMon), and serves up the most recent via a BDII server.
Schematic of OSG BDII Information Service
CEMon-BDII Aggregator Components
CEMon Consumer
This process listens on port 14001, and waits for connections from CEMons running on OSG resources. It then saves the data sent to a file in the CEMon consumer's own data space (currently $SERVER_ROOT/data/cemon_raw_incoming/). The file name is that of the FQDN reported by the CEMon consumer.
LDIF Translator/Pruner Script
A translator script runs periodically via cron (every minute); It scans the CEMon raw data directory for CEMon data files (LDIF) from active registered OSG resources, that are less than N minutes old (currently 10 minutes), and modifies the LDIF data to a format that is acceptable to the OSG BDII as well as the WLCG Interop BDII; The data is stored in $SERVER_ROOT/data/cemon_processed_osg and $SERVER_ROOT/data/cemon_processed_wlcg_interop respectively.
Data for OSG BDII: In particular, it adds the following lines to the top of each data file slated to be fed into the OSG BDII:
dn: mds-vo-name=CIT_ITB_1,mds-vo-name=local,o=grid objectClass: GlueTop objectClass: MDS mds-vo-name: CIT_ITB_1And it converts each dn: ...mds-vo-name=local,o=grid line to something like this:
dn: ... ,mds-vo-name=CIT_ITB_1,mds-vo-name=local,o=grid
Data for WLCG Interop BDII: The translator script adds the following lines to the top of each data file slated to be fed into the WLCG interop BDII:
dn: mds-vo-name=CIT_ITB_1,o=grid objectClass: GlueTop objectClass: MDS mds-vo-name: CIT_ITB_1And it converts each dn: ...mds-vo-name=local,o=grid line to something like this:
dn: GlueCEUniqueID=cithep201.ultralight.org:2119/jobmanager-condor-fermilab,mds-vo-name=CIT_ITB_1,o=gridFinally, it removes any non-Glue schema attribute i.e any lines that may have been added into CEMon by an OSG site to allow a tool like ReSS to publish additional information. At this time, valid data lines include lines that begin with the following:
dn: ... objectClass: ... GlueXYZ: ... mds-vo-name: ...
BDII Server
An information provider script grabs all the available BDII data and feeds into the two BDII servers running on port 2170 (OSG), and port 2180 (WLCG Interop). In the underlying BDII, the data is loaded into one of two ldap servers - one ldap server serves data while the other is being loaded. When the process is done, it pauses for a small amount of time (60 seconds) then starts again. The typical update time should be about 1 minute.OSG Resource Information (from OIM)
In the server schematic shown above, each element that needs to know the shortname, fqdn, etc. of a OSG resource does so by consulting the master list file -- which in turn is populated with data from OIM's resource catalog.
Interface to BDII
LDAP Queries on BDII Server
The LDAP servers are primarily accessed using ldap queries on the LDAP servers running on:
- Production OSG BDII: ldap://is.grid.iu.edu:2170
Production WLCG Interop BDII: ldap://is.grid.iu.edu:2180 - ITB OSG BDII: ldap://is-itb.grid.iu.edu:2170
ITB WLCG Interop BDII: ldap://is-itb.grid.iu.edu:2180 - Dev OSG BDII: ldap://is-dev.grid.iu.edu:2170
Dev WLCG Interop BDII: ldap://is-dev.grid.iu.edu:2180
An example command to pull data for a resource named SITENAME from the production OSG BDII would look like this:
ldapsearch -xLLL -p2170 -h is.grid.iu.edu -b mds-vo-name=SITENAME,mds-vo-name=local,o=grid
Data - Status Display
As a matter of convenience, a dynamically generated page containing the current information about the data file ages for the raw CEMon data, as well as the processed data that is fed into BDII, is provided at:
Please note that since all of these units run independently it is possible to see newer data in a feed than is currently being served or vice-versa.
