AccountingCampaign 2014 08

August 2014 Accounting Campaign - #AAROCAPEL2014

Campaign overview

• Leader: Bouchra Rahim (CNRST, MAGRID)
• Start: 11/08/2014
• End: 20/08/2014

Motivation

Accounting is a big deal - for many obvious reasons. While many sites have local accounting, a global, comprehensive accounting system is necessary for interoperability (and is a requirement of the EGI.eu interoperability).

The accounting system we are using is the APEL accounting system, which consists of a client installed on the sites, which interacts with the Compute Elements, reporting usage to a regional server. The regional server publishes data and summaries to the central EGI accounting service, which can be used to generate reports of usage.

Description of work to be done

• ROC : Deployment and integration of the regional accounting server.
• Sites : The APEL client needs to be installed on the CE (or a local machine which can interact with the CE).

The regional server has been installed and configured. After a very long and confusing period, we are successfully publishing data from at least 2 SAGrid sites (ZA-UJ, ZA-WITS-CORE) and one MAGRID site (MA-01-CNRST). There is a major difference between the APEL client for EMI/UMD2 and EMI/UMD3.

The procedure to follow is described by EGI at https://wiki.egi.eu/wiki/MAN09_Accounting_data_publishing

the simplest way is to install the apel client in the CE , Please find bellow a quick how-to :

• Install & configure apel client

  • yum install apel-ssm apel-lib apel-client

  • Install the apel client database (The schema is located at /usr/share/apel/client.sql)

  • edit the /etc/apel/client.cfg (you should specify the database settings and the site name)

  • edit the /etc/apel/sender.cfg (here are the setting used for AAROC :

       bdii: ldap://lcg-bdii.cern.ch:2170
    
       network: PROD
    
       use_ssl: false
    
       destination:  /queue/global.accounting.cpu.ZA-MERAKA
    

  • Set up a cron job to run the APEL parsers once per day.

• Install & configure apel parser

  • yum install apel-parsers
  • Apply the following fix if the version is 1.2.1-1 :
    replace lines (180 to 185 ) in /usr/bin/apelparser with the following :

                     try:
                         try:
                             fp = gzip.open(abs_file)
                             parsed, total = parse_file(parser, apel_db,
                                                        fp, reparse)
                         except IOError, e:  # not a gzipped file
                             fp = open(abs_file, 'r')
                             parsed, total = parse_file(parser, apel_db,
                                                        fp, reparse)

  • edit the /etc/apel/parser.cfg (you should specify the apel client database settings , the site name and the lrms and the blah log directories)
  • set up a cron job to run the APEL parsers once per day.

• Testing

Once the site configured , you can run the parser and the client manually and check the log files

/usr/bin/apelclient   -> /var/log/apel/client.log

/usr/bin/apelparser  -> /var/log/apelparser.log

• Add glite-apel endpoint in GOC-DB

Add the configured glite-apel endpoint to your site in the GOC-DB.

• References

For more information , please check the Apel client administration guides:

[APEL_Publisher_System_Administrator_Guide](* https://twiki.cern.ch/twiki/pub/EMI/EMI3APELClient/APEL_Publisher_System_Administrator_Guide.pdf)

APEL_Parsers_System_Administrator_Guide

• Server-side

The admin of the apel server will add the DN of the new apel client to list of accepted dns (/etc/apel/dns ) .