This web site should be used for installations up to gLite 3.2. For EMI installations please refer to the new web site https://wiki.italiangrid.it/CREAM

Software Description

The glite-SGE_utils release integrates SGE LRMS with CREAMCE (Version 1.6) in SL5, x86_64. The CREAMCE integration consists in setting up the BLAH configuration to interoperate with SGE. For BLAH to work, the following set of SGE scripts and binaries should be available under /opt/glite/bin:

• BUpdaterSGE
• sge_helper
• sge_hold.sh
• sge_submit.sh
• sge_resume.sh
• sge_status.sh
• sge_cancel.sh

These scripts are installed by glite-ce-blahp rpm (but the same files can also be downloaded from http://www.egee.cesga.es/cream/releases/0.60/bin_0.60.tar.gz

Recommendations

• It is assumed that you already have a running QMASTER and gLite WNs configured like sgeexecd hosts. Please remember to use the same queue

names under the QUEUES variable of your site-info.def during CREAMCE+SGE_utils YAIM configuration.

• The CREAMCE must be installed in a separate node from the SGE QMASTER, and the same SGE software version should be used in all services.
• Some sites use SGE installations shared via NFS (or equivalent) in the CE. In order to prevent changes in that setup when YAIM is executed, define SGE_SHARED_INSTALL=yes in your site-info.def, otherwise YAIM may change your setup according to the definitions in your site-info.def.

Installation and Configuration

• In the CreamCE:
  • [root ~]# yum install glite-SGE_utils
  • Setup relevant YAIM variables:

  CREAM_DB_USER="cream"
  CREAM_DB_PASSWORD="cream"
  CEMON_HOST="ce01.lip.pt"
  BLPARSER_WITH_UPDATER_NOTIFIER="true"
  JOB_MANAGER=sge
  CE_BATCH_SYS=sge
  SGE_ROOT=<Path to your SGE installation>. Default: "/usr/local/sge/pro"
  SGE_CELL= <Path to your SGE CELL>. Default: "default"
  SGE_QMASTER=<SGE QMASTER PORT>. Default: "536"
  SGE_EXECD=<SGE EXECD PORT>. Defaul: "537"
  SGE_SPOOL_METH="classic"
  BATCH_SERVER=<FQDN of your QMASTER>
  BATCH_LOG_DIR=<Path for the SGE accounting file>.
  BATCH_BIN_DIR=<Path for the SGE binaries>
  BATCH_VERSION=<SGE version>
  

  • [root ~]# /opt/glite/yaim/bin/yaim -c -s /root/site-cfg/siteinfo/site-info-egee.def -n creamCE -n SGE_utils
  • Remarks:
    • Don't be afraid of the message INFO: BATCH SYSTEM = sge. Does not execute config_cream_blah!: it is normal
    • Check that the glite-SGE_utils installation also fetches perl-XML-LibXML, perl-XML-Simple, perl-XML-Twig
    • glite-apel-sge-2.0.8-1.noarch is a noarch package and depends on jdk because it can be also used in SL4 (where you only have jdk available). The problem is that jdk is not available in SLC5. Apel Support has been contacted and they have raised this question to EMT. The jdk installation will solve that dependency issue. You may install it for the SL5 repositories, for example (rpm -ivh http://ftp.scientificlinux.org/linux/scientific/5x/x86_64/SL/jdk-1.6.0_16-fcs.x86_64.rpm)
    • Check that Bnotifier and BUpdaterSGE daemons start properly
• In the WNs:
  • If you do not have your cluster homes shared via NFS, the transferring of files between WN and CE is handled by a script, called sge_filestaging. That script is available in your CreamCE:/opt/glite/bin/sge_filestaging, and should be copied to all your WNs:/opt/glite/bin/sge_filestaging).
  • Remarks:
    • If you have used glite-yaim-sge-client (>= 4.1.1-3) to configure your SGE client in the WN , /opt/glite/bin/sge_filestaging should be already in place in your WNs
    • For this copy mechanism to work, passwordless scp must be configured from WN to your CreamCE. YAIM will not take care of that task on your behalf.
• In SGE QMASTER:
  • The CreamCE machine must be set as a submission machine (use qconf -as <CE.MY.DOMAIN> in the QMASTER to set it up). If possible, the SGE QMASTER configuration should have the definition execd_params INHERIT_ENV=false (use qconf -mconf to set it up). This setting allows to propagate the environment of the submission machine (CE) into the execution machine (WN).
  • Add /opt/glite/bin/sge_filestaging --stagein and /opt/glite/bin/sge_filestaging --stageout to your prolog and epilogue scripts, respectively in SGE global configuration (use qconf -mconf) or in each queue configuration qconf -mq <QUEUE>.

Info provided by Goncalo Borges <goncalo@lip.pt>

More information available in the SGE CookBook