TWiki> SWT2 Web>LRCTesting (2007-07-09, PatrickMcGuigan)EditAttach

Local Replica Catalog (LRC) Testing

Components

Components of the LRC are:
  • MySQL server
  • HTTP server
  • CGI scripts
  • DQ2 installation
  • Python
  • mod_python
  • POOL/SEAL

Background

The LRC is a glue-up of several software items that can be frustrating when things change. Principally there are two interfaces into the LRC, although the second will, in turn, use the first. There is a MySQL server that contains the LRC data and provides one interface and there are CGI scripts within an HTTP server that provides methods for interacting with the LRC data

The LRC data is contained in a MySQL database named localreplicas and provides the mappings between LFN's/PFN's to GUID's as well as metadata for GUID's.

This database is accessed by serveral customers: DQ2 (through site services and client commands), Panda (pilots, central monitoring) and Nagios (centralized monitoring). Dq2 talks directly to the MySQL server while Panda uses the HTTP interface. The Nagios monitoring use both MySQL and HTTP to determine the availability of the LRC system.

DQ2 always uses direct MySQL interaction (via python's MySQLdb module) to operate on the LRC. Remote DQ2 clients and sites use the MySQL account published in the ToA to query your LRC to see if you have a particular file and what URL can be used to retrieve the file. Your local DQ2 siteservices installation should use a different, i.e. unpublished, MySQL account that can write to the database. How DQ2 finds this unpublished account differs between v0.2 and v0.3 and is discussed further below.

Panda components (central monitor and pilots) always use the HTTP interface when accessing LRC through several CGI scripts. The CGI scripts will then access the localreplicas database through MySQL calls. Two of the CGI scripts perform reads/writes on the localreplicas database and they need to use the unpublished account to access the MySQL server. The CGI scripts actually use DQ2 components to retrieve the unpublished MySQL account. In addition, the CGI scripts will use DQ2 code to retrieve a default storage location.

A recent change in the LRC has been the introduction of centralized monitoring of the LRC subsystems (MySQL and the HTTP interface) via Nagios. The MySQL test involves connecting to the MySQL system using a well known account (private password) from the central host www.atlasgrid.bnl.gov. The HTTP testing involves checking the results of a standard interaction with one of the CGI scripts.

Testing

There are several hosts to consider while testing and include:
  1. MySQL server host
  2. DQ2 siteservices host(s)
  3. HTTP server host
  4. Nagios monitoring host
  5. An outside host

If multiple DQ2 sites were to use a single LRC and each site used a different host for running siteservices, then item 2) above would represent different machines.

MySQL Testing

This testing is meant to be used to verify that the MySQL server is correctly setup and to determine that the different accounts have the correct access from the appropriate machines.

This assumes the following:

  1. The software is installed.
  2. The MySQL server is listening on its port (most likely 3306)
  3. You have access to the root MySQL account
  4. The localreplicas database is created with the correct schema
  5. You have created the published, unpublished, and monitoring accounts

By convention, the published account is dq2user, but it is always a good idea to verify what the ToA contains for your site and verify that you have the correct account.

In the testing examples below, the account name lrc_writer is used for the unpublished account. This may differ on your installation.

Verfify accounts

This test is meant to verify that you have the correct accounts created with permissions to the correct hosts. 

[mcguigan@gk03 mcguigan]$ mysql -u root -p -h `hostname` mysql -e 'select user,host,password from user'
Enter password: 
+------------+-----------------------+------------------+
| user       | host                  | password         |
+------------+-----------------------+------------------+
| root       | localhost             | XXXXXXXXXXXXXXXX | 
| root       | gk03.swt2.uta.edu     | XXXXXXXXXXXXXXXX | 
| root       | 127.0.0.1             |                  | 
| dq2user    | localhost             | XXXXXXXXXXXXXXXX | 
| dq2user    | %                     | XXXXXXXXXXXXXXXX | 
| lrc_writer | localhost             | XXXXXXXXXXXXXXXX | 
| lrc_writer | gk03.swt2.uta.edu     | XXXXXXXXXXXXXXXX | 
| lrc_writer | gk02.swt2.uta.edu     | XXXXXXXXXXXXXXXX | 
| monitor    | www.atlasgrid.bnl.gov | XXXXXXXXXXXXXXXX | 
+------------+-----------------------+------------------+

The output above indicates that the published account (dq2user) can access the system from any host while the unpublished account (lrc_writer) can access the system from the local machine (gk03.swt2.uta.edu) and the DQ2 siteservices machine (gk02.swt2.uta.edu) and that the Nagios account (monitor) can only access the system from one machine at BNL. On this system, both the HTTP service and the MySQL service run on the gk03 machine.

You can verify the rights of each account with another mysql command 

[mcguigan@gk03 mcguigan]$ mysql -u root -p -h `hostname` mysql -e "select * from db where Db='localreplicas'"
Enter password: 
+-----------------------+---------------+------------+-------------+-------------+-------------+-------------+-------------+-----------+------------+-----------------+------------+------------+-----------------------+------------------+------------------+----------------+---------------------+--------------------+--------------+
| Host                  | Db            | User       | Select_priv | Insert_priv | Update_priv | Delete_priv | Create_priv | Drop_priv | Grant_priv | References_priv | Index_priv | Alter_priv | Create_tmp_table_priv | Lock_tables_priv | Create_view_priv | Show_view_priv | Create_routine_priv | Alter_routine_priv | Execute_priv |
+-----------------------+---------------+------------+-------------+-------------+-------------+-------------+-------------+-----------+------------+-----------------+------------+------------+-----------------------+------------------+------------------+----------------+---------------------+--------------------+--------------+
| localhost             | localreplicas | dq2user    | Y           | N           | N           | N           | N           | N         | N          | N               | N          | N          | N                     | N                | N                | N              | N                   | N                  | N            | 
| %                     | localreplicas | dq2user    | Y           | N           | N           | N           | N           | N         | N          | N               | N          | N          | N                     | N                | N                | N              | N                   | N                  | N            | 
| localhost             | localreplicas | lrc_writer | Y           | Y           | Y           | Y           | Y           | Y         | N          | Y               | Y          | Y          | Y                     | Y                | Y                | Y              | Y                   | Y                  | Y            | 
| gk03.swt2.uta.edu     | localreplicas | lrc_writer | Y           | Y           | Y           | Y           | Y           | Y         | N          | Y               | Y          | Y          | Y                     | Y                | Y                | Y              | Y                   | Y                  | Y            | 
| gk02.swt2.uta.edu     | localreplicas | lrc_writer | Y           | Y           | Y           | Y           | Y           | Y         | N          | Y               | Y          | Y          | Y                     | Y                | Y                | Y              | Y                   | Y                  | Y            | 
| www.atlasgrid.bnl.gov | localreplicas | monitor    | Y           | N           | N           | N           | N           | N         | N          | N               | N          | N          | N                     | N                | N                | N              | N                   | N                  | N            | 
+-----------------------+---------------+------------+-------------+-------------+-------------+-------------+-------------+-----------+------------+-----------------+------------+------------+-----------------------+------------------+------------------+----------------+---------------------+--------------------+--------------+

The above shows that only the lrc_writer account has complete privileges with respect to the localreplicas database. The dq2user only has the SELECT privilege. In this LRC, the monitor account has been granted SELECT on localreplicas. This is more access than originally requested by Yuri when setting up the monitor account, so your system may not have this account listed.

Published Account Access

The next tests involve verifying that the published account can actually talk to the MySQL server and access the localreplicas database from the correct locations. The test below looks for data in the t_metaspec table which should always contain these four items, regardless of the number of files in the LRC.

There may be some sites that do not allow general access to the MySQL port. Your LRC, as a whole, will still operate with respect to your local cluster but remote DQ2 clients and sites will not be able to fetch files from your site. In addition, Nagios monitoring will not be able to determine the availability of your MySQL installation if the MySQL port is blocked to BNL.

MySQL server host
From the host running the MySQL server. 
[mcguigan@gk03 mcguigan]$ mysql -u dq2user -p -h gk03.swt2.uta.edu localreplicas -e "select * from t_metaspec"
Enter password: 
+--------------+---------+
| colname      | coltype |
+--------------+---------+
| md5sum       | string  | 
| fsize        | string  | 
| lastmodified | string  | 
| archival     | string  | 
+--------------+---------+

Foreign host
This test should repeat the same test as above from a machine outside of as many firewalls as possible. In this case from a BNL machine. 
[acas0008] ~ > mysql -u dq2user -p -h gk03.swt2.uta.edu localreplicas -e "select * from t_metaspec"
Enter password:
+--------------+---------+
| colname      | coltype |
+--------------+---------+
| md5sum       | string  |
| fsize        | string  |
| lastmodified | string  |
| archival     | string  |
+--------------+---------+
If this test fails, but the first test succeeds you should first look at firewall configurations and see which machines can access the MySQL port 3306.

Unpublished account

You should verify that the account that can write to the localreplicas database can connect to the MySQL server from the appropriate machines. The minimum set of machines would be the host running DQ2 siteservices and the host running the HTTP server. There may be additional hosts if your LRC is supporting multiple sites. You may need an additional host if your MySQL server and HTTP server are on different machines. These test do not verify that the unpublished account can actually write to the database. It assumed simply connecting to the DB is sufficient if the privileges have been verified (see above).

HTTP/MySQL host 
[mcguigan@gk03 mcguigan]$ mysql -u lrc_writer -p -h `hostname` localreplicas -e "select * from t_metaspec"
Enter password: 
+--------------+---------+
| colname      | coltype |
+--------------+---------+
| md5sum       | string  | 
| fsize        | string  | 
| lastmodified | string  | 
| archival     | string  | 
+--------------+---------+

DQ2 Siteservices host 
[mcguigan@gk02 ~]$  mysql -u lrc_writer -p -h gk03.swt2.uta.edu localreplicas -e "select * from t_metaspec"
Enter password: 
+--------------+---------+
| colname      | coltype |
+--------------+---------+
| md5sum       | string  | 
| fsize        | string  | 
| lastmodified | string  | 
| archival     | string  | 
+--------------+---------+

Foreign host
This verifies that some random host can not connect to the localreplicas database as the unpublished user. It also indicates the type of error you would expect to see if the user table in the mysql DB was not properly setup for the DQ2 siteservices host or the HTTP server host. 
[acas0008] ~ > mysql -u lrc_writer -p -h gk03.swt2.uta.edu localreplicas -e "select * from t_metaspec"
Enter password:
ERROR 1045 (28000): Access denied for user 'lrc_writer'@'130.199.48.58' (using password: YES)

If the above is working, then DQ2 siteservices should be able to use the LRC and deliver files correctly to your site. There are additional tests for the HTTP service to verify that the site can be used by the Panda production system.

HTTP Server Testing

The HTTP service is used to provide a simplified interface for the Panda interface that is free from the direct use of MySQL . The HTTP interface must be available to the worker nodes of a cluster so that Panda jobs can succeed, and it is recommended to allow HTTP access to the Panda Monitor and the Nagios testing host. Panda jobs determine how to contact the http server from an entry in the siteinfo.py file used on the Panda submit host. For these tests, we are using the siteinfo.py entry for UTA_SWT2 which lists the HTTP interface as http://gk03.swt2.uta.edu:8000/dq2

Testing of the HTTP interface is best done by calling specific methods of the CGI scripts and comparing the output to expected results.

isAlive

This is the most basic test of the HTTP interface. It provides confidence that the HTTP interface is callable, The CGI scripts are accessible and that mod_python is functioning correctly.

HTTP Server 
[mcguigan@gk03 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/monitor/isAlive
isAlive=yes
Worker Node 
[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/monitor/isAlive
isAlive=yes

In our case, because the siteinfo.py lists the public host name of our HTTP server, the http request is sent through the cluster's NAT (Network Address Translation) host and the access log will indicate the request came from the NAT host. If the first test works correctly, but the second fails. You should check for firewall restrictions on port 8000 of the HTTP server and potential issues with the cluster NAT/Router in conjunction.

Default Storage

This test checks the HTTP server's ability to retrieve information from the TiersOfATLAS file used by DQ2. This method of the LRC is used by pilots to determine a location for storing output/log files. The information is retrieved by calling a DQ2 function that returns the srm attribute associated with the site. 
[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/storages/default
gsiftp://gk01.swt2.uta.edu/ifs1/dq2_cache/storageA/

Things to check if the test fails:

  • Check how the site is passed to the dq2 code within the storages CGI script
  • Check that the dq2 siteservices code is importable in the python environment created within mod_python
  • Check the TiersOfATLASCache .py file to verify the definition of the srm attribute of the site.

PoolFileCatalog

This checks that your HTTP system can retrieve the correct MySQL account information from DQ2, use the POOL software to retrieve information from the localreplicas database and return the results. Panda pilots use this method to determine the location of input files before using them. This method takes a parameter that dictates whether to search the LRC by GUID or by LFN. Additionally, the method allows the caller to search for more than one file at a time.

Determining if the output is correct for these tests requires the use of known test cases; i.e. you want to search for files that are known to exist in the LRC. The Nagios monitoring of the HTTP interface relies on the existence of a known file and the retrieval of a POOLFileCatalog document. If this file is not installed on your LRC, you can retrieve example LFN's/GUID's by using the MySQL query: 

mysql -u dq2user -p -h  localreplicas -e "select * from t_lfn limit 5"

No File
First, a test for a file that does not exist in the LRC by GUID and LFN: 

[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/PoolFileCatalog?guids=000
Error. GUIDs not found[mcguigan@compute-1-1 mcguigan]$ 

[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/PoolFileCatalog?lfns=fakefile
Error. LFNs not found[mcguigan@compute-1-1 mcguigan]$

Note: The output shows the returned information does not contain a trailing new0line character.

One file by GUID
This test uses the same file expected by the Nagios monitoring of the HTTP interface 
[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/PoolFileCatalog?guids=b61eb577-2f19-476c-b2f4-f491ed48c220
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<!-- Edited By POOL -->
<!DOCTYPE POOLFILECATALOG SYSTEM "InMemory">
<POOLFILECATALOG>

  <META name="fsize" type="string"/>

  <META name="md5sum" type="string"/>

  <META name="lastmodified" type="string"/>

  <META name="archival" type="string"/>

  <File ID="b61eb577-2f19-476c-b2f4-f491ed48c220">
    <physical>
      <pfn filetype="" name="gsiftp://gk01.swt2.uta.edu/ifs1/dq2_cache/storageA/user/user.HironoriIto.LRC_Alive2/user.HironoriIto.LRC_Alive2.txt.1"/>
    </physical>
    <logical>
      <lfn name="user.HironoriIto.LRC_Alive2.txt.1"/>
    </logical>
    <metadata att_name="archival" att_value="V"/>
    <metadata att_name="fsize" att_value="33"/>
    <metadata att_name="lastmodified" att_value="1183607393"/>
    <metadata att_name="md5sum" att_value="c98270b7f758904eb5a0f52adb5ac300"/>
  </File>

</POOLFILECATALOG>

If this test fails, as opposed to not finding the file, you should check:

  • That the POOL software is configured correctly in the environment before the HTTP server is started
  • That the mysql account used by HTTP interface is correct (see below)
  • That /tmp is not full

If this test passes, it generally means that the HTTP interface is processing queries of the LRC correctly. The remaining tests of the PoolFileCatalog method are shown for completeness and examples of debugging.

One file by LFN
This test uses the same file as the previous test but searches by LFN 
[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/PoolFileCatalog?lfns=user.HironoriIto.LRC_Alive2.txt.1
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<!-- Edited By POOL -->
<!DOCTYPE POOLFILECATALOG SYSTEM "InMemory">
<POOLFILECATALOG>

  <META name="fsize" type="string"/>

  <META name="md5sum" type="string"/>

  <META name="lastmodified" type="string"/>

  <META name="archival" type="string"/>

  <File ID="b61eb577-2f19-476c-b2f4-f491ed48c220">
    <physical>
      <pfn filetype="" name="gsiftp://gk01.swt2.uta.edu/ifs1/dq2_cache/storageA/user/user.HironoriIto.LRC_Alive2/user.HironoriIto.LRC_Alive2.txt.1"/>
    </physical>
    <logical>
      <lfn name="user.HironoriIto.LRC_Alive2.txt.1"/>
    </logical>
    <metadata att_name="archival" att_value="V"/>
    <metadata att_name="fsize" att_value="33"/>
    <metadata att_name="lastmodified" att_value="1183607393"/>
    <metadata att_name="md5sum" att_value="c98270b7f758904eb5a0f52adb5ac300"/>
  </File>

</POOLFILECATALOG>
Multiple Files By GUID
This example searches for two files based on GUID. Not that multiple GUID's are separated by the "+" character. 

[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/PoolFileCatalog?guids=6a17dfe4-9697-45b1-b632-02e2b8b3524e+86f92af2-f1d3-4786-bcf8-0127958ae005
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<!-- Edited By POOL -->
<!DOCTYPE POOLFILECATALOG SYSTEM "InMemory">
<POOLFILECATALOG>

  <META name="fsize" type="string"/>

  <META name="md5sum" type="string"/>

  <META name="lastmodified" type="string"/>

  <META name="archival" type="string"/>

  <File ID="86f92af2-f1d3-4786-bcf8-0127958ae005">
    <physical>
      <pfn filetype="" name="gsiftp://gk01.swt2.uta.edu/ifs1/dq2_cache/storageA/other/D/DBRelease-3.1.1.tar.gz"/>
    </physical>
    <logical>
      <lfn name="DBRelease-3.1.1.tar.gz"/>
    </logical>
    <metadata att_name="archival" att_value="P"/>
    <metadata att_name="fsize" att_value="53619987"/>
    <metadata att_name="lastmodified" att_value="1170253145"/>
    <metadata att_name="md5sum" att_value="None"/>
  </File>

  <File ID="6a17dfe4-9697-45b1-b632-02e2b8b3524e">
    <physical>
      <pfn filetype="" name="gsiftp://gk01.swt2.uta.edu/ifs1/dq2_cache/storageA/testpanda/testpanda.3a6b0ecf-5783-46f7-b5d0-ed20cfcc6b9d_dis2125023/DBRelease-4.0.1.tar.gz"/>
    </physical>
    <logical>
      <lfn name="DBRelease-4.0.1.tar.gz"/>
    </logical>
    <metadata att_name="archival" att_value="V"/>
    <metadata att_name="fsize" att_value="130511613"/>
    <metadata att_name="lastmodified" att_value="1183760138"/>
    <metadata att_name="md5sum" att_value="None"/>
  </File>

</POOLFILECATALOG>
Multiple Files By LFN
Looking for the same files as the previous example but by LFN. Again, multiple LFN's are separated by the "+" character. 

[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/PoolFileCatalog?lfns=DBRelease-4.0.1.tar.gz+DBRelease-3.1.1.tar.gz    
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<!-- Edited By POOL -->
<!DOCTYPE POOLFILECATALOG SYSTEM "InMemory">
<POOLFILECATALOG>

  <META name="fsize" type="string"/>

  <META name="md5sum" type="string"/>

  <META name="lastmodified" type="string"/>

  <META name="archival" type="string"/>

  <File ID="6a17dfe4-9697-45b1-b632-02e2b8b3524e">
    <physical>
      <pfn filetype="" name="gsiftp://gk01.swt2.uta.edu/ifs1/dq2_cache/storageA/testpanda/testpanda.3a6b0ecf-5783-46f7-b5d0-ed20cfcc6b9d_dis2125023/DBRelease-4.0.1.tar.gz"/>
    </physical>
    <logical>
      <lfn name="DBRelease-4.0.1.tar.gz"/>
    </logical>
    <metadata att_name="archival" att_value="V"/>
    <metadata att_name="fsize" att_value="130511613"/>
    <metadata att_name="lastmodified" att_value="1183760138"/>
    <metadata att_name="md5sum" att_value="None"/>
  </File>

  <File ID="86f92af2-f1d3-4786-bcf8-0127958ae005">
    <physical>
      <pfn filetype="" name="gsiftp://gk01.swt2.uta.edu/ifs1/dq2_cache/storageA/other/D/DBRelease-3.1.1.tar.gz"/>
    </physical>
    <logical>
      <lfn name="DBRelease-3.1.1.tar.gz"/>
    </logical>
    <metadata att_name="archival" att_value="P"/>
    <metadata att_name="fsize" att_value="53619987"/>
    <metadata att_name="lastmodified" att_value="1170253145"/>
    <metadata att_name="md5sum" att_value="None"/>
  </File>
</POOLFILECATALOG>

Files (Advanced)

This method allows the caller to insert information into the LRC via the HTTP interface. Some care should be used with this method as you should ensure that fake data added to the LRC should be removed when done with testing. Panda pilots use this call to register output/log files in the LRC.

The files method does not perform extensive validation, thus, making it possible to register completely false data. The example below registers a file in the LRC at gk03.swt2.uta.edu but the PFN provides an URL for storage at OU. Further this entry uses a PFN that is longer than 256 characters. It is possible to reuse the command below and only change the host gk03.swt2.uta.edu to your HTTP host. 

[mcguigan@compute-1-1 mcguigan]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/files? -d 'guids=00000000001&fsizes=12345&lfns=wood&md5sums=01010101&pfns=gsiftp%3A%2F%2Fouhep00.nhn.ou.edu%2Fraid2%2Fdq2-cache%2Fdata%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2Fabcdefghijklmnopqrstuvwxyz%2F&archivals=V'
1[mcguigan@compute-1-1 mcguigan]$

Note the return value is a "1" without a trailing newline.

You can verify the file registration by performing a search based on the GUID (00000000001) or by LFN (wood).

To remove the fake data from the LRC, you can use the following commands: 

[mcguigan@gk03 dq2]$ mysql -u lrc_writer -p -h gk03.swt2.uta.edu localreplicas -e "delete from t_pfn where guid=00000000001"
Enter password: 
[mcguigan@gk03 dq2]$ mysql -u lrc_writer -p -h gk03.swt2.uta.edu localreplicas -e "delete from t_lfn where guid=00000000001"
Enter password: 
[mcguigan@gk03 dq2]$ mysql -u lrc_writer -p -h gk03.swt2.uta.edu localreplicas -e "delete from t_meta where guid=00000000001"
Enter password:

Debugging

The following are some ways that you can debug items related to the LRC. These allow you to partially inspect some of the internals of the HTTP interface. The LRC relies on a number of software packages to operate correctly and determining why something is not working correctly can be a frustrating task.

endpoint

This method can be used to determine the account/password that the HTTP interface will use when contacting the MySQL server. The lrc.py CGI script contains a method that is normally commented out. This prevents outside users from discovering the LRC write MySQL account details. However, you can uncomment this method and call the method to verify the correct account is being used when the HTTP interface is accessing the database. The method should look like the following: 
def endpoint(req):
    """Returns site's LRC endpoint"""
    return TiersOfATLAS.getLocalCatalog(siteid) + '\n'

To use this method, one simply calls the method using: 

[mcguigan@gk03 dq2]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/endpoint
mysql://lrc_writer:XXXXXXXX@gk03.swt2.uta.edu:3306/localreplicas

Environment Variables

It is possible to add some additional methods to the CGI scripts to provide additional detail about the environment that the CGI script executes from. As an example you could use the following method to dump the environment variables and their values. This can be helpful in verifying that the POOL software is available in PATH and LD_LIBRARY_PATH. 

def debug(req):
    retval=""
    for i in os.environ.keys():
        retval = retval + "%s=%s\n" % (i, os.environ[i])
    return retval

A call to this method would look like (Your system's output will vary somewhat): 

[mcguigan@gk03 dq2]$ curl http://gk03.swt2.uta.edu:8000/dq2/lrc/debug
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
SEAL_KEEP_MODULES=true
SCRAMRT_SEAL_KEEP_MODULES=true
ODBCINST=/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323
ODBCSYSINI=/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323/etc
SSH_CLIENT=129.107.255.3 59130 22
SCRAMRT_ODBCINST=/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323
LOGNAME=atlasddm
USER=atlasddm
INPUTRC=/etc/inputrc
HOME=/ifs1/home/atlasddm
PATH=/opt/lrc/lrc/externals/Python/2.3.4/slc3_ia32_gcc323/bin:/opt/lrc/mysql/mysql-5.0.41-linux-i686/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/ifs1/home/atlasddm/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/gccxml/0.6.0_patch2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/mysql/4.0.24/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/oracle/10.1.0.3-1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/sqlite/3.2.1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/Python/2.3.4/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/root/4.04.02f/slc3_ia32_gcc323/root/bin:/opt/lrc/lrc/pool/external/lfc/1.3.4/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/Oval/3_5_0/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/valgrind/2.4.0/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/QMtest/2.2.1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib:/opt/lrc/lrc/pool/app/spi/scram:/opt/lrc/lrc/externals/Python/2.3.4/slc3_ia32_gcc323/bin:/opt/lrc/mysql/mysql-5.0.41-linux-i686/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/ifs1/home/atlasddm/bin
SCRAMRT_LD_LIBRARY_PATH=/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/uuid/1.32/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/Boost/1.32.0/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/mysql++/1.7.31_mysql.4.0.24/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/mysql/4.0.24/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/myodbc/3.51.06_mysql.4.0.24/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/oracle/10.1.0.3-1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/sqlite/3.2.1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/Python/2.3.4/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/root/4.04.02f/slc3_ia32_gcc323/root/lib:/opt/lrc/lrc/pool/external/XercesC/2.3.0/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/lfc/1.3.4/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/CppUnit/1.8.0/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/pcre/4.4/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/glite/1.1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/vdt/1.2.3_globusrls215b/slc3_ia32_gcc323/globus/lib:/opt/lrc/lrc/pool/external/cgsigsoap/2.6-1.1.9/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib
LD_LIBRARY_PATH=/opt/lrc/lrc/httpd/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/uuid/1.32/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/Boost/1.32.0/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/mysql++/1.7.31_mysql.4.0.24/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/mysql/4.0.24/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/myodbc/3.51.06_mysql.4.0.24/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/oracle/10.1.0.3-1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/sqlite/3.2.1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/Python/2.3.4/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/root/4.04.02f/slc3_ia32_gcc323/root/lib:/opt/lrc/lrc/pool/external/XercesC/2.3.0/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/lfc/1.3.4/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/CppUnit/1.8.0/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/pcre/4.4/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/glite/1.1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/vdt/1.2.3_globusrls215b/slc3_ia32_gcc323/globus/lib:/opt/lrc/lrc/pool/external/cgsigsoap/2.6-1.1.9/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib:/opt/lrc/lrc/externals/Python/2.3.4/slc3_ia32_gcc323/lib:/opt/lrc/mysql/mysql-5.0.41-linux-i686/lib
SSH_CONNECTION=129.107.255.3 59130 129.107.255.4 22
LANG=en_US.UTF-8
SCRAMRT_ODBCSYSINI=/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323/etc
SHELL=/bin/bash
SHLVL=2
SCRAMRT_TNS_ADMIN=/afs/cern.ch/project/oracle/admin
G_BROKEN_FILENAMES=1
SCRAMRT_PATH=/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/gccxml/0.6.0_patch2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/unixodbc/2.2.6/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/mysql/4.0.24/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/oracle/10.1.0.3-1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/sqlite/3.2.1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/Python/2.3.4/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/root/4.04.02f/slc3_ia32_gcc323/root/bin:/opt/lrc/lrc/pool/external/lfc/1.3.4/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/Oval/3_5_0/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/valgrind/2.4.0/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/QMtest/2.2.1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib
HISTSIZE=1000
DQ2_SITE=UTA_SWT2
SCRAMRT_PYTHONPATH=/opt/lrc/lrc/pool/external/Boost/1.32.0/slc3_ia32_gcc323/lib/python2.3/site-packages:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/python:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib
TNS_ADMIN=/afs/cern.ch/project/oracle/admin
SCRAMRT_OVAL_FLAVOR=Linux
SCRAMRT_ROOTSYS=/opt/lrc/lrc/pool/external/root/4.04.02f/slc3_ia32_gcc323/root
EDITOR=emacs
SEAL_PLUGINS=/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib/modules
POOL_VERSION=POOL_2_1_2
LRC_LOCATION=/opt/lrc/lrc
PYTHONPATH=/opt/lrc/lrc/pool/external/Boost/1.32.0/slc3_ia32_gcc323/lib/python2.3/site-packages:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/python:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/bin:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib:/opt/lrc/lrc/python_libs:/opt/lrc/lrc/htdocs/dq2:/opt/lrc/dq2/config/UTA_SWT2:/opt/lrc/dq2/DQ2:
TERM=vt100
PAC_ANCHOR=/opt/lrc/lrc
LOCALRT=/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2
NLS_DATE_FORMAT=DD-MON-FXYYYY
LRC_APACHE=/opt/lrc/lrc/httpd
LESSOPEN=|/usr/bin/lesspipe.sh %s
OVAL_FLAVOR=Linux
_=/opt/lrc/lrc/httpd/bin/httpd
NLS_LANG=american_america.WE8ISO8859P9
LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
SCRAMRT_SEAL_PLUGINS=/opt/lrc/lrc/pool/external/../app/releases/SEAL/SEAL_1_7_1/slc3_ia32_gcc323/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/tests/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib/modules:/opt/lrc/lrc/pool/app/releases/POOL/POOL_2_1_2/slc3_ia32_gcc323/examples/lib/modules
SCRAM_ARCH=slc3_ia32_gcc323
SCRAMRT_NLS_DATE_FORMAT=DD-MON-FXYYYY
SSH_TTY=/dev/pts/0
HOSTNAME=gk03.swt2.uta.edu
ROOTSYS=/opt/lrc/lrc/pool/external/root/4.04.02f/slc3_ia32_gcc323/root
PWD=/ifs1/home/atlasddm
MAIL=/var/spool/mail/atlasddm
SCRAMRT_NLS_LANG=american_america.WE8ISO8859P9
POOL_OUTMSG_LEVEL=8

-- PatrickMcGuigan - 06 Jul 2007

Edit | Attach | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r4 - 2007-07-09 - 21:43:18 - PatrickMcGuigan
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback