content/api-docs/master/compute_resource

<!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8" /> <link href="style.css" rel="stylesheet" type="text/css"/> <title>Thrift module: compute_resource_model</title></head><body> <div class="container-fluid"> <h1>Thrift module: compute_resource_model</h1> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Module</th><th>Services</th><th>Data types</th><th>Constants</th></tr></thead><tbody> <tr> <td>compute_resource_model</td><td></td> <td><a href="#Struct_BatchQueue">BatchQueue</a><br/> <a href="#Struct_CloudJobSubmission">CloudJobSubmission</a><br/> <a href="#Struct_ComputeResourceDescription">ComputeResourceDescription</a><br/> <a href="#Enum_DMType">DMType</a><br/> <a href="#Enum_FileSystems">FileSystems</a><br/> <a href="#Struct_GlobusJobSubmission">GlobusJobSubmission</a><br/> <a href="#Enum_JobManagerCommand">JobManagerCommand</a><br/> <a href="#Struct_JobSubmissionInterface">JobSubmissionInterface</a><br/> <a href="#Enum_JobSubmissionProtocol">JobSubmissionProtocol</a><br/> <a href="#Struct_LOCALSubmission">LOCALSubmission</a><br/> <a href="#Enum_MonitorMode">MonitorMode</a><br/> <a href="#Enum_ProviderName">ProviderName</a><br/> <a href="#Struct_ResourceJobManager">ResourceJobManager</a><br/> <a href="#Enum_ResourceJobManagerType">ResourceJobManagerType</a><br/> <a href="#Struct_SSHJobSubmission">SSHJobSubmission</a><br/> <a href="#Struct_UnicoreJobSubmission">UnicoreJobSubmission</a><br/> </td> <td></td> </tr></tbody></table> <hr/><h2 id="Enumerations">Enumerations</h2> <div class="definition"><h3 id="Enum_ResourceJobManagerType">Enumeration: ResourceJobManagerType</h3> <pre>* Enumeration of local resource job manager types supported by Airavata * * FORK: * Forking of commands without any job manager * * PBS: * Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. * * SLURM: * The Simple Linux Utility for Resource Management is a open source workload manager. * * UGE: * Univa Grid Engine, a variation of PBS implementation. * * LSF: * IBM Platform Load Sharing Facility is dominantly installed on IBM clusters. * </pre><br/><br/><table class="table-bordered table-striped table-condensed"> <tr><td><code>FORK</code></td><td><code>0</code></td><td> </td></tr> <tr><td><code>PBS</code></td><td><code>1</code></td><td> </td></tr> <tr><td><code>SLURM</code></td><td><code>2</code></td><td> </td></tr> <tr><td><code>LSF</code></td><td><code>3</code></td><td> </td></tr> <tr><td><code>UGE</code></td><td><code>4</code></td><td> </td></tr> <tr><td><code>CLOUD</code></td><td><code>5</code></td><td> </td></tr> <tr><td><code>AIRAVATA_CUSTOM</code></td><td><code>6</code></td><td> </td></tr> </table></div> <div class="definition"><h3 id="Enum_JobManagerCommand">Enumeration: JobManagerCommand</h3> <pre>Enumeration of resource job manager commands SUBMISSION: Ex: qsub, sbatch JOBMONITORING: Ex: qstat, squeue DELETION: Ex: qdel, scancel CHECK_JOB: Detailed Status about the Job. Ex: checkjob SHOW_QUEUE: List of Queued Job by the schedular. Ex: showq SHOW_RESERVATION: List all reservations. Ex:showres, show_res SHOW_START: Display the start time of the specified job. Ex: showstart </pre><br/><br/><table class="table-bordered table-striped table-condensed"> <tr><td><code>SUBMISSION</code></td><td><code>0</code></td><td> </td></tr> <tr><td><code>JOB_MONITORING</code></td><td><code>1</code></td><td> </td></tr> <tr><td><code>DELETION</code></td><td><code>2</code></td><td> </td></tr> <tr><td><code>CHECK_JOB</code></td><td><code>3</code></td><td> </td></tr> <tr><td><code>SHOW_QUEUE</code></td><td><code>4</code></td><td> </td></tr> <tr><td><code>SHOW_RESERVATION</code></td><td><code>5</code></td><td> </td></tr> <tr><td><code>SHOW_START</code></td><td><code>6</code></td><td> </td></tr> </table></div> <div class="definition"><h3 id="Enum_FileSystems">Enumeration: FileSystems</h3> <pre>Enumeration of File Systems on the resource FORK: Forking of commands without any job manager PBS: Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. UGE: Univa Grid Engine, a variation of PBS implementation. SLURM: The Simple Linux Utility for Resource Management is a open source workload manager. </pre><br/><br/><table class="table-bordered table-striped table-condensed"> <tr><td><code>HOME</code></td><td><code>0</code></td><td> </td></tr> <tr><td><code>WORK</code></td><td><code>1</code></td><td> </td></tr> <tr><td><code>LOCALTMP</code></td><td><code>2</code></td><td> </td></tr> <tr><td><code>SCRATCH</code></td><td><code>3</code></td><td> </td></tr> <tr><td><code>ARCHIVE</code></td><td><code>4</code></td><td> </td></tr> </table></div> <div class="definition"><h3 id="Enum_JobSubmissionProtocol">Enumeration: JobSubmissionProtocol</h3> <pre>Enumeration of Airavata supported Job Submission Mechanisms for High Performance Computing Clusters. SSH: Execute remote job submission commands using via secure shell protocol. GRAM: Execute remote jobs via Globus GRAM service. UNICORE: Execute remote jobs via Unicore services </pre><br/><br/><table class="table-bordered table-striped table-condensed"> <tr><td><code>LOCAL</code></td><td><code>0</code></td><td> </td></tr> <tr><td><code>SSH</code></td><td><code>1</code></td><td> </td></tr> <tr><td><code>GLOBUS</code></td><td><code>2</code></td><td> </td></tr> <tr><td><code>UNICORE</code></td><td><code>3</code></td><td> </td></tr> <tr><td><code>CLOUD</code></td><td><code>4</code></td><td> </td></tr> <tr><td><code>SSH_FORK</code></td><td><code>5</code></td><td> </td></tr> <tr><td><code>LOCAL_FORK</code></td><td><code>6</code></td><td> </td></tr> </table></div> <div class="definition"><h3 id="Enum_MonitorMode">Enumeration: MonitorMode</h3> <pre>Monitoring modes POLL_JOB_MANAGER: GFac need to pull job status changes. XSEDE_AMQP_SUBSCRIBE: Server will publish job status changes to amqp servert. </pre><br/><br/><table class="table-bordered table-striped table-condensed"> <tr><td><code>POLL_JOB_MANAGER</code></td><td><code>0</code></td><td> </td></tr> <tr><td><code>CLOUD_JOB_MONITOR</code></td><td><code>1</code></td><td> </td></tr> <tr><td><code>JOB_EMAIL_NOTIFICATION_MONITOR</code></td><td><code>2</code></td><td> </td></tr> <tr><td><code>XSEDE_AMQP_SUBSCRIBE</code></td><td><code>3</code></td><td> </td></tr> <tr><td><code>FORK</code></td><td><code>4</code></td><td> </td></tr> <tr><td><code>LOCAL</code></td><td><code>5</code></td><td> </td></tr> </table></div> <div class="definition"><h3 id="Enum_DMType">Enumeration: DMType</h3> <br/><table class="table-bordered table-striped table-condensed"> <tr><td><code>COMPUTE_RESOURCE</code></td><td><code>0</code></td><td> </td></tr> <tr><td><code>STORAGE_RESOURCE</code></td><td><code>1</code></td><td> </td></tr> </table></div> <div class="definition"><h3 id="Enum_ProviderName">Enumeration: ProviderName</h3> <pre>Provider name </pre><br/><br/><table class="table-bordered table-striped table-condensed"> <tr><td><code>EC2</code></td><td><code>0</code></td><td> </td></tr> <tr><td><code>AWSEC2</code></td><td><code>1</code></td><td> </td></tr> <tr><td><code>RACKSPACE</code></td><td><code>2</code></td><td> </td></tr> </table></div> <hr/><h2 id="Structs">Data structures</h2> <div class="definition"><h3 id="Struct_ResourceJobManager">Struct: ResourceJobManager</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>resourceJobManagerId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> <tr><td>2</td><td>resourceJobManagerType</td><td><code><a href="#Enum_ResourceJobManagerType">ResourceJobManagerType</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>pushMonitoringEndpoint</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>4</td><td>jobManagerBinPath</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>5</td><td>jobManagerCommands</td><td><code>map<<code><a href="#Enum_JobManagerCommand">JobManagerCommand</a></code>, <code>string</code>></code></td><td></td><td>optional</td><td></td></tr> <tr><td>6</td><td>parallelismPrefix</td><td><code>map<<code><a href="parallelism_model.html#Enum_ApplicationParallelismType">parallelism_model.ApplicationParallelismType</a></code>, <code>string</code>></code></td><td></td><td>optional</td><td></td></tr> </tbody></table><br/><pre>Resource Job Manager Information resourceJobManagerType: A typical HPC cluster has a single Job Manager to manage the resources. pushMonitoringEndpoint: If the job manager pushes out state changes to a database or bus, specify the service endpoint. Ex: Moab Web Service, Moab MongoDB URL, AMQP (GLUE2) Broker jobManagerBinPath: Path to the Job Manager Installation Binary directory. jobManagerCommands: An enumeration of commonly used manager commands. </pre><br/></div><div class="definition"><h3 id="Struct_BatchQueue">Struct: BatchQueue</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>queueName</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> <tr><td>2</td><td>queueDescription</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>3</td><td>maxRunTime</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>4</td><td>maxNodes</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>5</td><td>maxProcessors</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>6</td><td>maxJobsInQueue</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>7</td><td>maxMemory</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>8</td><td>cpuPerNode</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>9</td><td>defaultNodeCount</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>10</td><td>defaultCPUCount</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>11</td><td>defaultWalltime</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>12</td><td>queueSpecificMacros</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>13</td><td>isDefaultQueue</td><td><code>bool</code></td><td></td><td>optional</td><td></td></tr> </tbody></table><br/><pre>Batch Queue Information on SuperComputers maxRunTime: Maximum allowed run time in hours. </pre><br/></div><div class="definition"><h3 id="Struct_LOCALSubmission">Struct: LOCALSubmission</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> <tr><td>2</td><td>resourceJobManager</td><td><code><a href="#Struct_ResourceJobManager">ResourceJobManager</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> </tbody></table><br/><pre>Locally Fork Jobs as OS processes alternativeSSHHostName: If the login to ssh is different than the hostname itself, specify it here sshPort: If a non-default port needs to used, specify it. </pre><br/></div><div class="definition"><h3 id="Struct_SSHJobSubmission">Struct: SSHJobSubmission</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>resourceJobManager</td><td><code><a href="#Struct_ResourceJobManager">ResourceJobManager</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>4</td><td>alternativeSSHHostName</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>5</td><td>sshPort</td><td><code>i32</code></td><td></td><td>optional</td><td><code>22</code></td></tr> <tr><td>6</td><td>monitorMode</td><td><code><a href="#Enum_MonitorMode">MonitorMode</a></code></td><td></td><td>optional</td><td></td></tr> <tr><td>7</td><td>batchQueueEmailSenders</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> </tbody></table><br/><pre>Authenticate using Secured Shell alternativeSSHHostName: If the login to ssh is different than the hostname itself, specify it here sshPort: If a non-default port needs to used, specify it. batchQueueEmailSenders: If a resource always sends the monitoring from a specific address, specify the full email address. If a resource sends emails from multiple addresses ( example: based on the submitted login node) then use the wildchar * to indicate the same. Example: *@*.example.com or *@example.com </pre><br/></div><div class="definition"><h3 id="Struct_GlobusJobSubmission">Struct: GlobusJobSubmission</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>globusGateKeeperEndPoint</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> </tbody></table><br/></div><div class="definition"><h3 id="Struct_UnicoreJobSubmission">Struct: UnicoreJobSubmission</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>unicoreEndPointURL</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> </tbody></table><br/><pre>Unicore Job Submission unicoreEndPointURL: unicoreGateway End Point. The provider will query this service to fetch required service end points. authenticationMode The authenticationMode defines the way certificate is fetched. </pre><br/></div><div class="definition"><h3 id="Struct_CloudJobSubmission">Struct: CloudJobSubmission</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> <tr><td>2</td><td>securityProtocol</td><td><code><a href="data_movement_models.html#Enum_SecurityProtocol">data_movement_models.SecurityProtocol</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>nodeId</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> <tr><td>4</td><td>executableType</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> <tr><td>5</td><td>providerName</td><td><code><a href="#Enum_ProviderName">ProviderName</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>6</td><td>userAccountName</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> </tbody></table><br/><pre>Cloud Job Submission </pre><br/></div><div class="definition"><h3 id="Struct_JobSubmissionInterface">Struct: JobSubmissionInterface</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>jobSubmissionInterfaceId</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> <tr><td>2</td><td>jobSubmissionProtocol</td><td><code><a href="#Enum_JobSubmissionProtocol">JobSubmissionProtocol</a></code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>priorityOrder</td><td><code>i32</code></td><td></td><td>required</td><td><code>0</code></td></tr> </tbody></table><br/><pre>Job Submission Interfaces jobSubmissionInterfaceId: The Job Submission Interface has to be previously registered and referenced here. priorityOrder: For resources with multiple interfaces, the priority order should be selected. Lower the numerical number, higher the priority </pre><br/></div><div class="definition"><h3 id="Struct_ComputeResourceDescription">Struct: ComputeResourceDescription</h3> <table class="table-bordered table-striped table-condensed"><thead><tr><th>Key</th><th>Field</th><th>Type</th><th>Description</th><th>Requiredness</th><th>Default value</th></tr></thead><tbody> <tr><td>1</td><td>computeResourceId</td><td><code>string</code></td><td></td><td>required</td><td><code>"DO_NOT_SET_AT_CLIENTS"</code></td></tr> <tr><td>2</td><td>hostName</td><td><code>string</code></td><td></td><td>required</td><td></td></tr> <tr><td>3</td><td>hostAliases</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> <tr><td>4</td><td>ipAddresses</td><td><code>list<<code>string</code>></code></td><td></td><td>optional</td><td></td></tr> <tr><td>5</td><td>resourceDescription</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>6</td><td>enabled</td><td><code>bool</code></td><td></td><td>optional</td><td></td></tr> <tr><td>7</td><td>batchQueues</td><td><code>list<<code><a href="#Struct_BatchQueue">BatchQueue</a></code>></code></td><td></td><td>optional</td><td></td></tr> <tr><td>8</td><td>fileSystems</td><td><code>map<<code><a href="#Enum_FileSystems">FileSystems</a></code>, <code>string</code>></code></td><td></td><td>optional</td><td></td></tr> <tr><td>9</td><td>jobSubmissionInterfaces</td><td><code>list<<code><a href="#Struct_JobSubmissionInterface">JobSubmissionInterface</a></code>></code></td><td></td><td>optional</td><td></td></tr> <tr><td>10</td><td>dataMovementInterfaces</td><td><code>list<<code><a href="data_movement_models.html#Struct_DataMovementInterface">data_movement_models.DataMovementInterface</a></code>></code></td><td></td><td>optional</td><td></td></tr> <tr><td>11</td><td>maxMemoryPerNode</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>12</td><td>gatewayUsageReporting</td><td><code>bool</code></td><td></td><td>optional</td><td></td></tr> <tr><td>13</td><td>gatewayUsageModuleLoadCommand</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>14</td><td>gatewayUsageExecutable</td><td><code>string</code></td><td></td><td>optional</td><td></td></tr> <tr><td>15</td><td>cpusPerNode</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>16</td><td>defaultNodeCount</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>17</td><td>defaultCPUCount</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> <tr><td>18</td><td>defaultWalltime</td><td><code>i32</code></td><td></td><td>optional</td><td></td></tr> </tbody></table><br/><pre>Computational Resource Description computeResourceId: Airavata Internal Unique Identifier to distinguish Compute Resource. hostName: Fully Qualified Host Name. hostAliases: Aliases if any. ipAddress: IP Addresses of the Resource. resourceDescription: A user friendly description of the resource. JobSubmissionProtocols: A computational resources may have one or more ways of submitting Jobs. This structure will hold all available mechanisms to interact with the resource. The key is the priority DataMovementProtocol: Option to specify a prefered data movement mechanism of the available options. fileSystems: Map of file systems type and the path. </pre><br/></div></div></body></html>

content/api-docs/master/compute_resource_model.html (319 lines of code) (raw):