Running Jobs
The polly_jobs class contains functions which can be used to create, cancel and monitor polly jobs. Polly CLI jobs can now be initiated, managed and have a status-checked for from Polly Python. This lets users run jobs on the Polly cloud infrastructure by scaling computation resources as per need. Users can start and stop tasks and even monitor jobs.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
token |
str
|
token copy from polly. |
None
|
Usage
from polly.jobs import jobs
jobs = jobs(token)
cancel_job(project_id, job_id)
cancel a polly job.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
project_id |
str
|
workspace id |
required |
job_id |
str
|
job id |
required |
job_logs(project_id, job_id, mode='all')
get logs of job
Parameters:
Name | Type | Description | Default |
---|---|---|---|
project_id |
str
|
workspace_id |
required |
job_id |
str
|
job_id |
required |
Other Parameters:
Name | Type | Description |
---|---|---|
mode |
str
|
either 'latest' or 'all' logs (default: {"all"}) |
Raises:
Type | Description |
---|---|
err
|
RequestException |
job_status(project_id, job_id='', internalCalls=False)
Get the status of a job given the rproject id and job id. If no job id given, gives status for all the jobs in the provided workspace.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
project_id |
str
|
workspace id |
required |
job_id |
str
|
job id |
''
|
Returns:
Name | Type | Description |
---|---|---|
DataFrame |
dict
|
Contains job id, job name and status sorted as per created timestamp |
submit_job(project_id, job_file)
Submits a polly cli job in the given workspace. A message will be displayed on the status of the operation.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
project_id |
str
|
workspace id |
required |
job_file |
str
|
required configuration json file path |
required |
Examples
import os
from polly.auth import Polly
from polly.jobs import jobs
token = os.environ['POLLY_REFRESH_TOKEN']
Polly.auth(token)
job = jobs()
# Making a JSON job file with all the required params
job_map = {
"cpu":"1",
"memory":"1Gi",
"image": "ubuntu",
"tag": "latest",
"name": "This is a job for testing bust mode",
"command": ["/bin/bash","-c", "TERM=xterm free -h; echo '\nnumber of vCPU';nproc;sleep 30"]
}
import json
json_object = json.dumps(job_map, indent=4)
with open ("/import/test_job.json", "w") as input:
input.write(json_object)
submit_job()
#job submission
workspace_id = "12345"
job_file = "/import/test_job.json"
job.submit_job(workspace_id,job_file)
job_status()
#getting job status immediately after running
job.job_status(workspace_id,"1f445778c0034b44a799a0e72d48ab94")
Job ID Job Name \
0 1f445778c0034b44a799a0e72d48ab94 This is a job for testing bust mode
Job State
0 RUNNING
Job ID Job Name \
0 1f445778c0034b44a799a0e72d48ab94 This is a job for testing bust mode
Job State
0 Success
Job ID Job Name \
0 cc1ad0d41eb64cc68dd12c4ddbefad9a This is a job for testing bust mode
1 9d9895b453fe42fcb13bb7902e5de457 This is a job for testing bust mode
2 d9e7c71c6a154a2bb018458c07db0733 This is a job for testing bust mode
3 6d9617e623444ec2888ded77c1affa2a This is a job for testing bust mode
Job State
0 Success
1 Success
2 Success
3 Success
#workspace id can be int or string
workspace_id = 12345
job.job_status(workspace_id, "1f445778c0034b44a799a0e72d48ab94")
# invalid workspace id
workspace_id = 12345
job.job_status(workspace_id, "1f445778c0034b44a799a0e72d48ab94")
Not able to get the status of the Job(s)
Unexpected exception formatting exception. Falling back to standard exception
Job ID Job Name \
0 1f445778c0034b44a799a0e72d48ab94 This is a job for testing bust mode
Job State
0 Success
# invalid job id provided while status
workspace_id = 12345
job.job_status("12345", "1f445778c0034b44a799a0e72d4adac")
Not able to get the status of the Job(s)
Unexpected exception formatting exception. Falling back to standard exception
cancel_job()
{'errors': [{'status': '400', 'code': 'bad_req', 'title': 'Bad Request', 'detail': 'Cannot cancel a job in Success state'}]}
Failed to cancel the job.: Cannot cancel a job in Success state
#cancelling jobs before success
job.submit_job(workspace_id,job_file)
job.cancel_job(workspace_id, "a51187d93aea402b8f40789404398bfb")
job_logs()
Error: Not able to find the logs. It seems to be not generated yet