prisms_jobs.JobDB

class prisms_jobs.JobDB(dbpath=None)

Bases: object

A primsms_jobs Job Database object

Usually this is called without arguments (prisms_jobs.JobDB()) to open or create a database in the default location.

Parameters:dbpath (str, optional) – Path to JobDB sqlite database. By default, uses prisms_jobs.config.dbpath().

__init__(dbpath=None)

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__([dbpath])	Initialize self.
abort_job([jobid, job])	Delete a job and mark job taskstatus as Aborted
add(job_status)	Add a record to the jobs database.
close()	Close the connection to the jobs database.
complete_job([jobid, job])	Mark job taskstatus as ‘Complete’
connect([dbpath])	Open a connection to the jobs database.
continue_all()	Resubmit all jobs eligible to continue
continue_job([jobid, job])	Resubmit one job with given jobid.
delete_job([jobid, job, series])	Delete job if running, and delete job from the database.
eligible_to_abort(job)	Check if job is eligible to be aborted
eligible_to_complete(job)	Check if job is eligible to be completed
eligible_to_continue(job)	Return True if job is eligible to be continued, else return False
eligible_to_delete(job)	Check if job is eligible to be aborted
eligible_to_error(job)	Check if job is eligible to be marked as error
eligible_to_reset(job)	Check if job is eligible to be reset
error_job(message[, jobid, job])	Mark job taskstatus as ‘Error: message’
print_active([full, series])	Print active jobs
print_all([full, series])	Print all jobs
print_header()	Print header rows for record summary
print_job([jobid, job, full, series])	Print job with given jobid
print_selected([curs, full, series])	Fetch and print jobs selected with SQL SELECT statement using cursor ‘curs’.
print_untracked([full])	Print untracked jobs.
reset_job([jobid, job])	Mark job taskstatus as ‘Incomplete’
select_active_series_id()	Return a list of lists of jobids (one list for each active series).
select_all_active_id()	Return a list with all active jobids.
select_all_id()	Return a list with all jobids.
select_all_series_id()	Return a list of lists of jobids (one list for each series).
select_child(jobid)	Return record for the child of a job
select_job(jobid)	Return record (sqlite3.Row object) for one job with given jobid.
select_parent(jobid)	Return record for the parent of a job
select_range_id(min_jobid, max_jobid)	Return a list of all jobids which are between (and including) min_jobid and max_jobid.
select_range_series_id(min_jobid, max_jobid)	Return a list of lists of all jobids for series (one list for each series) which have the last job between (and including) min_jobid and max_jobid.
select_recent_id(recent_time)	Return a list of all jobids which were modified in the last ‘recent_time’
select_recent_series_id(recent_time)	Return a list of lists of jobids (one for each series) which were modified in the last ‘recent_time’
select_regex_id(key, regex)	Return a list of all jobids in which the column ‘key’ matches the regular expression ‘regex’
select_regex_series_id(key, regex)	Return a list of lists of jobids (one for each series) in which the column ‘key’ matches the regular expression ‘regex’
select_series(jobid)	Return records (sqlite3.Row objects) for a series of auto jobs
select_series_id(jobid)	Return a list with all jobids for a series of auto jobs.
update()	Update records using qstat.

_print_full_record(r)

Print record as list of key-val pairs.

Parameters:r (dict) – a dict-like object

_print_record(r)

Print record summary

Parameters:r (dict) – a dict-like object containing: “jobid”, “jobname”, “nodes”, “procs”, “walltime”, “jobstatus”, “elapsedtime”, “taskstatus”, “auto”, and “continuation_jobid”

abort_job(jobid=None, job=None)

Delete a job and mark job taskstatus as Aborted

Parameters:

• jobid – jobid of the job to continue
• job – (sqlite3.Row) If this is given, jobid is not necessary and is ignored if given

Raises:

EligibilityError if job not eligible to be aborted

add(job_status)

Add a record to the jobs database.

Parameters:job_status (dict) – Accepts a dictionary of data comprising the record. Create job_status using prisms_jobs.jobdb.job_status_dict().

close()

Close the connection to the jobs database.

complete_job(jobid=None, job=None)

Mark job taskstatus as ‘Complete’

Parameters:

• jobid (str) – ID of job
• job (sqlite3.Row object) – Job record from database

Raises:

prisms_jobs.EligibilityError – If not job not eligible to be marked ‘Complete’

connect(dbpath=None)

Open a connection to the jobs database.

Parameters:

• dbpath (str, optional) – path to a JobDB database file. By default,
• prisms_jobs.config.dbpath() (uses) –

continue_all()

Resubmit all jobs eligible to continue

continue_job(jobid=None, job=None)

Resubmit one job with given jobid.

Parameters:

• jobid – jobid of the job to continue
• job – (sqlite3.Row) If this is given, jobid is not necessary and is ignored if given

Raises:

EligibilityError if job not eligible to be continued

delete_job(jobid=None, job=None, series=False)

Delete job if running, and delete job from the database.

Parameters:

• jobid (str) – jobid of the job to continue
• job (sqlite3.Row) – If this is given, jobid is not necessary and is ignored if given
• series (bool) – If ‘series’=True, deletes entire job series

eligible_to_abort(job)

Check if job is eligible to be aborted

Jobs are eligible to be aborted if:

jobstatus != “C” or (jobstatus == “C” and (taskstatus == “Incomplete” or taskstatus == “Check”))

Parameters:job – a sqlite3.Row, as obtained by self.select_job() Returns:(0, jobid, None) if eligible (1, jobid, msg) if not eligible

eligible_to_complete(job)

Check if job is eligible to be completed

Jobs are eligible to be completed if:

taskstatus != “Complete” and taskstatus != “Continued”

Parameters:job – a sqlite3.Row, as obtained by self.select_job() Returns:(0, jobid, None) if eligible (1, jobid, msg) if not eligible

eligible_to_continue(job)

Return True if job is eligible to be continued, else return False

Job must have jobstatus=”C” and taskstatus=”Incomplete” and auto=1,

or else the job can not be continued.

Parameters:job – a sqlite3.Row, as obtained by self.select_job() Returns:(0, jobid, None) if eligible (1, jobid, msg) if not eligible

eligible_to_delete(job)

Check if job is eligible to be aborted

All jobs are eligible to be deleted

Parameters:job – a sqlite3.Row, as obtained by self.select_job() Returns:(0, jobid, None) if eligible (1, jobid, msg) if not eligible

eligible_to_error(job)

Check if job is eligible to be marked as error

All jobs are eligible to be marked as error. (Should this exclude “Continued” jobs?)

Parameters:job – a sqlite3.Row, as obtained by self.select_job() Returns:(0, jobid, None) if eligible (1, jobid, msg) if not eligible

eligible_to_reset(job)

Check if job is eligible to be reset

Jobs are eligible to be reset if they are ‘auto’ jobs, and

taskstatus == “Error:.*” or “Aborted”

Parameters:job – a sqlite3.Row, as obtained by self.select_job() Returns:(0, jobid, None) if eligible (1, jobid, msg) if not eligible

error_job(message, jobid=None, job=None)

Mark job taskstatus as ‘Error: message’

Any job can be marked as error.

Parameters:

• jobid – jobid of the job to mark as error
• job – (sqlite3.Row) If this is given, jobid is not necessary and is ignored if given

print_active(full=False, series=False)

Print active jobs

“Active” jobs are those with taskstatus=’Incomplete’ or ‘Check’

Parameters:

• full (bool) – If True, print as key:val pair list. If (default) False, print single row summary in ‘qstat’ style.
• series (bool) – If True, print records as groups of auto submitting job series. If (default) False, print in order found.

print_all(full=False, series=False)

Print all jobs

Parameters:

• full (bool) – If True, print as key:val pair list, If (default) False, print single row summary in ‘qstat’ style.
• series (bool) – If True, print records as groups of auto submitting job series. If (default) False, print in order found.

print_header()

Print header rows for record summary

print_job(jobid=None, job=None, full=False, series=False)

Print job with given jobid

Parameters:

• jobid (str) – ID of job
• job (sqlite3.Row object) – Job record from database
• full (bool) – If True, print as key:val pair list, If (default) False, print single row summary in ‘qstat’ style.
• series (bool) – If True, print records as groups of auto submitting job series. If (default) False, print in order found.

print_selected(curs=None, full=False, series=False)

Fetch and print jobs selected with SQL SELECT statement using cursor ‘curs’.

Parameters:

• curs – Fetch selected jobs from sqlite3 cursor ‘curs’. If no ‘curs’ given, use self.curs.
• full (bool) – If True, print as key:val pair list, If (default) False, print single row summary in ‘qstat’ style.
• series (bool) – If True, print records as groups of auto submitting job series. If (default) False, print in order found.

print_untracked(full=False)

Print untracked jobs.

Untracked jobs are stored in self.untracked after calling JobDB.update().

Parameters:full (bool) – If True, print as key:val pair list, If (default) False, print single row summary in ‘qstat’ style.

reset_job(jobid=None, job=None)

Mark job taskstatus as ‘Incomplete’

Jobs are eligible to be reset if they are ‘auto’ jobs, and

taskstatus == “Error:.*” or “Aborted”

Parameters:

• jobid – jobid of the job to mark as error
• job – (sqlite3.Row) If this is given, jobid is not necessary and is ignored if given

Raises:

prisms_jobs.EligibilityError – If not job not eligible to be marked ‘Complete’

select_active_series_id()

Return a list of lists of jobids (one list for each active series).

“Active” series of auto jobs are those with one job with

taskstatus=’Incomplete’ or ‘Check’

select_all_active_id()

Return a list with all active jobids.

“Active” jobs are those with taskstatus=’Incomplete’ or ‘Check’

select_all_id()

Return a list with all jobids.

select_all_series_id()

Return a list of lists of jobids (one list for each series).

select_child(jobid)

Return record for the child of a job

The child is the job whose jobid = continuation_jobid of job with given jobid

select_job(jobid)

Return record (sqlite3.Row object) for one job with given jobid.

select_parent(jobid)

Return record for the parent of a job

The parent is the job with continuation_jobid = given jobid

select_range_id(min_jobid, max_jobid)

Return a list of all jobids which are between (and including) min_jobid and max_jobid.

select_range_series_id(min_jobid, max_jobid)

Return a list of lists of all jobids for series (one list for each series) which have the last job between (and including) min_jobid and max_jobid.

select_recent_id(recent_time)

Return a list of all jobids which were modified in the last ‘recent_time’

select_recent_series_id(recent_time)

Return a list of lists of jobids (one for each series) which were modified in the last ‘recent_time’

select_regex_id(key, regex)

Return a list of all jobids in which the column ‘key’ matches the regular expression ‘regex’

select_regex_series_id(key, regex)

Return a list of lists of jobids (one for each series) in which the column ‘key’ matches the regular expression ‘regex’

select_series(jobid)

Return records (sqlite3.Row objects) for a series of auto jobs

select_series_id(jobid)

Return a list with all jobids for a series of auto jobs.

update()

Update records using qstat.

Any jobs found using qstat that are not in the jobs database are saved in ‘self.untracked’.