src.common package

Submodules

src.common.calibrate module

src.common.calibrate.calc_dualtotFlux(slcp, dslcp, srcp, dsrcp, bslcp, bdslcp, bsrcp, bdsrcp)

Calculate the total flux density of the source

src.common.calibrate.calc_flux(ta, dta, pss, dpss)

Calculate the flux density.

src.common.calibrate.calc_pc_eq(Ta1, Ta2, term2='')

Calculate the pointing correction and its derivatives. The pointing equations are given in the method calc_pc_pss()

Parameters:

• Ta1 – first temperature

• Ta2 – second temperature

• term2 – the second term of the pointing correction equation as defined

• thesis/reference. (in) –

Returns:

the pointing correction der1: first derivative with respect to Ta1 der2: first derivative with respect to Ta2

Return type:

pc

src.common.calibrate.calc_pc_pss(scanNum, hpsTa, errHpsTa, hpnTa, errHpnTa, onTa, errOnTa, flux, data)

Calculate the pss for pointing corrected observations.

Parameters:

• scanNum – the index of the current scan, see plot_manager.py

• hpsTa – the half power south antenna temperature

• errHpsTa – the error in the half power south antenna temperature

• hpnTa – the half power north antenna temperature

• errHpnTa – the error in the half power north antenna temperature

• onTa – the on source antenna temperature

• errOnTa – the error in the on source antenna temperature

• flux – the source flux density

• data – the dictionary containing all the drift scan parameters

Returns:

the point source sensitivity errPss: the error in the point source sensitivity appEff: the apperture effeciency corrTa: the corrected antenna temperature errCorrTa: the error in the corrected antenna temperature appEff: the apperture effeciency

Return type:

pss

src.common.calibrate.calc_pss(flux, Ta, errTa)

Calculate the Point source sensitivity (PSS) and its error for data with no pointing correction applied.

Parameters:

• flux – the flux density of the source

• Ta – the antenna temperature

• errTa – the error in the antenna temperature

Returns:

the point source sensitivity errPSS: the error in the point source sensitivity appEff: the apperture effeciency

Return type:

pss

src.common.calibrate.calc_tcorr(Ta, pc, data)

Calculate the antenna temperature correction for high frequencies.

Parameters:

• temperature (Ta - the on scan antenna) –

• correction (pc - the pointing) –

• parameters (data - the dictionary containing all the drift scan) –

Returns:

corrTa - the corrected antenna temperature

src.common.calibrate.calc_totFlux(lcp, dlcp, rcp, drcp)

Calculate the total flux density of the source

src.common.calibrate.calibrate(hpsTa, errHpsTa, hpnTa, errHpnTa, onTa, errOnTa, data, log)

Calculate the pointing corrected antenna temperature.

Parameters:

• hpsTa – the half power south antenna temperature

• errHpsTa – the error in the half power south antenna temperature

• hpnTa – the half power north antenna temperature

• errHpnTa – the error in the half power north antenna temperature

• onTa – the on source antenna temperature

• errOnTa – the error in the on source antenna temperature

• data – the dictionary containing all the drift scan parameters

Returns:

the pointing correction corrTa: the corrected antenna temperature errCorrTa: the error in the corrected antenna temperature

Return type:

pc

src.common.calibrate.getpss(pssdf, date)

Get the PSS of a given bin time-range.

pssdf: dataframe containing the pss bins. date: the date used to estimate the required bin range in order to extract the PSS.

src.common.calibrate.set_corr_ta_to_zero()

Set the estimation of the corrected antenna temp to zero.

src.common.calibrate.set_pss_to_zero()

Set the estimation of the pss and apperture effeciency to zero

src.common.calibrate.test_for_nans(Ta, errTa)

Test if data has nans

Returns:

antenna temperature errTa: error in antenna temperature

Return type:

Ta

src.common.contextManagers module

src.common.dataProcessingFlowManager module

class src.common.dataProcessingFlowManager.DataProcessingFlowManager(fileName: str, freq: float, src: str, x: <built-in function array>, y: <built-in function array>, log: object, flag: int, applyRFIremoval: str, savefolder: str, srcTag: str, pol: str, frontend: str, HPBW: float, FNBW: float, theoFit: str, autoFit: str, force: str = 'n')

Bases: object

FNBW: float

HPBW: float

applyRFIremoval: str

autoFit: str

clean_data()

Clean the data of possible RFI. Remove all data points larger that 3 times the standard deviation.

fileName: str

fit_theoretical_baselines(coeff)

Fit the data at the theoretical baseline location.

flag: int

force: str = 'n'

freq: float

frontend: str

locate_baseline_blocks_fnbw(x, hfnbw)

Find the locations to fit a baseline. We assume the baseline gives a good fit beyond the fnbw points on either side of the beam.

locate_baseline_blocks_masked(offset, y, coeff)

Locate baseline by masking the main beam. This doesn’t work well because the data is not perfect and may contain sidelobes which may not necessarily be best addressed by looking at using higher order polynomials.

locate_main_beam_data(offset, coeff)

log: object

make_initial_plots(plot, title, plotlab, plotlabCleaned)

Make initial plots of the raw and cleaned data

pol: str

savefolder: str

src: str

srcTag: str

theoFit: str

x: array

y: array

src.common.driftScanAttributes module

class src.common.driftScanAttributes.DriftScanAttributes

Bases: object

Class of parameters to be collected/saved to a database from the drift scan fits file. These are the values that will be populated in the CALDB or TARDB databases.

Parameters:

parameters (dict) – dictionary of all properties to be stored.

print_dict()

Print the dictionary key value pairs.

print_keys()

Print the keys of the dictionary.

reset_dict()

Reset the dictionary.

set_comm_weather_parameters()

Set Common weather parameters.

set_common_parameters()

Dictionary of default values for parameters common to all scan types.

set_db_dict()

Set dual beam observation parameters.

set_db_fit_parameters(pols, beams, scanDirection)

Set dual beam fit parameters.

Attributes:

polslist

List of polarizations - left circular polarization: LCP (L), : right circular polarization RCP (R).

beamslist

List of drift scan beams - left (A) and right (B), beams are either +ve or -ve depending on frequency.

scanDirectionlist

List of scan directions - north (N), south (S) or center ON (O). Using O because C makes things confusing.

set_db_weather_parameters()

Set dual beam weather related parameters.

set_nb_dict()

Set Narrow beam calibration data dictionary.

set_nb_fit_parameters(pols, scanDirection)

Set the parameters specific to a narrow beam drift scan.

Attributes:

polslist

List of polarizations - left circular polarization: LCP (L), : right circular polarization RCP (R).

scanDirectionlist

List of scan directions - north (N), south (S) or center ON (O). Using O because C makes things confusing.

set_tau_parameters()

Set tau (optical depth) parameters.

set_tbatmos_parameters()

Set tbatmos (atmospheric temperature) parameters.

set_wb_dict()

Combine wide beam common calibration parameters.

set_wb_parameters(pols)

Set the parameters specific to a wide beam on lcp drift scan.

src.common.driftScanData module

src.common.driftScans module

src.common.enums module

class src.common.enums.Month(value, names=None, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Apr = '04'

Aug = '08'

Dec = '12'

Feb = '02'

Jan = '01'

Jul = '07'

Jun = '06'

Mar = '03'

May = '05'

Nov = '11'

Oct = '10'

Sep = '09'

class src.common.enums.ScanType(value, names=None, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Drift scan type

Parameters:

Enum (enum) – Enum of the different scan types

SBW - wide single beam

SBN - narrow single beam

DB - dual beam

DB = 3

SBN = 2

SBW = 1

src.common.exceptions module

exception src.common.exceptions.BeamTypeNotFoundError

Bases: Error

Beam type not found error

exception src.common.exceptions.DB_READ_ERROR

Bases: Error

Database read error

exception src.common.exceptions.DB_WRITE_ERROR

Bases: Error

Databae write error

exception src.common.exceptions.EmptyFilePathError

Bases: Error

Raised when an expected file path is empty.

exception src.common.exceptions.EmptyFolderError

Bases: Error

Raised when a folder is empty

exception src.common.exceptions.EmptyTableError

Bases: Error

Raised when a database table is empty.

exception src.common.exceptions.Error

Bases: Exception

Base class for other exceptions.

exception src.common.exceptions.FileResourceNotFoundError

Bases: Error

Raised when a file resource is not found or wasn’t included in the distribution files

exception src.common.exceptions.InvalidFileExtensionError

Bases: Error

Raised when a file has an invalid file extension.

exception src.common.exceptions.InvalidFilePath

Bases: Error

Raised when a file path is invalid.

exception src.common.exceptions.MissingChartHeaderError

Bases: Error

Raised when a fits file is missing a chart header unit.

exception src.common.exceptions.MissingParameterException

Bases: Error

Raised when a fits file is missing a chart header unit.

exception src.common.exceptions.ValueOutOfRangeException

Bases: Error

Raised when a value is out of range.

src.common.fitting module

src.common.fitting.calc_residual(model, data, log='')

Calculate the residual and rms between the model and the data.

Parameters:

• model (array) – 1D array containing the model data

• data (array) – 1D array containing the raw data

• log (object) – file logging object

Returns:

• res (1d array) – the residual

• rms (int) – the rms value

src.common.fitting.calc_residual_and_rms(x, y, log, order=1)

Calculate the residual and rms from data

Parameters:

• x (array) – 1d array

• x-axis (data representing the) –

• y (array) – 1d array data representing the y-axis

• deg (int) – degree of the polynomial

Returns:

model of res (array): rms (float): coeff():

Return type:

model (array)

src.common.fitting.clean_data(spl, x, y, scanLen, log='')

Clean the data using iterative fitting.

Parameters:

• spl – 1d array the splined data

• x – 1d array data representing the x-axis

• y – 1d array data representing the y-axis

• scanLen – int length of the drift scan array

• log – object file logging object

Returns:

1d array

the residual before the rfi has been removed

rmsBeforeClean: int

the rms before removing rfi data

rmsAfterClean: int

the rms after removal of rfi data

finX: 1d array

the data representing the x-axis after the removal of rfi data

finY: 1d array

the data representing the y-axis after the removal of rfi data

finRes: 1d array

the residual of the cleaned data after the rfi has been removed

finMaxSpl: int

the maximum of the spline of the cleaned data

finspl: 1d array

the spline of the cleaned data

pointsDeleted: int

number of points deleted when cleaning the data

Return type:

resRFI

src.common.fitting.clean_data_iterative_fitting(x, y, scanLen, res, rms, log='', x2='')

Find the best fit to the data by iteratively eliminating data points that fall beyond an established cut-off limit.

x1d array

data representing the x-axis

y1d array

data representing the y-axis

scanLenint

length of the drift scan array

res: 1d array

the residual

rms: int

the rms value

logobject

file logging object

x21d array

filenames

finalX: 1d array

the data representing the x-axis after the removal of rfi data

finalY: 1d array

the data representing the y-axis after the removal of rfi data

finalRms: int

the rms after removal of rfi data

finRes: 1d array

the residual of the cleaned data after the rfi has been removed

finMaxSpl: int

the maximum of the spline of the cleaned data

finalSplinedData: 1d array

the spline of the cleaned data

pointsDeleted: int

number of points deleted when cleaning the data

src.common.fitting.clean_rfi(x, y, log='')

Clean the RFI in the data using iterative rms cuts.

Parameters:

• x (array) – Array of the independent variable

• y (array) – Array of the dependent variable

• log (object) – file logging object

Returns:

the data representing the x-axis after the removal of rfi data finY (array):the data representing the y-axis after the removal of rfi data rmsBeforeClean (int): the rms before removal of rfi data rmsAfterClean (int): the rms after removal of rfi data finspl (array): the spline of the cleaned data pointsDeleted (int): number of points deleted when cleaning the data

Return type:

finX (array)

src.common.fitting.correct_drift(xBase, yBase, x, log, order=1)

Correct for a drifting baseline in the scan by fitting a first order polynomial to a region with no signal.

Parameters:

• xBase (x data of the baseline) –

• yBase (y data of the baseline) –

• x (x data of the entire drift scan) –

src.common.fitting.fit_beam(x, y, p, fnbw, force, log, saveTag, fitTheoretical, autoFit=None)

Fit single beam data.

Parameters:

• x (array) – 1D array of data representing the x-axis

• y (array) – 1D array of data representing the y-axis

• p (list) – list of initial fit parameters

• fnbw (float) – source first null beam width from file

• dec (float) – source declination

• data (dict) – dictionary of source parameters

• scanNum (int) – Value representing the current scan

• force (str) – String to determine whether to force a fit or not

• log (object) – loffing object

src.common.fitting.fit_dual_beam(x, y, hpbw, fnbw, factor, saveTo, log)

src.common.fitting.gauss(x, *p)

Gaussian for fitting the beam

src.common.fitting.gauss_lin(x, *p)

Generate a gaussian plus first-order polynomial to fit the drift scan beam. Note that the width of the Gaussian is hard-coded to the half-power beamwidth.

Parameters:

• x (1D array) – 1D array of data representing the x-axis

• p (tuple) – tuple of gaussian parameters

Returns:

gaussFit – array of data representing the gaussian fit

Return type:

1d array

src.common.fitting.get_base(localMinPositions, block_width, scan_len)

get the baseline block/s from data with large sidelobes.

src.common.fitting.locate_baseline_blocks_auto(x, y, peakCenterGuess, hfnbw, log, saveLoc)

Find the locations to fit a baseline automatically.

These locations are found/determined by fitting a spline to the data and using the locations of the local minimum as baseline regions.

Parameters:

• peakCenterGuess (float) – value of x at peak center in x array

• hfnbw (float) – half the first null beam width

• log (object) – loffing object

src.common.fitting.poly_coeff(x, y, deg)

Calculate the polynomial coeffecients depending on the degree/order of the polynomial

Parameters:

• x (array) – 1d array data representing the x-axis

• y (array) – 1d array data representing the y-axis

• deg (int) – degree of the polynomial

Returns:

array of polynomial fitted data

src.common.fitting.spline(x, y, anchor_points=9, order=3, log='')

Given a set of data points (x,y) determine a smooth spline approximation of degree k on the interval x[0] <= x <= x[n]

Parameters:

• x (array) – 1D array of data representing the x-axis

• y (array) – 1D array of data representing the y-axis

• anchor_points (int) – the number of anchor points in the data

• order (int) – polynomial order to fit, preferrably a cubic spline

Returns:

Spline of the data

Return type:

spline_fit (array)

src.common.fitting.test_gauss_fit(x, y, p0, log='')

Fit the data using a gaussian

Parameters:

p0 – initial fit guess

src.common.fitting.test_position_validity(localMaxPositions, localMinPositions, maxPoints)

Test the position validity of the local min/max positions. The local minimum positions cannot fall within the range of the local maximum positions.

Parameters:

• localMaxPositions (list) – list of local max positions

• localMinPositions (list) – list of local min positions

• p (int) – maximum number of permissable points.

Returns:

index list of positions to delete

Return type:

pointsToDelete

src.common.getResources module

src.common.getResources.get_cal_list() → DataFrame

Get list of calibrator names from file

src.common.getResources.get_jpl_results() → DataFrame

Get list of calibrator names from file The data used here is obtained from the NASA HORIZONS website https://ssd.jpl.nasa.gov/horizons/app.html#/

params: Ephemeris type: Observer Table, Target body: Jupiter, Observer Location: Geocentric [500] Time specification: 1950 to 2100, step=1 day

src.common.logConfiguration module

src.common.miscellaneousFunctions module

src.common.miscellaneousFunctions.calc_log_freq(freq)

Calculate the log(frequency)

src.common.miscellaneousFunctions.catch_zeroDivError(col, colerr)

src.common.miscellaneousFunctions.check_for_nans(x, log)

Check for missing data or incomplete data.

Parameters:

• x (array) – A 1D array

• log (object) – Logging object.

Returns:

The flag value x (array) : A 1D array without any NAN values

Return type:

flag (int)

src.common.miscellaneousFunctions.create_current_scan_directory()

src.common.miscellaneousFunctions.delete_logs()

Delete the logfile if it exists

src.common.miscellaneousFunctions.delete_object(objectName) → None

Delete an object from memory

Parameters:

objectName (str) – the name of the object to delete

src.common.miscellaneousFunctions.fast_scandir(dirname)

Scan directory for all folders in the given directory

src.common.miscellaneousFunctions.set_dict_item(dictionary, key, value, description) → None

Set dictionary key, value and description

Parameters:

• key (str) – the key you are creating for the dictionary entry

• value (str,float,list,dict or int) – the value of the dictionary entry

• description (_type_) – the description of the dictionary entry

src.common.miscellaneousFunctions.set_table_name(src, log)

Set the table name based on the declination.

Parameters:

src (str) – source or table name

src.common.miscellaneousFunctions.sig_to_noise(signalPeak, noise, log)

Calculate the signal to noise ratio. i.e. Amplitude / (stdDeviation of noise) Taken from paper on ‘Signal to Noise Ratio (SNR) Enhancement Comparison of Impulse-, Coding- and Novel Linear-Frequency-Chirp-Based Optical Time Domain Reflectometry (OTDR) for Passive Optical Network (PON) Monitoring Based on Unique Combinations of Wavelength Selective Mirrors’

Photonics 2014, 1, 33-46; doi:10.3390/photonics1010033 https://www.mdpi.com/2304-6732/1/1/33 https://www.mdpi.com/68484

Parameters:

• signalPeak (float) – The maximum valueof a desired signal

• noise (array) – array of fit residuals

• log (object) – file logging object

Returns:

signal to noise ratio

Return type:

sig2noise (float)

src.common.msgConfiguration module

src.common.msgConfiguration.disclaimer()

Prints disclaimer.

src.common.msgConfiguration.load_prog(prog: str)

Print message to load the selected program.

src.common.msgConfiguration.msg_wrapper(logName: str, log: object, msg: str)

Wraps logging messages.

Parameters:

• log_name (str) – The name of the logger e.g. info

• log (object) – The logging object

• msg (str) – The message to be wrapped.

src.common.msgConfiguration.print_start()

Prints program banner.

src.common.plotting module

src.common.plotting.fit_plot()

src.common.plotting.make_plot(plotlab)

plot data from plotting ionary

src.common.plotting.plotBaselineEstimate(x, y, yspl, posLeft, posRight, lab1, lab2, title, saveTo, lb=[], rb=[], xlabel='', ylabel='')

Plot the positions or locations of the baseline local minimum estimate.

Parameters:

• x (list/array) – x-axis data

• y (list/array) – y-axis data

• yspl (list/array) – splined y-axis data

• posLeft (int) – index of the local min to the left of the central peak

• posRight (int) – index of the local min to the right of the central peak

• lab1 (str) – label of posLeft

• lab2 (str) – label of posRight

• title (str) – title of plot

• saveTo (str) – saving location

src.common.plotting.plotCorrectedData(x, ycorr, posLeft, posRight, lab1, lab2, title, saveTo, xlabel='', ylabel='')

Plot corrected data.

Parameters:

• x (list/array) – x-axis data

• y (list/array) – y-axis data

• ycorr (list/array) – corrected y-axis data

• posLeft (int) – local mins to the left of the central peak

• posRight (int) – local mins to the right of the central peak

• lab1 (str) – label of raw data

• lab2 (str) – label of corrected data

• title (str) – title of plot

• saveTo (str) – saving location

src.common.plotting.plotDualBeamFit(x1, y1, x2, y2, x3, y3, title, plotLab1, plotLab2, plotLab3, saveTo, xlabel='', ylabel='')

Plot dual beam scan

src.common.plotting.plotPeakFit(title, x, ycorr, ypeak, err_peak, peak_beam, saveTo, xlabel='', ylabel='')

_summary_

Parameters:

• title (_type_) – _description_

• x (_type_) – _description_

• ycorr (_type_) – _description_

• ypeak (_type_) – _description_

• err_peak (_type_) – _description_

• peak_beam (_type_) – _description_

• saveTo (_type_) – _description_

• xlabel (str, optional) – _description_. Defaults to “”.

• ylabel (str, optional) – _description_. Defaults to “”.

src.common.plotting.plot_no_fit(x, y, title, subtitle, saveTo, label='', xlabel='', ylabel='')

Plot corrupted data.

src.common.plotting.plot_overlap(x1, y1, x2, y2, title, plotLab1, plotLab2, saveTo, xlabel='', ylabel='')

Overplot 2 scans

src.common.plotting.plot_scan(x, y, plotLab, title, saveTo, xlab='Scandist [deg]', ylab='T$_{A}$ [K]')

Plot the drift scan

src.common.sqlite_db module

class src.common.sqlite_db.SQLiteDB(dbPath: str, log: object)

Bases: object

close_db()

Close the database connection.

commit_changes()

Commit/save changes you have implemented to the database.

create_db()

Create a new database.

create_table(data, tableName)

Create an sql statement to create a table.

create_table_stmt(data, tableName)

Create table from dictionary.

dbPath: str

get_all_table_coloumns(table_name)

Get coloumns of table return index, coloumn name and coloumn type

get_rows(tbname)

Get the rows in the database table.

Parameters:

tbname (str) – table name

Returns:

rows – table row list

Return type:

str

get_table_names(db)

Get table names from the database.

Parameters:

db (str) – The name of the database

Returns:

List of table names

Return type:

table_names (list)

insert_into_table_stmt_with_pk(data, tableName)

Insert values into table and create a primary key.

log: object

populate_table(data, tableName, key='')

populate a database table with values.

set_database_name(databaseName)

Set the name of the database.

Module contents