Register new data elements added to the system by the analyst
extract_delivery_from_script(env:DataIntakeEnv, folder:Path)
TODO:
[summary]
Parameters
env : DataIntakeEnv
[description]
folder : Path
[description]
Returns
str
[description]
register_data_group(env:DataIntakeEnv, folder:Path, group_type:Union[Delivery, Raw_Data, Dataset], record:Union[Delivery, Raw_Data, Dataset]=None)
Register a single data group to the database.
Parameters
env : DataIntakeEnv
The application environment settings.
folder : Path
The full path to the folder of the data group to register. Relative paths won't work.
group_type : str
The group type to put it in the proper table.
record : dm.Data_Group, optional
If this is a pre-existing record and you have it provide it here, by default None
Raises
TypeError
group_type must be one of ['delivery', 'raw_data', 'dataset']
register_data_folder(env:DataIntakeEnv, group_type:Union[Delivery, Raw_Data, Dataset], force:bool=False)
Scans a folder containing data groups and registers them if they
don't already exist in the database.
Parameters
env : DataIntakeEnv
The environment with data intake process pathnames.
group_type : str
The type of folder to scan, delivery, raw_data, or dataset.
force : bool, optional
If we force we ignore the current data and regenerate all stats. This will
overwrite previous stats.
Raises
ValueError
The right type must be passed.
register_all(env:DataIntakeEnv)
Register all new data groups in the data intake process environment.
This on purpose, ignores In_Progress and . files.
Parameters
env : DataIntakeEnv
The data intake process environment to scan and register new data groups.